Software Development
Data / Delaware / Delaware Innovation Week / Municipal government /

Hey, DE governments: this is why local devs are going to start scraping your websites

Former City of Philadelphia Chief Data Officer Mark Headd told a room full of developers that the best way to make their governments adapt more quickly to modern open data trends is to challenge them with action.

Former City of Philadelphia Chief Data Officer Mark Headd speaking at Delaware Innovation Week. (Photo by Christopher Wink)
Making data open can mean different things to different people. Mark Headd says Delaware needs to expect more.

Shaved bald, with thick rimmed glasses and a hooded sweatshirt, he is a recognizable face to many who follow national open data and open government conversations. Headd is the fiery former embattled Chief Data Officer of the City of Philadelphia, past executive with Code for America and self-identified civil servant. He spent 10 years living in Wilmington while serving an array of Delaware state government jobs beginning in 1999. Now he lives in upstate New York, while traveling frequently to conferences and tech events to evangelize for Accela, a Silicon Valley government IT services company.
He was back in Delaware for the first time in several years last week to speak at our Delaware Innovation Week Dev Talks, which brought nearly 70 technologists to the coIN Loft in Wilmington.
“So I wanted to check in on how things were going,” Headd said of an open data movement that has really only started in earnest here in the last six months. “I self-identify as a civil servant. There are people in all governments working really hard to serve citizens in lots of ways. But we need better here.”
To show that, Headd took a look at the State of Delaware’s Food Establishment Inspection Records. Though some might say that a government site that lists such records could constitute open data, Headd said that it is practically unusable. It is clunky, hard to understand and lacks any useful structure — like geolocation to map the offenders or “fuzzy search” by restaurant name. There is only a first-letter alphabetized order.
“This bad version isn’t helping anyone,” said Headd. The state doesn’t get help enforcing records by putting pressure on restaurants with poor records, restaurant goers don’t get the benefit of being able to easily identify problem locales and the restaurants that are following state law don’t get the benefit of being able to be discovered for a strong track record.
So in a 20-minute case study, he showed how quickly something better could be created. He scraped the government site (“throwaway work” that all experienced developers hate doing, he said) and used Google’s OpenRefine to add longitude and latitude data to a spreadsheet that had the inspected locations. Then Headd was able to use geolocation service CartoDB to create a simple map of the locations.

Here's the simple map Mark Headd created of inspected restaurants according to state data.

Here’s the simple map Mark Headd created of inspected restaurants according to state data.

“A lot of people in government identify as the ones who build the apps. Open data is pushing them to see they are the enablers of apps,” said Headd. It’s a concept that Wilmington Office of Economic Development Director Jeff Flynn is trying to build a constituency around.
“We need this,” said Flynn, who spoke to Headd after the presentation but was not endorsing Headd’s approach.
The “this” is something that’s happening across the country — tech communities pushing their governments to supply the marketplace with data and information in formats that can be easily manipulated in new and creative ways (think .CSV files and APIs, not proprietary websites or text documents).
“I am not just calling out the state of Delaware,” said Headd. “Often, there is still not an awareness that this is good for government.”
So what could make the Food Inspections website better?
“The data is clearly already in a database,” said Headd. “Render it as a .CSV and allow us to download the whole file or, even better, put an API in front of it so an application we build using that data stays accurate.”
Look at Brixsy in Philadelphia as an example of a solution coming into the market because of shared data, Headd said. What can developers do to push government to modernize their data releases?
“Go scrape some data from the state,” said Headd, of using automated tools to pull down information. “There isn’t a more effective way to push open data forward than scrape it and share it. Then tell government: ‘I’ll stop scraping it when you start sharing it.'”

Before you go...

Please consider supporting to keep our independent journalism strong. Unlike most business-focused media outlets, we don’t have a paywall. Instead, we count on your personal and organizational support.

3 ways to support our work:
  • Contribute to the Journalism Fund. Charitable giving ensures our information remains free and accessible for residents to discover workforce programs and entrepreneurship pathways. This includes philanthropic grants and individual tax-deductible donations from readers like you.
  • Use our Preferred Partners. Our directory of vetted providers offers high-quality recommendations for services our readers need, and each referral supports our journalism.
  • Use our services. If you need entrepreneurs and tech leaders to buy your services, are seeking technologists to hire or want more professionals to know about your ecosystem, has the biggest and most engaged audience in the mid-Atlantic. We help companies tell their stories and answer big questions to meet and serve our community.
The journalism fund Preferred partners Our services

Join our growing Slack community

Join 5,000 tech professionals and entrepreneurs in our community Slack today!


After shutdown threat, transformative Wilmington art space finds a new home

Drug tested in Delaware? A new medical marijuana law may protect you from getting fired

He started at Neya as an intern. 10 years later, he’s director of robotics — and loving life

Often overlooked, middle schoolers get their own STEAM job fair to jumpstart their careers

Technically Media