Philadelphia’s open data effort may be losing momentum, but OpenDataPhilly isn’t giving up

This article is a part of Every Voice, Every Vote, a collaborative project managed by the Lenfest Institute for Journalism with lead support from the William Penn Foundation, and additional funding from Lenfest, Comcast NBC Universal, the John S. and James L. Knight Foundation, Henry L. Kimelman Family Foundation, Judy and Peter Leone, Arctos Foundation, Wyncote Foundation, 25th Century Foundation, Dolfinger-McMahon Foundation and Philadelphia Health Partnership. This article was created independently of the project’s donors.

If you want to easily sift through years of Philly campaign finance data or see a map of all the nearest athletic fields, the city’s open data visualization tools make it easy. 

However, a quick dig around OpenDataPhilly, one of the platforms where the city publishes its data, shows some obvious weak spots. Some sources get updated regularly, while others like Police Advisory Commission complaints haven’t been touched in years. 

The city says it’s working to make the data updating process more seamless with automated systems and more dashboards to display information. The people behind OpenDataPhilly, a volunteer-run platform that collects open datasets across Philly into one place, say the open data ecosystem faces slowing enthusiasm from the city as administrations change. 

“There is essentially still a motivation for the city to maintain open datasets,” Robert Cheetham, “godfather of open data” and OpenDataPhilly volunteer, told Technical.ly. “But I don’t think there’s the same kind of energy, commitment [and] resources.”

The city established its open data program in 2012 after Mayor Michael Nutter issued an executive order. The initial goal was to publish city datasets in an open data portal, hire a chief data officer, establish an open data working group and appoint a data governance advisory board. The executive order also called for an open government plan. Within a year, the city achieved six out of 10 of its original goals. 

However, open data has not been as much of a priority for the last few mayors, Cheetham said, and the movement lost some of its energy. By 2019, some of these initiatives, like the OpenDataPhilly platform, remained strong, while others like the working groups and advisory boards had fizzled out. 

Lack of requirements and automation slow public releases

The city hasn’t given up on the program, though. It still has a chief data officer, Tim Haynes, and the program still has a system for updating its data with a focus on turning the data into digestible and usable resources, Kistine Carolan, senior program manager with the Office of Innovation and Technology (OIT), told Technical.ly.

Without a specific requirement for city departments to provide updates to data, though, there’s a lot of variance, despite years of progress. Frequency depends on the capacity of each city department, Carolan said. 

“The sharing of open data has become more of a habit for city departments as part of their process of having data that they’re using internally,” Carolan said. “That’s really exciting because often we have been approaching open data with a more holistic model of sharing the datasets themselves.”

Still, the system for updating open data isn’t as automated as it could be, she added. 

To combat this, OIT is in the process of moving its central data warehouse to a new version of data integration service, Data Bridge. This shift will hopefully lead to updated data at a faster pace, she said. Leaving more time for staff to work on projects that present that data in useful ways. 

“We’re pretty excited about that process both for allowing fresher data publicly, but also for freeing our staff to be able to engage in more projects like these dashboards or applications,” Carolan said. 

OIT helps make tools like the Campaign Finance Dashboard, which comes from the Board of Ethics and presents visualizations of data from campaign finance reports. The dashboard also links out to the metadata and the raw version of that data on OpenDataPhilly. 

Another use for the data is “finder apps,” which help people find resources around the city like primary care providers and recycling centers, Carolan said. The free meal finder app presents an interactive map where people can look up locations for food and meal sites throughout the city, for example. 

“It gives an immediate resource to people who just want to explore the data for general trends,” she said. 

From datasets to dashboards that the public can understand

Once OIT has updated data, staff publishes it to various platforms within the hour, Haynes, geographic information officer and chief data officer, told Technical.ly. The data is published through Data Bridge to two software platforms where OIT hosts APIs, ArcGIS Online and Carto. 

Those platforms are posted to the OpenDataPhilly website, the city’s metadata catalog, and the Philadelphia website along with its applications, Haynes said

From there, the data is available for use by students, journalists, researchers and other residents for projects on anything from voter turnout maps to sharing information about gun violence in the city.

OIT then works with city departments to turn their data into interactive dashboards and applications so it’s easier to understand. Having open data available is important to transparency and supports city residents working on private data-related projects, but also to improve internal operations and make city services accessible, Carolan said. 

“Creating an interactive dashboard where people can explore the data, particularly if there’s spatial information related to where they live or where they’re working, really allows a broader audience to engage with this data and use it in meaningful ways,” Carolan said. 

A focus on usability, not just transparency, for the project’s next upgrade

OIT put out a survey to residents in 2020 to get a better understanding of how people use the open datasets and what challenges they have. 

In response, it developed an open data dashboard in 2021 to track the metrics around datasets that have been shared publicly. 

“You’ll see a list of datasets and shows when they were last updated, and if there’s a visualization available,” Carolan said. “[It] helps people get a better sense of when something was last updated. That was one of the major pieces of feedback.” 

The dashboard currently says there have been 552 datasets released, but this includes multiple versions of previously released data. There are also 142 automated datasets and 179 datasets with visualizations. 

Overall, the city’s open data structure has become more “holistic,” Carolan said. Not only is the data available, but they are working to present it in a useful way that will hopefully improve resident experiences. 

“More and more we’re being approached by departments themselves who want to build these dashboards,” Carolan said. “They want to do that both to have transparency about their work, but also they are able to use it more.”