Civic News
Crime / Data / Hackathons / Municipal government

Philly Rap Sheet: web scraper shows new arrests in Philadelphia, made by Andrew McGill

Though for now the tool is dependent on court clerks uploading docket sheets, McGill, 23, said "they're pretty prompt with getting this information online after an arraignment."

Interested in who’s getting arrested in Philadelphia?
Then visit Philly Rap Sheet, “a web scraper that scans Philadelphia’s municipal court system every half hour for new arrests and posts them online,” says the developer, Andrew McGill. “You can filter by date, by bail amount, by crime, by the arresting officer and by the judge.”
Though for now the tool is dependent on court clerks uploading docket sheets, McGill, 23, said “they’re pretty prompt with getting this information online after an arraignment.”
McGill said the city courts online docket sheet database interface is fine for finding specific people but has always come up short in showing any more nuanced requests, like recent arrests or specific crimes on specific days.

For now, though a sure improvement, his side project is also short on features but he has plans for many to come, he said, like email alerts based on keywords or bail amount threshold. He’s also in the process of building an API that would allow third parties to query the data set, for options like an RSS feed, and is debating whether to move from its mySQL database to Google Fusion Tables for more open access. He recently added permalinks for individual arrests.

Andrew McGill

His clearest ask of the city courts to improve his tool is that more geographical information be included, like where the crime happened, “but you can’t have everything,” he said. His courts-based system also shows the divide between the legal system and the public safety agencies, which present their own data options.
The Penn State journalism graduate grew up in suburban Cheltenham and follows the local technology community, though he currently lives in Allentown working for that city’s daily newspaper the Morning Call.
“I’ve always had a passion for programming and web design, started out making computer games in BASIC in elementary school, am now versed in PHP/python etc.,” he said. “It’s a nice homecoming to be able to use those nerd skills in journalism.”
The project took three weeks of his free time, McGill said. The site gets about 1,000 new arrests a week, roughly 20 an hour during weekdays, McGill added.
“It endlessly frustrated me that while there was this great crime data out there, it wasn’t categorized in a way that made it easy to search,” McGill said. “By liberating this data, hopefully some meaning cross-referencing can be made and trends will emerge.”

Companies: Penn State

Before you go...

Please consider supporting to keep our independent journalism strong. Unlike most business-focused media outlets, we don’t have a paywall. Instead, we count on your personal and organizational support.

3 ways to support our work:
  • Contribute to the Journalism Fund. Charitable giving ensures our information remains free and accessible for residents to discover workforce programs and entrepreneurship pathways. This includes philanthropic grants and individual tax-deductible donations from readers like you.
  • Use our Preferred Partners. Our directory of vetted providers offers high-quality recommendations for services our readers need, and each referral supports our journalism.
  • Use our services. If you need entrepreneurs and tech leaders to buy your services, are seeking technologists to hire or want more professionals to know about your ecosystem, has the biggest and most engaged audience in the mid-Atlantic. We help companies tell their stories and answer big questions to meet and serve our community.
The journalism fund Preferred partners Our services

Join our growing Slack community

Join 5,000 tech professionals and entrepreneurs in our community Slack today!


Gopuff lays off 6% of workforce, as it prepares for 'next leg of growth'

5 local orgs with services and resources for startups and entrepreneurs

AI can now design greener cities, but architects still have the final say

Hopeworks to double Kensington student base with $1 million expansion in Philly

Technically Media