Where’s the data? A ten year old problem, city CTO says

For all the city information technology issues that Allan Frank addresses daily, its perhaps the availability of transparent city data that plagues him most publicly.
Sure, IT consolidation efforts mandated by mayoral executive order — which have transformed the Division of Technology from an agency once one-fifth the size it is now — have been a priority for Frank, the city’s Chief Technology Officer.
But often, the cry from the city’s industrious technology community has been one caused by a national intrigue in government transparency that tech can facilitate.
Cities like New York — which opened an impressive amount of city datasets for public use, and sponsored a $20,000 contest to attract software developers to create interesting technology applications and web apps — are pressing ahead with new data initiatives.
But Philadelphia lags behind. The city’s first big data win came when SEPTA released raw data around station geolocation and schedules, well after developers took their own stab at collecting data — by scraping HTML pages. Since, we’ve seen little movement from either developers or the city.
When we first covered Frank in May last year, he spoke before a crowd of Refresh Philly attendees and gave them a charge to come up with data they wanted. The effort dwindled, due in part to a lack of movement in the community and too, on actionable steps from the city.
Now, as Frank enters his first fiscal year with a serious $120 million capital investment in city technology, we’re wondering what’s next.
Late last week, we met with the CTO to discuss problems plaguing the department around opening those datasets and followed-up with Frank about how things have been for the last year, his first in public office. After the jump, that conversation.

Our community wants to understand why Philadelphia is so far behind when it comes to the availability of transparent city data for use by third-party developers.
It’s pretty hard to give you access to data if I don’t have a platform to give you data on, if I don’t have the infrastructure. Of the [$120 million in IT capital], one big chunk of it is to completely rebuild Phila.gov, and on the inside, the city’s intranet Citynet. Really, at its core, build a whole new kind of web platform that both sides use. Underneath it, we’re plugging in architecturally how we’re going to provide access transparently to city data.
We have such a large amount of remedial work to do and I feel through large measure, we’re catching up over a 10-year period. We have so much legacy [infrastructure]. It doesn’t do me any good if data is in Access databases. It’s not shareable. I’m not converting every database in the city. It’s not uncommon — I won’t name specific departments — to have 500 or 1,000 Access database datasets. What do you do with that? My point about infrastructure, I really need to create an environment that’s leveragable.
Everybody is like ‘why cant we do it yet?’ Besides that there so many moving parts, now there is an interesting set of questions we are asking ourselves. How do we leverage the broader volunteer web and open-source community to give us the best ideas around Phila.gov and how we make it accessible to our citizens. Now that I have the capital and I’m building the back-end stuff, I can now work with an open-source group to say, ‘what standards? How do we want to do this?’
What’s the timeline for this?
Now that I’m in fiscal year 2011, we have a whole separate group which is rearchitecting the application and data infrastructure for the city. Right now, we’re writing RFPs and doing things we need to do around actually standing up and building components of the architecture. We’re settling on our technology architecture for the web—I plan on using Alfresco for content store. We have Sharepoint we’re leveraging for a .NET framework and we’re connecting the two. We’re building a data engine that can be used both fo the public web and intranet.
Of the city’s 52 agencies, who’s on board? Who has data that is already accessible?
What I want to do is target low-hanging fruit first and say ‘what are the data sets that we can make available now.’ We’re looking at building a generic engine we can use for data access, but that doesn’t mean we cant do a one-off pilot with a specific data set so we can start experimenting.
When will we see the first dataset available to the public?
My goal would be over the next six months. Only reason I say six months is because I’m focused on building an engine and focusing on the intranet side of the data. Over the next month, I’m going to be booting up the [web community] group, to help me define input from the outside. Inside, the internal data group, I think the goals are different.
Congratulations on landing $6.4 million in broadband stimulus funding for public computer centers. A year ago, you were applying for the first round, and ultimately lost. What made this round work?
[After the first round], we went to the state and we got their input. And this time, DOT supported the submission of three proposals again, but it was really just this one that the city was [helping author]. The public computer center piece was something that as I got input from the last round, I got a sense that we would have strength. Second, it really addresses the issue of digital divide.
It’s been a little more than a year since we started covering you. How are you gauging your success and failure?
I think about that constantly. I came into city government bright-eyed, let’s call it naïve. I had never been a part of city government, having been involved in the private sector. I’ve done consulting to government but there’s nothing like being in the middle. You have to say, that in many ways, it’s taking longer to accomplish certain things that I took for granted. When I talk about building those databases, I’m very frustrated with the progress of being able to roll out a core platform. Now I think I’ve got there.
But it’s also a great accomplishment that the mayor now has fully embraced a major technology investment in the city. He fully gets it.
I would have to say I’m dissapointed in not being able to create more magic sooner. My impacts aren’t as readily visibile as I’d like to see, to have to come in and take the little money we had during the budget crisis. Remember, we now are merged. Before, I was less than one-fifth of the total IT in the city, so my ability to get things done has increased. So the ability to get $120 million and to be able to reinvest my savings into IT and create a whole new governing structure, I give myself pretty good grades on that, an “A” or an “A-minus.”
How would you grade your progress on other fronts?
I would have to say that in being able to develop external applications and things like that, I would give myself, if I was allowed to curve it, I give myself a “B.” You just can’t move as quickly as a the city. But, our business services portal is moving and as we get through this calendar year, I expect to be able to do full online licences and permits. We’re building a whole new paradigm for how you do business.
How have your conversations with the mayoral cabinet changed over the last year?
Being at the table has proved the point. There’s a voice at the table that sees the world with technology glasses on. Whether we like it or not, [technology] has a tremendous societal impact.
Every Friday, Technically Philly brings you an interview with a leader or innovator in Philadelphia s technology community. See others here.