Civic News
Budgets / Cybersecurity / Municipal government / Technical.ly

As Baltimore rebuilds from 2019 ransomware attack, is $10 million for a cure better than prevention?

Baltimore city's own documents, sourced by Technical.ly through a FOIA request, paint a picture of how and with who the city worked to rebuild. Local cyber contractors give insight on what that $10 million for cybersecurity actually buys.

Baltimore City Hall. (Photo by Flickr user David Kirsch, used under a Creative Commons license)

This editorial article is a part of Cybersecurity Month of Technical.ly's 2022 editorial calendar.

What does $10 million, committed for curing a city’s ransomware affliction, actually buy you?

AJ Nash, VP of intelligence at cybersecurity company ZeroFOX, says it depends on how you want to handle the situation. Firms can negotiate the ransom. For Baltimore, trying to recover from the devastating 2019 ransomware attack, that ransom was $76,000 worth of Bitcoin.

Beyond these cyber companies, intelligence organizations can help a victim better understand the cyberattack and the organization claiming responsibility. Contractors can come in and attempt to decrypt data, as well as restore backups and networks.

These are all specialties and there are very few organizations, if any, that can provide all the different services that are necessary during an attack. Perhaps that helps explain why the city of Baltimore ultimately spent $10 million purely on IT recovery.

Documents that Technical.ly obtained from Baltimore’s city government, via a FOIA request, paint a picture of how this money was spent — and who it was spent on — if not exactly how much each entity received.

For instance, on May 24, 2019, the city bought a month’s worth of IT support from Trigyn Technologies. From SecuLore Solutions of Odenton, Maryland, the city purchased cyber forensics and monitoring assistance, as well as security software. IT solutions company Quest sold the city consultation with archive managers. Microsoft sold it consultation and implementation of active directory remediation and hardening, which included the billable hours of a solutions architect, senior consultant, senior project manager and account delivery executive. FireEye, doing business through Mandiant, was contracted by the city as a security consultant and engineer. California-based Dyntek was hired to rebuild Microsoft products. A consultant from Glen Burnie, Maryland-based Skyline Technology Solutions was contracted to act as the city’s Chief Information Security Officer and, after Todd Carter was promoted, acted as director and deputy director of infrastructure.

Regardless of how all this contracted work shook out, COO Terry Bazemore Jr. of cybersecurity firm Ey3 Technologies noted that cyber companies are also, in a sense, being paid so clients can have a certain sense of coverage.

“As a cybersecurity company, you come in as either the hero or the scapegoat,” he said. “I’m coming in to help you but let’s say, while we’re in the process of helping this company, something goes wrong on their end, or something happens later. Now they can say, ‘Hey, well, we had this company come in to do this work.’ What you’re paying for is also coverage for yourself as an organization.”

The cost of all these contractors may explain why cities pay $20 million in cybersecurity insurance. Otherwise, the full expense comes out of the taxpayers’ pockets. Instead of paying a $569,000 premium to pay the contractors or the ransom, a city pays millions.

“It’s a challenge. What’s the right amount of money to spend? What’s the right amount of prevention to have,” Nash said. “The challenge in cybersecurity is: Prevention is hard, sometimes, to document what the value was. The value comes after something bad happens.”

Another option, and why it’s not often used

Cities could theoretically hire internal cybersecurity professionals. This raises another problem, however: Cities cannot always offer salaries that compete with what the private sector offers. Baltimore’s former CIO Frank Johnson in 2019 had the city government’s highest salary at $250,000 a year, according to The Baltimore Sun. As the highest-paid employee, that’s still less than the median salary for a CIO throughout the US. It’s no secret that you make can more in the private sector than in the public one.

The city IT budget skyrocketed after the 2019 ransomware attack without actually leading to an increase in positions — a detail that city councilmember Eric Costello noted during an FY2021 city budget meeting at which the Baltimore City Information & Technology Department (BCIT) proposed its plans.

[youtube https://www.youtube.com/watch?v=w9Z73_Fn9do?start=489]

That BCIT proposal was actually trying to stabilize the budget over the next few years. That’s estimated to be around $5 million in FY22, which is higher than before the ransomware attack but about half the budget now.

Along with that budget, the city’s philosophy IT security has changed under new Chief Information Security Officer Kevin Kearney. As he told Technical.ly, security must be an integral and early part of IT planning.

“Security is less effective and often costs more to try and bolt on later,” he said. “The city has been investing in IT improvements to support the mayor’s pillars and security is being integrated into all of our IT projects.”

Nash is similarly proactive about cybersecurity and estimates a cost of about  $3 or $4 million to establish a decent cyber intelligence organization. To keep it running still costs a couple of million per year.

“If you spend a lot of money year over year and bad things don’t happen, there’s a tendency to try to cut that budget,” Nash said. “‘Maybe we’re overspending here. We don’t really know what we’re defending.’ Until something bad happens. Then you fire a couple people. Spend a whole bunch of money on people and new technologies. We see that happen a lot of times [in both the]private sector and public sector.”

Kearney also confirmed that the city has an incident response plan, and all city agencies are required to update their continuity-of-operation plans this year.

“I joined the city after the ransomware attack, but in talking to leaders and employees throughout the city, the most common feedback I’ve heard is that communication during the incident could have been better,” Kearney recalled. “Employees and leadership sometimes felt like they didn’t have the information they needed. When we updated our incident response plan, that was an area that received a lot of attention.”

A longterm shift, and a persistent question

The change in mentality about cybersecurity is great and to be expected after an attack that shut down city services for a month. But memories fade as time goes on, and the budget always reflects a city’s current priorities.

“[Victimized organizations] still have a mentality that security is important to get it fixed but not important to pay for,” Bazemore said. “They still oftentimes think $50 an hour should cover that. If you understand the caliber of people that have to come in and clean this up, triage and understand the advanced nature of the individuals that hit you. We’re going to have to bring in some really advanced people.”

Nash comes from an NSA and US Air Force background. Bazemore built his cybersecurity toolkit with Lockheed Martin a little over a decade ago and has worked with Maryland cybersecurity firms ever since. Both of them have specialties under the IT umbrella that demand top dollars, and the market reflects their importance: Of the 1,500 senior tech managers that Robert Half Technology recently surveyed, 30% say that security, privacy and compliance professionals are among the most challenging to find.

“In demand” and “hard to find” means “expensive to hire,” sure, but it’s also expensive to hire consultants after the fact. The question for cities and businesses big and small remains: Is an ounce of prevention worth a pound of the cure?

Donte Kirby is a 2020-2022 corps member for Report for America, an initiative of The Groundtruth Project that pairs young journalists with local newsrooms. This position is supported by the Robert W. Deutsch Foundation.
Companies: ZeroFOX / City of Baltimore
Series: Cybersecurity Month 2022

Before you go...

Please consider supporting Technical.ly to keep our independent journalism strong. Unlike most business-focused media outlets, we don’t have a paywall. Instead, we count on your personal and organizational support.

3 ways to support our work:
  • Contribute to the Journalism Fund. Charitable giving ensures our information remains free and accessible for residents to discover workforce programs and entrepreneurship pathways. This includes philanthropic grants and individual tax-deductible donations from readers like you.
  • Use our Preferred Partners. Our directory of vetted providers offers high-quality recommendations for services our readers need, and each referral supports our journalism.
  • Use our services. If you need entrepreneurs and tech leaders to buy your services, are seeking technologists to hire or want more professionals to know about your ecosystem, Technical.ly has the biggest and most engaged audience in the mid-Atlantic. We help companies tell their stories and answer big questions to meet and serve our community.
The journalism fund Preferred partners Our services
Engagement

Join our growing Slack community

Join 5,000 tech professionals and entrepreneurs in our community Slack today!

Trending

Major state funding boost means more Maryland college students can get tech internships

Cal Ripken Jr. essay: The MLB legend explains his drive to build STEM centers in schools across the nation

The end of software as technology

From quantum to biotech, meet this year’s Maryland Tech Council ICON nominees

Technically Media