It’s rare that my home city of Baltimore makes the national news for anything other than violence. But when Baltimore City computer systems were hit with a ransomware attack at the beginning of May, it made headlines around the country. And they’re still trying to recover.
The city notified the FBI immediately of the attack once it was discovered, but much damage had already been done. Among the systems taken down were voice mail, email, a parking fines database, and a system used to pay water bills, property taxes and vehicle citations.
As a Baltimore County resident & business, I’m not as heavily affected as those living inside the city. But, even county residents depend on the city for water & sewer service. And while sewer service is built into our county property tax, water bills come directly from, and are paid to, the city. As of right now, this is what the payment link for water bills looks like on Baltimore City’s website:
The city is currently accepting payments by mail, I’m curious as to whether the bills we receive will actually be accurate.
Currently the city has a FAQ online where one can go to find additional information.
How Did The Baltimore City Ransomware Attack Happen?
The City has been rather tight-lipped about the specifics of where the ransomware infection originated, citing the ongoing FBI investigation into the matter. However, it is known that the infection made use of a relatively new malicious software known as “RobbinHood”.
RobbinHood differs from other ransomware applications in that it’s not usually spread by email spam.
This ransomware is not being distributed through spam but rather through other methods, which could include hacked remote desktop services or other Trojans that provide access to the attackers.Lawrence Abrams – BleepingComputer: A Closer Look at the RobbinHood Ransomware
The ransomware uses a level of encryption that has yet to be broken; there is no way to recover the decryption key without paying the ransom.
Since the city has been very quiet about how the infection took place, I can only speculate that either a trojan was accidentally installed (which could have come from email spam), or a system with Remote Desktop Services enabled was exposed directly to the internet.
Why is the Recovery Taking So Long?
This is probably a bigger question, and much more important than how the infection occurred. As an IT professional who has organized the recovery of a business affected by ransomware, I have found there are 3 key steps to recovering from a ransomware attack.
1. Identify & Disconnect Affected Systems
Once you realize you’re in the midst of a ransomware attack, it’s critical to determine which systems are affected and get them disconnected from the network as soon as possible. While the RobbinHood program doesn’t seem to affect network resources, other ransomware programs such as WannaCry will. Disconnecting the infected systems will help to minimize the damage.
Time is of the essence when it comes to a ransomware attack. Encrypting files takes time and CPU cycles, so the sooner you act, the better.
2. Evaluate Your Losses
Once you’ve stopped the attack by removing all the affected systems, you must catalog what has been lost. Which servers did you have to remove? Was network data encrypted? Which files & directories were involved?
Once you catalog these items, you can prioritize which items need to be brought back up first. For most companies or organizations, the immediate priority will be to ensure that your recovery systems are brought back first. For instance, if you lost a domain controller, restoring that will make the rest of the jobs easier.
This brings us to the last step in the process, which happens to be the most important:
3. Have a Solid System Backup Procedure in Place
The reality is that malicious attacks happen and no system is 100% secure. So the most important piece of dealing with a ransomware attack, including the one that hit Baltimore City, is having recent and reliable backups of all your systems and data.
Assume that a system hit with ransomware (once disconnected from the network) is no different than a smashed computer. It’s useless and probably should never be used again. The only way to replace it, would be to have a good backup of that computer to deploy in place of the infected one – if you don’t have a good backups, you’ll need to rebuild the system from scratch which will cost considerable time and money.
Baltimore City’s Slow Recovery
As the recovery wanes past its second month, cyber security professionals can’t help but point out that in many cases, servers can be restored from backup in a matter of hours. Given the slow recovery and the price tag of over $18 million, it’s my professional opinion that Baltimore City didn’t have a solid set of backups for the systems affected by the ransomware attack.
The slow recovery times indicate that Baltimore City IT professionals are likely rebuilding the lost systems from scratch rather than restoring them from backups. Perhaps a good backup system wasn’t in place or the backup files themselves were damaged.
I love my city. But just like myriads of businesses and organizations before them, Baltimore seems to have been a victim of poor IT planning & funding. This ultimately resulted in a system that wasn’t suited to properly recover from a major disaster regardless of whether that be a ransomware attack or some other failure (data center fire etc…).
In a world where IT decisions are frequently made by non-IT professionals, the frustration for those of us who understand how these systems work is high.
Even the Governor of Maryland got involved stating that he was going to be creating a cybersecurity council to help protect the state of Maryland, which includes Baltimore City. The problem? His proposed cybersecurity council doesn’t include plans for any cybersecurity personnel – just governors & cabinet members.
Let’s hope the State of Maryland doesn’t get hit with ransomware anytime soon. Until cybersecurity experts get to actually make the decisions, what’s happening with the Baltimore City ransomware attack will continue to be repeated over and over by government agencies and corporations.
Whether it’s a government or a business, the most qualified people to deal with cybersecurity concerns are those who have been trained to do so. Politicians and CEOs should have limited control at deciding what’s best in regards to digital data. Baltimore’s lag in getting back up and running after a cyber security incident is simply a symptom of this larger problem in the industry.
Sources & Additional Reading
1. The New York Times: Hackers are Holding Baltimore Hostage: How They Struck and What’s Next
2. BleepingComputer: A Closer Look at the RobbinHood Ransomware
3. WJZ CBS Baltimore: Gov. Hogan Signs Executive Order To Strengthen Maryland’s Defense Against Cyberattacks