Enterprises and Service Providers today sure do have their work cut out for them. Security work. They deal with a lot of data, much of it being confidential. With all the looming inside and outside threanetworkts, it is paramount and in everyone's interest to keep this valuable and confidential data secure.
IT organizations need to have a deep understanding of their organization's flow of data, both internally and externally. Minimizing risk is the name of the game nowadays, as completely eliminating the threat of a myriad of network violations, for the time being, is a work in progress.
In this post we look at the possible thought process and actions involved in investigating data leaks for a fictitious service provider, an energy utility in the New York area. Let’s refer to them in this article as PWRState. The utility collects and stores personal data for a large metropolitan area on an SDN-based Network. This data includes everything from credit card numbers customers can choose to keep with the utility for quick pay transactions to less confidential info such as phone numbers and addresses. Although the names and characters in this post are fictitious, the story presented may well represent real hypothetical scenarios.
PWRState experienced an attack that resulted in a data leak, one that exposed information belonging to customers from a certain section of a large metropolitan area of NY, comprising about half a million residents — a rather large breach.
A phone call came in a day earlier to Neil, the CSO’s office from a blocked number. The disguised voice on the other end claimed they had hacked into the system. He proved this fact by telling Neil the password he used to the main server, the server that held a portion of confidential information such as credit cards.
The voice said he was holding the data for ransom. He gave Neil a figure of 1,000,000 dollars to not publish the information. So, now PWRState was in a race to see what data was actually leaked to validate the hacker’s claim and assess the damage.
While trying to prevent a security breach is a constant responsibility for any CSO, investigating one is a whole other ball game, one that requires a whole other set of tools, resources and a completely different mindset.
For this reason, Neil unhesitatingly tasked Sam, his veteran Security Systems Engineer, and Ned, his rather new Network System Engineer, to investigate the security incident (All names used here are alias names used for the sake of confidentiality).
Investigating security leaks promptly is of course very important in order to constantly strive to protect customer data. It is all the more important for protecting corporate stakeholders as an investigation from a higher authority, such as the justice department or FBI, may hold corporate stakeholders accountable. This could mean jail time due to negligence.
Here’s the list of questions that guided PWRState as they investigated the security leak, and you can use this list as a general guideline:
Luckily for PWRState, the company had implemented an advanced network visibility layer which allowed them to look at traffic history and event logs, and could allow them to answer the above questions. They used their network recording tools to see what the makeup of the traffic looked like over a period of time, and to identify which traffic looked like it was deviating from the norm. They then started analyzing that traffic to pinpoint exactly which machine was the one that was penetrated.
Their network recording technology gave them an inside view into the network happenings down to the finest detail. The recording technology was vital in showing the entire process that lead to the data leak. Solely monitoring traffic without recording it is kind of like looking at crime scene fingerprints and broken door handles rather than looking at a surveillance video.
A network packet broker allows you to groom and choose the parts of your traffic that need to be recorded (because they are more at risk in the event of of an attack) and send only that part to the recording devices.
By implementing the packet broker across the network, Ned and Sam from PWRState were able to balance their data load and filter out what data was not needed to record by source or traffic type. By monitoring and recording link and port-specific traffic, the recording solution becomes much more cost effective, and also investigating the traffic history becomes a lot more manageable.
Let’s take the actual scenario to see how the packet broker was used:
PWRState kept the data that was leaked on an internal database server that was used by the customer service and billing departments. The applications with access to this Database were considered to be secure and were not connected to the Internet.
The connected Customer Account Management (CAM) web server had access to a different database server that was supposed to contain only non-sensitive data.
Web application developers had created a script to copy updated records from the first Database to the second one, which was stored directly on the CAM server. They did this in an insecure way, storing the password to the customer service/billing database server directly in the script.
The Attacker (Later identified as a Russian “Advanced Persistent Threat” (APT) team) used a well known Apache Struts vulnerability to gain entry to the CAM server. Their reconnaissance team identified the script mentioned above, and the password contained therein.
Russian APT team exfiltrated the data and used it to gain access to the server
Review of forensic data, including recorded network traffic and server logs (backup copies of the log, the attacker "cleaned up" the original logs) showed the initial probes against the CAM server, with an IP address located outside of Moscow. Further investigation found traces of unauthorized activity and a modified user account on the CAM server.
Additionally large amounts of SQL traffic were identified going between the CAM server and the "secure" database at odd hours when customer traffic was very light. Finally, large bulk data transfers were identified, originating from the CAM server and terminating at a host who's IP address was in Moldova. It is assumed that this host is controlled by the APT.
Because the Network Packet Broker was configured to copy the data that was coming through the SQL and TCP port and streamed to the network analysis tool, the operator was able to see the trends in the data that was allegedly hijacked, saw that it was off a server that stored confidential data, but that this data was not leaked, as it had another set of security codes in order for it to be accessed. The team was able to assess that it was just phone numbers that were leaked..Thus, saving the company the ransom amount and allowing them to avoid a PR nightmare.
PWRState Board of directors hail both Sam and Ned as heroes for the way they configured the Network Packet Brokers to record the data and identify that the traffic hijacked only included phone numbers, not confidential data as the hacker originally claimed.
Regardless, that confidential data was not involved, Neil, the CSO fired the Web App team lead for being negligent and allowing the password to be stored in the script.
Remember, no enterprise or service provider, no matter how large or small is immune to a data breach. You need to ensure the correct infrastructure is in place so that when the day comes, that you have to deal with a data breach, you will have “the footage” in place to allow you to investigate effectively and react immediately.
Want to learn more about maximizing visibility into your network? Sign up to schedule a consultation session with a Niagara network visibility expert.