Incident Response

Dealing with a security incident is difficult to do well, but easy to do badly. The headlines are filled with examples of bungled security incidents.

There’s the fudging: UK telco TalkTalk initially confused customers with conflicting statements after its 2015 breach, which saw it lose 157,000 customers’ financial details.

There’s the failure: The OPM’s mismanagement of a hack that eventually exposed the personal details of 22 million people is now legendary (and the Congressional report makes great reading).

There’s the finger pointing: Child monitoring app firm UKnowKids disparaged security researcher Chris Vickery after he notified them about a database of theirs serving up publicly accessible data.

Missteps like these don’t do companies any favours. How can they ensure that they handle security breaches more professionally? Here’s a short SecTor guide to building a robust incident response strategy.

More work needed

People aren’t happy with their current incident response capabilities. Interviewing 259 security pros in 2014, security training company SANS found that a quarter of them were dissatisfied with how their organizations tackled IR.

One of the biggest obstacles to good incident response was the lack of a formal team. 55% of security pros lamented this, and most of them dealt with the problem by drawing on IT staff as necessary. So creating a proper computer security incident response team (CSIRT) is a good place to start.

Security pros will tell you that a good incident response team goes beyond mere IT staff, trained in locking down compromised machines and unplugging networks. US-CERT identifies media relations staff, legal pros, and even marketing executives as part of the team. This is because an effective incident response plan goes beyond mere technical remediation.

The response plan is the next place to put your efforts. 43% of respondents said that they didn’t even have a formal IR plan, which tends to put a spanner in the works when a data breach hits. What does such a plan look like?

The National Institute of Standards and Technology (NIST) provides some excellent guidance on these issues. Its Computer Security Incident Handling Guide (Special publication 800-61) structures the response process into four main areas.


This involves collecting all the resources that you’ll otherwise spend time wishing you’d had when a breach occurs. These resources support the rest of the steps in the IR process.

Contact information and communications mechanisms for team members, along with incident reporting and tracking systems, will be key here. This also includes the tools to deal with event analysis, such as spare workstations to analyze malware samples and evidence-gathering equipment including portable printers.

Detection and Analysis

You can’t respond to what you can’t see. The detection and analysis phase highlights all potential attack vectors, and prioritizes what the risk management team deems most likely. These will include attacks arriving via removable media, web and email-borne compromise, and equipment theft, among others.

To properly enable detection and analysis, IR teams must document and then constantly look for signs of a security incident, which may range from web server log entries through to alerts from intrusion detection systems. It should also include manual detection, such as an admin noticing a unusual filename in an inappropriate folder. Think about how to formalise the noticing and communication of those signs.

Analyzing the incident is one of the most difficult parts of the response process, because there are typically so many of them. This is a process of selection; sorting the wheat from the chaff. Surfacing legitimate incidents and discarding false positives still entails manual processing for most CSIRTs. They must then prioritise legitimate incidents, as many of them won’t require further action.

NIST advises a mixture of security best practices here, including event correlation, along with system profiling to identify normal behaviour. This is something that vendors are increasingly pushing to automate, and you’ll hear the term machine learning bandied around a lot.

Containment, Eradication and Recovery

So, you’ve processed 1200 incidents in a single week, and found 50 of them that were legitimate. Almost all of them had simple explanations, like users storing files in the wrong place or violating usage policy, but they weren’t damaging. Five of them gave cause for further concern, and of those, one of them revealed the worst: some jerk has infiltrated your system with zero-day malware. Now what?

By this point, you should be documenting everything for further use in forensics and mitigation. You will have notified the CSIRT team, and now you’re into dealing with it. This is the ‘response’ part of incident response that most people identify with.

Your IR playbook should have containment strategies based on the type of incident you’re dealing with. You might react to ransomware differently than to a DDoS attack.

When developing those strategies, include legal staff in the conversation. They will help you determine whether it’s advisable to sandbox an attacker and monitor their behavior to gather more intelligence. Consider the effect of containment on service disruption, and whether you have prepared emergency workarounds.

Have a policy around gathering evidence, which again should involve the legal team. Gathering forensic evidence that can be used in court involves chains of custody and documentation that will keep it admissible should a case ever come to trial against the perpetrators. The company may also later have to use this data when reporting its own response process to regulators, or when defending itself against a customer or shareholder lawsuit.

The response team must also recover the infrastructure, which will likely happen in a phased process taking weeks or months, depending on the severity of the incident. Recovery methods are particularly important here, with a properly-prioritized and scheduled approach to system rebuilds, data restoration, changes to authentication credentials and other things needed to restore the status quo.

Post-Incident Activity

Finally, when everything is once again humming along smoothly, the CSIRT should look at what it can learn from the incident. Were there gaps or missteps in how they dealt with the breach? Did it surface new attack techniques or tools that should be factored into the company’s ongoing risk analysis?

Depending on the size and severity of the breach, a full report may be useful. This becomes part of the CSIRT’s knowledge base.

Creating an incident response plan may not be enough for an organization to truly be prepared, though. According to SANS, one of the primary problems for organizations tackling incident response was a lack of time to practice response procedures (62%). It specifically mentioned mock exercises, which could be a cue to introduce a red team capability, although ideally one with a purplish hue.

An organization must perfect many other disciplines to create a truly stellar incident response operation. Proper threat intelligence, combined with automation and SIEM integration, can help to harden a response strategy. But getting a solid team and a plan are your first steps.


Bookmark and Share