DER.2.3 Remediation of Extensive Security Incidents

Description

Introduction

Advanced Persistent Threats (APTs) are targeted cyber attacks on selected institutions and organizations. Attackers gain persistent access to a network and expand that access to further IT systems. These attacks are characterized by very high resource investment and extensive technical capabilities on the part of the attackers. Attacks of this kind are generally difficult to detect.

After an APT attack has been discovered, those responsible at the affected institutions face major challenges. They must carry out a remediation that goes beyond the standard approach to handling IT security incidents. It must be assumed that the discovered attackers have already had access to the affected IT infrastructure for an extended period. They also use complex attack tools to circumvent standard security mechanisms and establish numerous backdoors. In addition, there is the risk that the attackers are closely monitoring the compromised environment and will react to remediation attempts by covering their tracks and sabotaging the investigation.

This building block assumes a high threat level from a targeted attack by highly motivated individuals with above-average resources. In practice, it is customary to always engage (certified) forensic service providers in such incidents if the institution does not have its own forensic expertise. Forensic service providers are involved during the forensic analysis phase. However, they are also at least consulted in an advisory capacity during remediation.

Objective

This building block describes how an institution should proceed in order to clean up IT systems after an APT attack and restore the regular and secure operating state of the information network.

Scope and Modeling

The building block DER.2.3 Remediation of Extensive Security Incidents is to be applied whenever IT systems are to be cleaned up after an APT incident in order to restore the regular and secure operating state of an information network. The building block is to be applied to the information network.

An information network can only be remediated if the APT incident has previously been successfully detected and forensically analyzed. Detection and forensics, however, are not the subject of this building block. These topics are addressed in building blocks DER.1 Detection of Security-Relevant Events and DER.2.2 Precautions for IT Forensics.

The present building block deals exclusively with the remediation of APT incidents. Other incidents are addressed in building block DER.2.1 Security Incident Handling. The building block also does not describe how so-called Indicators of Compromise (IOCs) — traces of intrusion — are to be derived and how they can be used to detect recurring attackers. Nor does it address how backdoors that may have been overlooked during analysis and remediation can be found.

Only cyber attacks are considered. This means that attacks involving physical access to the information network, for example, are not examined. Attack forms such as breaking into data centers, bribing administrators, intercepting and manipulating newly acquired hardware, or intercepting electromagnetic radiation are not considered in this building block.

If forensic service providers clean up IT systems in whole or in part, the requirements of this building block also apply to those service providers. Contractual agreements and audits can be used to ensure that the service providers also comply with them (see OPS.2.3 Use of Outsourcing).

Threat Landscape

Since IT-Grundschutz building blocks cannot address individual information networks, typical scenarios are used to illustrate the threat landscape. The following specific threats and vulnerabilities are of particular importance for building block DER.2.3 Remediation of Extensive Security Incidents.

Incomplete Remediation

APT attackers typically want to infiltrate an information network on a permanent basis. They have the resources necessary to do so and are capable of conducting long-term attack campaigns. They use tools and methods tailored to their attack target. Even when an APT incident is discovered, it cannot be assumed that all access paths of the attackers have been found, all infections and communication channels of malicious software have been eliminated, and all backdoors have been removed. With an incomplete remediation, however, it is very likely that attackers will at a later point — for example, after a longer dormancy period — access the IT systems again and expand their access. They can do this, for example, by placing backdoors not only in operating systems and application software, but also in hardware-level components such as firmware. Such modifications are very difficult to identify, and the knowledge needed to extract and analyze them is not widely distributed. If those responsible attempt to clean up the IT components by overwriting or updating the firmware, for example, it can still happen that the attackers have also modified the update routines. In this way, they can regain access to the IT systems.

Destruction of Traces

After an APT incident, IT systems are often reinstalled or decommissioned entirely. However, if forensic copies of the IT systems were not created beforehand, traces may be destroyed that would be necessary for further investigation of the incident or even for legal proceedings.

Premature Alerting of Attackers

Typically, before remediating an APT incident, the attack is observed over an extended period and forensically analyzed to identify all access paths as well as the tools and methods used. If attackers notice during this phase that they have been discovered, they may resort to countermeasures. For example, they may attempt to cover their tracks, or they may sabotage additional IT systems. They could also temporarily abort the attack or set up further backdoors to resume the attack at a later time.

Since it must generally be assumed in an APT attack that the entire IT infrastructure of the institution has been compromised, the risk is high that the attackers will discover remediation activities. This is especially true when the compromised IT infrastructure is used to plan and coordinate the remediation. If the key steps of remediation do not take place in the correct order, or if critical measures are not carried out simultaneously and in a coordinated manner, the risk of alerting the attackers increases. For example, if those responsible isolate the network in stages rather than all at once, the attackers may be warned before their access is effectively terminated.

Data Loss and IT System Outages

During the remediation of an APT incident, various IT systems are reinstalled and networks are temporarily isolated. This inevitably results in IT system outages and services becoming only partially available or unavailable. If the remediation takes a very long time, the productivity of the institution may be lost. This can in turn lead to significant economic losses that can even threaten the existence of the institution. This is particularly the case when no or insufficient documentation is available for reconstruction.

Failure to Restructure the Network After an APT Attack

In an APT attack, the attackers gain detailed knowledge of how the target environment is structured and configured. For example, they know the existing network segments, naming schemes for IT systems, accounts, and the software and services in use. Through this knowledge, the same attackers may be able to gain access to the target environment again even after a remediation. They can move within the network in a very targeted, efficient, and unobtrusive manner, and can reach a high level of infection again in a short period of time.

Requirements

The following are the specific requirements of building block DER.2.3 Remediation of Extensive Security Incidents. The Information Security Officer (ISO) is responsible for ensuring that all requirements are met and verified in accordance with the established security concept. The ISO MUST always be involved in strategic decisions.

Additional roles are defined in the IT-Grundschutz Compendium. These should be filled insofar as this is reasonable and appropriate.

Exactly one role should be primarily responsible. In addition, there may be additional responsibilities. If one of these additional roles is primarily responsible for fulfilling a requirement, that role is listed in square brackets after the requirement heading. The use of singular or plural does not indicate how many people should fill these roles.

Basic Requirements

The following requirements MUST be fulfilled as a priority for this building block.

DER.2.3.A1 Establishment of a Steering Committee (B)

To remediate an APT incident, a steering committee MUST be established that plans, coordinates, and oversees all necessary activities. All authority required for the tasks MUST be delegated to the committee.

If such a steering committee was already established at the time the APT incident was detected and classified, the same committee SHOULD also plan and lead the remediation. If specialized forensic service providers have already been engaged to analyze the APT incident, they SHOULD also be involved in the remediation of the incident.

If the IT infrastructure is too severely compromised to continue operating, or if the necessary remediation measures are very extensive, it SHOULD be examined whether a crisis team should be established. In this case, the steering committee MUST oversee the remediation measures. The steering committee MUST then report to the crisis team.

DER.2.3.A2 Decision on a Remediation Strategy (B)

Before an APT incident is actually remediated, the steering committee MUST establish a remediation strategy. In particular, it MUST be decided whether malicious software can be removed from compromised IT systems, whether IT systems must be reinstalled, or whether IT systems including the hardware should be completely replaced. Furthermore, it MUST be determined which IT systems will be remediated. The basis for these decisions MUST be the results of a previously conducted forensic investigation.

All affected IT systems SHOULD be reinstalled. The institution’s recovery plans MUST then be used. Before backups are restored, however, forensic investigations MUST ensure that no manipulated data or programs will be transferred to the newly installed IT system.

If an institution decides against reinstalling all IT systems, targeted APT remediation MUST be implemented. To minimize the risk of overlooked backdoors, the IT systems MUST be specifically monitored after remediation to determine whether they are still communicating with the attackers.

DER.2.3.A3 Isolation of Affected Network Segments (B)

The network segments affected by an APT incident MUST be completely isolated (cut-off). In particular, the affected network segments MUST be disconnected from the internet. To effectively lock out the attackers and prevent them from covering their tracks or sabotaging additional IT systems, the network segments MUST be isolated all at once.

Which network segments must be isolated MUST be determined in advance by a forensic analysis. All affected segments MUST be identified in the process. If this cannot be ensured, all suspected and even theoretically infected network segments MUST be isolated.

To effectively isolate network segments, all local internet connections — such as additional DSL connections in individual subnets — MUST be identified as completely as possible and also taken into account.

DER.2.3.A4 Blocking and Changing Credentials and Cryptographic Keys (B)

All credentials MUST be changed after the network has been isolated. Additionally, centrally managed credentials MUST also be reset — for example, in Active Directory environments or when the Lightweight Directory Access Protocol (LDAP) is used.

If the central authentication server (domain controller or LDAP server) has been compromised, all credentials stored there MUST be blocked and their passwords replaced. This MUST be implemented by experienced administrators, using internal or external forensic expertise if necessary.

If TLS keys or an internal Certification Authority (CA) have been compromised by the APT attack, the corresponding keys, certificates, and infrastructure MUST be regenerated and redistributed. The compromised keys and certificates MUST also be reliably revoked and withdrawn.

DER.2.3.A5 Closing the Initial Entry Point (B)

If a forensic investigation has determined that the attackers penetrated the institution’s network through a technical vulnerability, this vulnerability MUST be closed. If the attackers were able to compromise IT systems through human error, organizational, personnel, and technical measures MUST be taken to prevent similar incidents in the future.

DER.2.3.A6 Return to Production Operations (B)

After the network has been successfully remediated, the IT systems MUST be returned to production operations in an orderly manner. All IT systems and installed programs previously used to observe and analyze the attack MUST either be removed or transferred to production operations. The same MUST be done with communication and collaboration systems that were procured for the remediation. Evidence and decommissioned IT systems MUST either be securely deleted or destroyed, or appropriately archived.

Standard Requirements

Together with the basic requirements, the following requirements represent the state of the art for this building block. They SHOULD generally be fulfilled.

DER.2.3.A7 Targeted System Hardening (S)

After an APT attack, all affected IT systems SHOULD be hardened. The results of forensic investigations SHOULD serve as the basis for this. In addition, it SHOULD be re-examined whether the affected environment is still secure.

Where possible, IT systems SHOULD be hardened during the remediation process. Measures that cannot be implemented in the short term SHOULD be included in an action plan and implemented in the medium term. The ISO SHOULD compile the plan and verify that it has been correctly implemented.

DER.2.3.A8 Establishment of Secure, Independent Communication Channels (S)

Secure communication channels SHOULD be established for the steering committee and the employees tasked with remediation. If third-party communication services are used, it SHOULD also be ensured that a secure communication channel is selected.

Requirements for High Protection Needs

The following are exemplary proposals for requirements that go beyond the level of protection corresponding to the state of the art for this building block. The proposals SHOULD be considered when there are elevated protection needs. The specific determination is made within the framework of an individual risk analysis.

DER.2.3.A9 Hardware Replacement of Affected IT Systems (H)

After an APT incident, it SHOULD be considered replacing the hardware entirely. Even if suspicious behavior is still observed on individual IT systems after remediation, the affected IT systems SHOULD be replaced.

DER.2.3.A10 Structural Changes to Make a Renewed Attack by the Same Attackers More Difficult (H)

To prevent the same attackers from carrying out another APT attack on the institution’s IT systems, the internal structure of the network environment SHOULD be changed. In addition, mechanisms SHOULD be established that allow recurring attackers to be quickly detected.

Additional Information

Good to Know

The BSI has published the following documents on the subject of APT:

• Advanced Persistent Threats - Part 4 Response - Technical and Organizational Measures for Incident Handling
• Common Criteria Protection Profile for Remote-Controlled Browsers Systems (ReCoBS): BSI-PP-0040

CERT-EU has published the supplementary document “CERT-EU Security Whitepaper 2014-007: Kerberos Golden Ticket Protection: Mitigating Pass-the-Ticket on Active Directory” on the subject of APT.