Threat Emulation vs. Penetration Testing: Understanding the Differences

This was originally posted as an Insight from Wolf & Company, P.C.

Threat emulation (also referred to as adversary simulation) is an advanced form of testing where the attack tools, techniques, and procedures (TTPs) utilized are based on documented, real-world criminal actions. These tests are best suited for environments and organizations that have the basics covered and are seeking validation of their network security posture against realistic threats. A common example of a cyber threat would be well-documented ransomware gangs.

The final deliverable of the emulation is documented results of detective and preventive controls that management can use to communicate success to executives and the board. Alternatively, organizations can leverage results of the security assessment as an additional resource to help detect and prevent high-likelihood attacks. The likelihood is determined by actionable and observable attacker data that has been demonstrated in the real world. At Wolf, we examine publicly available cyber threat intelligence (CTI) to ensure we create a plan and project for management that is up to date on attacker trends.

Our firm offers recurring threat emulation tests (e.g., monthly or quarterly) to show tangible improvements over time. However, we can also do this as a one-off or a “point in time” test as needed for clients. Test progress is measured against the MITRE ATT&CK framework, which is the industry standard for documenting cybercriminal actions. Think of it as the periodic table for cybercriminals.

Threat emulation is like model validation or unit testing but specialized for network exploitation. These exercises can also scale in complexity to fit the maturity level of clients and their specific network security needs.

How threat emulation plans work

Each project is scoped based on the needs of a client. These plans can range to test:

• A new endpoint detection and response tool
• A SIEM tool
• System and Organization Controls (SOC)
• Incident response capabilities

 
Each emulation plan is tailored to ensure we can create a simple exercise that remains efficient for management to take immediate action against cyber threats if needed. Here are three examples of cybersecurity frameworks frequently performed to deliver comprehensive security assessments:

Atomic Testing

• Simplest exercise and the best place to begin
• Emulates single adversarial action and observes defensive outcomes
• Easy to test and automate (can be done in seconds)
• Data validation and visibility is present
• Loses accuracy and realism to some degree

Attack Chain Emulation

• Secondary stage with added complexity to plan with a deeper focus
• Emulates several compounded adversarial actions at once to build a chain of activity
• Observe defensive outcomes holistically with more accuracy and precision compared to atomic testing
• Set of procedures used to validate efficacy of incident response and determine gaps
• Scales against adversary goals and objectives compared to singular actions
• Realism is still lost to some degree

Purple Team

• Difficult to automate and involves specific resources and capabilities
• Emulates adversarial groups with high confidence and certainty
• Retains accuracy and realism to effectively test several components of a control environment
• Collaborative exercise with management
• Demonstrates the full attack life cycle

How is a threat emulation different from a penetration test?

A threat emulation or purple team exercise is intended to test, and re-test, the performance of detective and preventive controls in an organization’s environment. It acts as a security assessment against a specific, pre-defined set of attacker techniques and cyber threats based on real-world attack data.

A penetration test may incorporate some of these techniques, but the intent of a penetration test is to find as many issues as possible for the scope and time allotment. Oftentimes, cybersecurity solutions are unfortunately not tested until a breach occurs and a threat actor is inside the network. We perform this exercise to validate assumptions of the control environment and ensure the business understands the effectiveness of human processes as well as tools to maintain network security.

What does a threat emulation project look like?

Some of the primary client requirements are to properly prepare the target testing machine. This may involve:

• Including all controls for the target machine (physical or virtual) or physical server
• Providing the domain account associated with the local admin on the test host
• Outlining and assessing the relevant controls being tested
• Identifying an appropriate emulation plan and a goal for the exercise

 
With this information, our team will document the plan and perform the exercise.

Below is an example of an attack flow for the September 2022 Uber breach. This is an example of a well-documented breach we can learn from with actionable data that we can attempt to emulate.

Example attack flow (partial)

Example target attack flow (partial)

Conclusion

The DenSecure Team at Wolf & Company utilizes the most efficient procedures and trends to safeguard organizations against network security attacks. For details on Wolf’s suite of cybersecurity services, or to set up a scoping call, visit DenSecure and send us a message to discuss your needs.