17 Jan 2022

Agent-based discovery technique to address Log4j vulnerability

The cyber security community has been chasing remediation measures for the Log4j vulnerability present in the libraries of many applications. This article focuses on how industrial organisations similarly situated might address the challenges most effectively.

Industry needs lots of ideas to address this significant risk. There are many very good articles, including those by CISA and others that do a great job

We recommend the following articles:

The Log4j project security page
CISA guidance

Log4j exists within a library that other applications use for a variety of tasks. Most end-users – and in fact many software companies themselves – are unaware that the applications utilise or require this library, or at least the vulnerable version of it.

Second, the library cannot be updated without updating the application. Third, identifying where the library has significant challenges due to the depth and breadth one must search to find it.

There are many methods that have been offered to protect organisations from the threat posed by Log4j, the CISA guidance being a very good source.

However, to truly protect the endpoint, the organisation must find these libraries and patch or update them in most cases by updating the application. Many vendors have issued alerts to their customers and publicly announced their applications are vulnerable, offering customers an initial list to search. But many vendors are uncertain if the vulnerability will impact their products, so the list is still very incomplete.

Efforts in IT to find this have included:

  • Adding the publicly announced software vulnerabilities into the databases of common vulnerability assessment products – a great start but has significant gaps and potential errors depending on the level of automation desired.
  • Trying to monitor traffic to find active exploits or presence of the library – helps prioritise where to go deeper and could identify an active exploit but may occur after the horse has left the barn, so to speak.
  • Scanning devices to identify the presence of the library on that system – creates significant stress and system impact

Verve observed a common scanner taking 45 minutes, 400MB RAM, and an entire CPU core to scan a single system. It is risky in an enterprise data center with network-mounted SCSI drives, CPU, and disk oversubscription. It consumes these resources on devices whether or not they even run Java. Many of these are not cross-platform, requiring separate scanners for Windows, Linux, etc.

Even at the end of all this, they may miss applications on attached drives, network drives, etc.

In the environments in which Verve and its clients operate – critical infrastructure, industrial controls, etc. – these scanning technologies would cause significant operational challenges and there is no way an organisation could run the common scanners on all devices given the length of time and system impact. Therefore, we have taken a different approach.

360-degree view

The approach takes advantage of the 360-degree risk view that is part of the Verve platform. Using our agent and agentless capabilities, as well as our real-time monitoring of network AND end-point behavior, we narrow down the targets, reduce the system impact and sequentially work through this challenge.

The approach, in short, is as follows: we developed a very OT-sensitive “agent-based discovery” approach that doesn’t utilise significant CPU or RAM and runs in a matter of a few seconds rather than 45 minutes. This has been shared with the community.

This “agent-detection” approach allows the identification of devices you know are vulnerable. While this is not a comprehensive list, it initially eliminates the false positives and begins to make traction. Importantly, this is cross-platform, so you only need one detector across all device-types

Simultaneously, our dashboards are updated with the latest list from CISA on all the publicly announced vulnerable applications which we use to help customers identify where those applications exist in their environment. This list is being updated daily.

We use the Verve platform to triage these devices through a range of remediating actions from patching to managing ports and services.

We access our historical endpoint behaviour data captured in our OT Threat Detection component to look at process creation and metrics to identify machines with transient Java services running. This provides us with the next batch of target devices.

The threat detection capability allows monitoring of IOCs and attacks. IOCs and attacks pivot over time. Simple rules alone aren’t enough to keep up with the threat landscape. Verve’s Threat Feed is constantly updating with the latest available information about emerging threats and automatically detects when a host was found communicating with a known, malicious server.

Finally, we utilise the rest of our threat detections built into the platform to identify post-exploitation events: eg using Log4Shell to install and execute other malicious software.

Once the initial triage is complete, the user can go back to the Internet-facing devices and begin further investigation.

The industry needs lots of ideas to address this significant risk. This is our contribution to the effort. We look forward to continuing to work with others to push the boundaries on what is feasible, especially in sensitive OT environments.

Company info: Verve Industrial