During this past year, the Truesec CSIRT team has handled a notable number of incidents where the threat actor has gained initial access through the Client SSL VPN solution of the targeted organizations. In many cases, ransomware deployment has been the endgame, which has also been widely reported during the last month. In this blog series, I'll demystify the circumstances around the initial access, the common mistakes administrators make, and some forensic artifacts that can help us draw correct conclusions.
If you're as old as I am, chances are that you’ve seen some of the classic vampire movies. In those movies, the vampire couldn't enter someone else's house uninvited. But around 30 minutes into the movie, there was always this person inviting him in. This was the vampire’s initial access ‒ now the vampire could enter through any window, door, chimney, or whatnot into a house without garlic or crucifixes in every room. We all know what happened next: blood-drained people started to pile up.
The other day it struck me that this is very similar to many of the investigations we do. The vampire, or the threat actor in these cases, gets invited into the environments, and then encrypted servers start to "pile up."
Okay, I know what you’re thinking;
- Hackers hack their way in; how is that an “invitation”?
Well, maybe you've heard cybersecurity folks say, “Hackers don’t hack in; they log in”?
Unfortunately, that's not just a cool thing you can say at parties; that happens to be the reality in many cases.
Now, before we get into the details, I’d like to take a moment to draw your attention to this LinkedIn post by Markus Lassfolk, VP of Truesec IR domain, where he shows a MITRE ATT&CK matrix with data from our last 15 incidents at that time (see Figure 2).
What's interesting about this MITRE ATT&CK matrix is the reoccurrence of the attack technique “Valid Accounts” in the various “Tactics” columns. In simplified terms, this means that in half of these incidents, the threat actors used valid credentials in several stages of their attack advancing through the different environments, pursuing their end game. This phenomenon with threat actors using valid accounts is a topic of its own, and we’ll leave it there but keep it in the back of your mind.
Now, let’s get back to where we started, the vampire and the first invitation, the initial access. Figure 3 shows an updated MITRE ATT&CK matrix. If we take a look at the “Initial Access“ tactic column of the table, we see that the most common technique used is... *drum rolls*… “Valid Accounts” (in 13 of 25 incidents).
How Can a Threat Actor Have Valid Accounts in My Environment?
Attackers get hold of valid accounts in different ways; one way is to steal them (i.e., through a phishing campaign) or buy previously stolen credentials. Another way is to simply execute password attacks on the exposed services like the Client VPN Services discussed earlier. The threat actor may have figured out the account naming convention used by the targeted organization. With that information, lists containing account names together with common passwords or in combination with customized passwords (usually the case in more targeted attacks) can be used to automize login attempts to the Client VPN service. In addition to this, the threat actor can take advantage of common username and password combinations together with common default passwords used by different device vendors or used in IT system configurations. It's like a guessing game.
These attack methods go under different cool names such as "credential stuffing" or "password spraying" but are, in their essence, pretty simple and take advantage of common human behaviors (i.e., reuse of passwords, keeping a default password, setting guessable passwords, thinking our passwords are unique, etc.).
A successful password attack against your perimeter services results in a bad actor finding one or more username and password combinations that work in your environment. The initial part of the attack usually involves these steps:
- The threat actor identifies your internet-exposed services, such as Remote Access VPN (Client VPN).
- The threat actor runs automated login attempts against any login-identified login pages, the Cisco remote access VPN login page in this case.
- Any successful login attempt does effectively mean that the threat actor has one or more valid accounts, enabling initial access to your environment.
- The threat actor logs into the VPN with a valid account. This step is usually not automated and marks the threat actor's first interaction with the environment.
- The threat actor starts enumeration and lateral movement attempts, any accounts found in Step 3 will also be used during enumeration and lateral movement attempts.
The Client VPN as Initial Access? Not on My Watch ‒ I Know What I'm Doing!
Imagine this scenario, several years ago, you or your IT partner configured an ASA and installed it in your corporate network. To make life a bit easier while configuring, upgrading, and installing the firewall, an administrator account “cisco “ with the password “cisco” was created. We can all agree that using complex passwords during configuration/installation is a bit tedious. However, adequate security hygiene was used, and administrative access to the firewall was restricted to a few known IP addresses.
Later, another administrative account (for instance, "admin") with a somewhat more complex password was created and used for the day-to-day administrative tasks, but that first account, “cisco,” was forgotten about (or kept as it was very convenient).
That part of the configuration would look something like this:
asafw# username cisco password ***** pbkdf2 privilege 15 username admin password ***** pbkdf2 privilege 15 ssh 18.104.22.168 255.255.255.255 outside ssh 22.214.171.124 255.255.255.255 outside http 126.96.36.199 255.255.255.255 outside http 188.8.131.52 255.255.255.255 outside
Okay, this is not optimal, but at least we've done our homework and restricted the administrative access to the firewall!
Now imagine that you sometime later had to enable the Client VPN feature on the same firewall (as it was required by your organization). You configured it with a VPN profile which made use of the local user database of the ASA firewall (see Figure 4), and created VPN users with complex passwords, 16 or more characters.
When creating user accounts, there are three different levels of access (easily seen in the ASA GUI - ASDM) you can assign to the user. Normally the user account used for the Client VPN solution is restricted from administrative access to the firewall device (see Figure 5), which is good.
But here’s the thing, there is no obvious option displayed that limits administrative accounts from being used for other purposes, such as Client VPN access (see Figure 6).
The inability to offer an easily configurable segmentation between VPN accounts and administrative accounts is a design flaw in the ASA software, which has survived throughout its entire lifecycle. That said, do you remember the administrative account "cisco” with the password “cisco”?
That account can now be used to log on to the Client VPN service configured and enabled on the device. This setup is essentially an invitation for the threat actor who just knocked on your front door, and if the threat actor has the cisco:cisco in their password list (and believe me, they do), initial access with a valid account is a fact.
Like the cherry on top, all ASA firewalls come with a default setting that allows all VPN traffic (including site-to-site VPN) to bypass any interface access list present on the inbound interface of the VPN traffic, see Figure 7.
This setting paves the way for the threat actor to continue with an enumeration of the internal environment and lateral movement.
When we have an internet-exposed service setup like described here, we invite the bad actors to simply log in with valid accounts; they don’t have to hack in. Even more concerning, if your internal environment lacks EDR, 24/7 monitoring, proper identity controls (i.e., AD tiering), and segmentation that makes sense, it's like encountering a vampire in your doorway without a crucifix, garlic, or holy water within reach; you're doomed.
Now, this scenario with the account “cisco” and the password “cisco” might seem farfetched or trivial, but it comes from a real-life incident we helped a customer to investigate and recover from earlier this year. The occurrence of local test accounts or multiple local administrator accounts is very common in this type of device; this is just a simple fact. And the rule of thumb is that as soon as you have one working “AnyConnect Connection Profile” in your ASA with “Authentication Method” set to “LOCAL,” all those accounts in the device’s local database are valid VPN users, regardless of how you originally intended to use them.
Most importantly, common administrative account names that do not have complex and unique passwords pose a threat to your environment. In many environments, administrative account passwords are seldom unique or complex. Many times, the passwords consist of one or two words where some letters have been replaced with special characters and/or numbers (like the most infamous password: “p@ssw0rd”). Often, the passwords have been set by administrators before us, which means that the passwords have been around for a while too. These types of passwords are, regardless of how ingenious and unique we find them to be, very common and occur very often in different password lists used by the bad actors and, consequently, in many of the incidents we investigate.
It should be noted that there are ways to prevent administrative accounts from being used as VPN accounts, but based on my own experience as a former ASA admin and the number of other firewall admins I know within the same field (not to mention all the investigation we have done), I think it is safe to say that it's rarely done.
I Don’t Use Local Accounts ‒ I’m Safe
Well, maybe, but there are still many considerations, and we've investigated many incidents where external authentication servers (i.e., Radius or LDAP services) have been used by the victim organization. In the next blog post, I’ll go through common mistakes in those types of setups and what strategies to use when configuring Client VPN solutions.
To conclude what's been discussed above in eight bullet points:
- Threat actors often obtain initial access by using valid accounts to log on to externally exposed services.
- One way of getting a hold of accounts is to simply guess them (password attacks against external services).
- Remote Access VPN services are targeted in such attacks.
- Cisco ASA firewalls have been targeted recently.
- The Cisco ASA firewalls lack the proper segmentation between local administrative accounts and local accounts used for Client VPN.
- Any local account configured on a Cisco ASA firewall (including Easy-VPN accounts) can be used for Client VPN connections if there is at least one connection profile with the “Authentication Method” set to “LOCAL.”
- Any valid accounts found and used for initial access will also be used in the internal enumeration and lateral movement attempts.
- If you lack proper protection, such as EDR with active monitoring and identity controls, your environment is at great risk.
If you're curious about what a ransomware deployment looks like, I can recommend reading this wrap-up by my colleague Viktor Uppströmer, in which he describes the details of an attack against a Cisco Client VPN solution that led to the Akira Ransomware deployment.