About The Project
CVE-2021-3156 is a 10-year-old sudo vulnerability that allows for privilege escalation in Linux environments. If you’re responsible for a Linux server, this definitely caught your attention due to the severity. Some rough PoCs wound up Github and also on exploit-db recently. Besides patching through upstream providers supplied pathches[0,1], how would you hunt for this in your environment? This leads me to leveraging auditd in the previously blogged about red team range. Let’s take a look at how to create and search for malicious activity with auditd.
Auditd - Linux Logging on Steroids
Auditd is the default auditing daemon for modern RHEL systems and other Linux systems. From file access to specific users running specific commands, Auditd can log it all! Even down to specific syscalls made by processes, auditd is an incredibly powerful logging utility. However, setting up the rules can be difficult, to begin with, if you’re new to the syntax. Red Hat’s documentation is a great starting point.
The auditd package comes with a handful of utilities that interact with the auditd daemon.
Three we’ll talk about in this blog post is as follows:
auditctl: add, delete, etc… rules for auditd.
auditctl -l: will list active rules
ausearch: Search the auditd logs.
ausearch -k $KEY_NAMEwill identify rules under $KEY_NAME
aureport: Get summary/statistic reporting from auditd.
The next section will focus on adding rules manually, but be aware you can add
several rules to
/etc/audit/rules.d/audit.rules and auditd will read from
Defining Rules - Watching Files
Let’s say we want to watch anytime someone reads or writes to
the command below adds a watch (
-w) to the file
permissions of read write (
-p rw) to be logged under the name passwd-rw-check
auditctl -w /etc/passwd -p rw -k passwd-rw-check
passwd and changing (or not changing) your passwd will result
in us reading from /etc/passwd and causing this rule to trigger. Searching for this rule via
ausearch -k passwd-rw-check we can see the time it occurred, the user that executed it, the pid, ppid, current working directory, and even some selinux related content.
All of the content in the image above came from just running
passwd. With that in mind, imagine the difficulties with effectively scaling these logs into some sort of SIEM to do hunting. Tuning your rules on a test system and kind of “purple teaming” yourself by executing what you’re looking for is a great way to understand what a utilitiy does and also limit false possitives in production.
Defining Rules - Checking for Executables
Let’s consider a handful of commands typically used in post-exploitation on Linux systems. In my experience, most droppers are small written in Bash or Python and simply triage the system and then bring a second-stage payload onto the system dependent on the underlying architecture or Linux distribution. From there the rest of the execution chain kicks of or additional processes are killed, etc…
To download second stage payloads wget, curl, and netcat can be leveraged. The three lines below add rules to always watch for when the exes exit if the x64 (
-F arch=b64) executable defined (
-F cmd=) runs a specific
syscall labeled by
auditctl -a always,exit -F arch=b64 -F exe=/usr/bin/wget -S execve -k www-cmd auditctl -a always,exit -F arch=b64 -F exe=/usr/bin/curl -S execve -k www-cmd auditctl -a always,exit -F arch=b64 -F exe=/usr/bin/nc -S execve -k www-cmd
The syscall we’re focusing on is execve.
It is common place to see
execve or another member of the
execve familiy be called as this performs execution of a particular binary from a syscall perspective. You could also focus on
read, as each binary listed below will be reading from some system library. This can be seen by executing an
strace -e read wget.
One large caveat to the syscall rule with the hardcoded path is that files that are moved or copied by the initial stager will not be caught. A file watch command would be great for the scenario of files being copied
and then executed under a new name. This is analogous to adversaries on Windows machines renaming “
powershell.exe” and bypassing allowlists that are configured within the environment.
Defining Rules - Adding Parameters
Expanding on the previous two examples, it is also possible to add parameters around additional paramaters (
man audit) to only log for certain user accoutns. For example, adding
-F uid!=0 would indicate our rule wants
non-root users to be logged. However if
UID was 0 (I.E root) then it would not be logged.
Hunting for CVE-2021-3156
A more in-depth write up on the vulnerability can be found here. The jist is that there’s a heap buffer overflow in sudo since 2011. The PoCs on
rely on a race condition to succeed. Both PoCs execute
sudoedit well over three thousand times by default.
sudoedit is going to be invoked several times within a small time period with strangely large parameters, we have a unique signature to inspect.
So to begin our hunting, lets create a rule that look for the execution of
sudo by a non-root user.
auditctl -a always,exit -F arch=b64 -S execve -F exec=/usr/bin/sudo -F uid!=0 -F key=sudo-cmd auditctl -a always,exit -F arch=b32 -S execve -F exec=/usr/bin/sudo -F uid!=0 -F key=sudo-cmd
Now, all execution of
sudo will be logged, but where does
sudoedit come into play?
ausearch, we can provide an argument of the
-c) which according to the man pages is “the executable’s name from the task structure”. The
comm-name parammeter is how to filter for
sudoedit as it’s actually a symlink to
sudo CentOS 8 systems.
After executing the previously discussed PoCs from exploit-db, and searching for our sudo-cmd rule, you should see an enermous amount of logs returning. That’s a big enough number to raise an eyebrow. To narrow it down further, the
--start argument can be added with a date and time set within quotes (ex:
--start 02/07/2021 '12:00:00') and you can do the same with an end time with
Searching and removing the pipe to
wc -l you will notice that the
proctitle seems a little bizzare, and you would be correct!
proctitle is hexadecimal encoded. However, you can decode it by appending
-i within your
Man that long string of characters seems pretty odd. Maybe it’s worth a deeper look!
- Anyone seeing this in a SIEM.
At this point, we can gather other context around a particular event and take a
deeper look with the other piexes of information that auditd gives us.For
example, the current working directory (
go pull artifacts from that directory. We see that the syscall was successful, and
have the time that it executed. Auditd can add a great amount of information to
a linux investigation if the rules are tuned ahead of time. You also can
always get a quick snapshot of your environment via
Beyond The Blog
This content can be extended into Auditbeat and integrate your logs into ELK or something similar. Plenty of awesome well structrured auditbeat rules exist on Github, and can give you a head start in your rule creation. If you frequently perform red team engagements in a Linux environment, consider leveraging auditctl to understand what is logged and how you may circumvent it. Thank you for reading!