The Terminal Escapes: Engineering unexpected execution from command line interfaces
By David Leadbeater, Open Source Engineer
At G-Research we have many developers using different tools for their work, and this includes many different terminals. Our open source team decided it was worth spending time looking for security issues in various terminals and related programs.
The result of this research was finding multiple remote code execution issues in terminals across nearly all platforms, including mobile phones.
The terminal is what displays the text and even graphics when you interact with a command line (for example bash or Zsh on Unix systems), which runs inside the terminal and is the primary tool you interact with. Different terminals have many different features, even including full graphics (via Sixel), sounds (DECPS, as implemented in DEC hardware terminals and now supported by Windows Terminal) and more.
Terminals use escape characters to support more than plain text, for example, in “\e[31m”, where “\e” is a literal ASCII escape character, the text will turn red.
As well as setting colour, terminals have escape sequences that reply with certain strings. Often these are not user controllable, for example the size of the terminal in characters. However, in some cases these are user controllable, or reply with a certain string that in an unexpected context can be misinterpreted by the program running in the terminal.
In order for these to be exploitable an attacker needs the ability to write raw unescaped text to the terminal, as well as either a bug in the terminal, or an escape sequence that results in a suitable string being written back.
The most notable previous work in this space was by HD Moore in 2003, where he discovered that the “report title” sequence in xterm could be combined with the ability to set the title and then essentially run commands (CVE-2003-0063).
Here I present the key findings as a set of case studies, showing a potential attack scenario and a way that would be exploitable by an attacker.
For each case study I will detail a vulnerability in a terminal as well as a way to deliver the string. In most cases the terminal vulnerability could be combined with nearly any other delivery method.
ConEmu and DNS
One of my first discoveries was that the ConEmu terminal emulator implemented the title report escape sequence. This meant the 2003 vulnerability (CVE-2003-0063) in xterm still worked. Worse, it also included control characters; this meant it was possible to include any character, including a newline.
I call this a “full echoback”; that is a case when the attacker can fully control the reply string, which makes full Remote Code Execution (RCE) possible.
I also found that in many cases the Windows nslookup (DNS lookup) tool does not escape special characters. This leads to a scenario where an attacker socially engineers an administrator to look up a DNS record, or another similar scenario.
Note ConEmu has only fixed the full “hands-off” RCE, there are still cases where a user in ConEmu can be tricked into running unexpected commands. While we support choice of terminals, we strongly suggest you consider using Windows Terminal, PuTTY or Mintty on Windows.
I also reported the lack of escaping in nslookup to Microsoft, but the company did not consider it a vulnerability.
Windows Terminal and Kubernetes
In 2021 Eviatar Gerzi found that kubectl did not escape characters in output (CVE-2021-25743). He combined this with rapidly updating the title and found various denial-of-service attacks against different terminals.
Windows Terminal is a recent development in terminal emulators, previously Windows did not use VT100 compatible escape sequences, but since Windows 10 and WSL, has gained support for these.
Windows Terminal has a way to update the working directory the user is in, which uses an escape sequence from ConEmu. I found this escape sequence, when combined with the “Duplicate tab” action, had a classic shell escaping issue, where an attacker could inject commands.
Further to Eviatar’s research, I discovered a way that any pod in Kubernetes can generate data that “kubectl describe” will report by writing to /dev/termination-log. This means even an unprivileged attacker without Kubernetes API access could potentially launch an attack against an administrator of the Kubernetes cluster.
While this example used Windows Terminal and Kubernetes, it is possible to combine the kubectl bug with many different terminal bugs. I have published a proof-of-concept Docker image that combines 5 different vulnerabilities on different terminals.
This was assigned CVE-2022-44702. In addition, I fixed the kubectl bug, which had been disclosed but not fixed (the fix is in kubectl 1.26).
Python and iTerm2
In HD Moore’s research, he uses the scenario of an attack via Apache log files. These days a common development workflow is to run tools directly in a terminal, for example “python -m http.server”.
I found that Python did not escape output from the http.server module and therefore it was possible to send escape characters to the screen
I combined this with a full echoback attack I found in iTerm2, which meant it was possible to quit the running Python program (by echoing back a Control-C character) and then run commands in the user’s shell, all from a HTTP request made by an attacker.
The iTerm2 bug is particularly interesting because it was a repeat of a bug from 2008 in xterm. It turns out that the xterm bug was fixed, but the documentation was not updated, so several terminals actually reimplemented the security bug.
Git and Mintty
Git is one of the most common development tools and it deals primarily with text. When git outputs text to the terminal, it in many cases uses a “pager” to deal with displaying multiple pages of text. The default configuration is to use “less,” I discovered that “less” did not correctly handle escape sequences when using OSC8 (hyperlink) support. This meant it was possible to put a string into a git repository (for example, displayed when “git log” is shown) that would result in raw escape sequences being sent to the terminal.
I combined this with a bug in mintty that was nearly identical to the iTerm2 bug outlined above.
These two programs are not a random choice, they are the default programs that “Git for Windows” ships, so the default configuration of Git for Windows was vulnerable to this, even though this vulnerability is not in the code for git itself.
The end result meant it was possible to craft a git repository that when a victim ran “git log” on it, using git for windows, the attacker could run commands on their system.
In total during this research, I found 10 CVEs that had potential for Remote Code Execution. The above case studies show some of the more impactful ones, a full list can be found in the extended whitepaper. I also found many possibilities for other attacks such as DoS and information leaks, which mostly did not get assigned CVEs.
These case studies are only a few examples of potential ways these can be exploited. STOK’s talk at Black Hat 2023 built on some of this work and his own findings and showed how this could be used to “weaponize ASCII“, for example attacking systems via logs.
I also detail in the BlueHat talk how in some cases an attacker can hijack an active SSH session and if the user’s terminal is vulnerable to one of these attacks; leverage that to run commands on their client system.
Hopefully it goes without saying that updating is important. However, development tooling is sometimes distributed via different mechanisms. One concrete example of this is “Git for Windows” as mentioned above, which can be installed as an option through Visual Studio, but this is not part of Visual Studio servicing itself, so it needs to be updated via a different process.
Anything that outputs text to a terminal can be used as a potential vector if an attacker can somehow control the output. As the case studies above show, there are many different vectors, from directly over the network (like DNS tools), to logs (like the Python http.server case), to system administration tools (where other local users can).
How to correctly escape depends on the programming language used, but in general any control character (except tabs and newlines) printed to the screen should be escaped. I go into more detail about this in the paper.
A request of terminal authors
Many of these vulnerabilities exist partly because of inconsistent handling of error conditions. For example, when an escape sequence can be ended by multiple different characters, different terminals behave differently.
The ANSI X3.64-1979 standard does not specify how to handle error conditions, this is true of many older standards – even HTML has only comprehensively defined what to do with error conditions in HTML 5. The internet used to live by Postel’s Law: “be conservative in what you send, be liberal in what you accept” however that can lead to security problems, see RFC 9413 for more modern advice.
If you are working on terminal software, you should use the reverse engineered VT100 state machine. This will both provide better interoperability and avoid some common security issues (more details are in the paper).
As a user, it may be tempting to go for the flashiest terminal with the most features, but, perhaps unsurprisingly, the more features the terminal has, the more likely it is to have security issues.
This research has proven that what some people considered a theoretical or historic attack class, can have serious current-day impacts. We responsibly disclosed the issues and many fixes have been made as a result, helping make the open source development ecosystem a safer place for everyone.
We are seeing more of this kind of supply chain attack lately. While it has not historically been considered the most serious class of vulnerability, that might be part of the reason why we are seeing more successful modern supply-chain attempts lately. When the attack surface shrinks, the would-be malicious coders are forced to pursue ever-more creative avenues.
For us, a key reason for publishing and talking about these findings is ensuring future awareness of this attack class. It is not new, but new programmers building today’s tools may not have considered these issues or understand the seriousness. While code execution is not possible in all cases, as we have shown, it was possible to achieve code execution on multiple terminals across all common client operating systems.
I recently gave a talk at DEF CON 31 on terminal security. That was the conclusion of a nearly year-long journey into the depths of a key tool most developers use but take for granted. Here’s a collection of additional talks and resources from this research: