iSH, JIT, and EU

2024-10-16

Earlier this year, Apple announced a sweeping set of changes to iOS aimed at bringing the platform in compliance with the European Union’s Digital Markets Act. Naturally, such a significant reform to app capabilities and distribution has left some wondering what this means for iSH users in the EU. We’ve been working to answer this question and have some updates to share.

While Apple’s DMA concessions are broad, the parts most relevant to iSH are new APIs to support just-in-time (JIT) compilation of code. Hardware support for this feature has been present in every iPhone going back to the original one released in 2007, but it took until 2011 for Apple to start taking advantage of it in their own software. That software is, of course, Apple’s Safari browser engine–the first and to date only software on iPhone that is permitted to use JIT compilation technology. Until now, that is.

JIT compilation plays a key part of high performance interpreters–including the ones present in every modern web engine–and Safari’s WebKit is no exception. While Apple has had a long-standing policy to disallow shipping of alternative browser engines on iOS, gating JIT to system webviews provides a technological barrier to using one, effectively dooming their interpreters to be unable to compete on performance with Apple’s. To level the playing field, the EU has asked Apple to make this functionality available to others. For the first time, third-party iOS apps will be able to use JIT compilation if Apple deems them in compliance with their requirements for browsers. It is unclear what the exact policy is for granting these requests, given that there do not appear to be any apps that are using this API yet, but we leave those questions to be answered by other browser vendors.

Apple’s JIT API is clearly designed to support browsers, and in many ways mirrors the architecture of their own engine. However, the JIT technology it exposes is generally useful to any interpreter, including the one that powers iSH. For the past several months, we have been privately evaluating whether we can use this API in our app. This has proven challenging: officially, even developing against this API officially requires permission from Apple, and iSH does not meet the requirements for us to make such a request. Still, we’ve been able to use workarounds to test some prototypes. This has allowed us to confirm that such an approach would significantly improve the performance of iSH–in some cases, by an order of magnitude. This is in line with academic results and findings from other related projects, and thus is not particularly surprising.

Under the DMA, third parties can lodge requests for interoperability with gatekeepers (Apple, in this case) to request free and fair access to hardware or software features available to the platform’s owner. From our testing, we were able to demonstrate that the requisite JIT features we needed are in fact already available and useful to iSH. The only obstacle preventing us from using them, besides updating our app with support, is Apple’s policy limiting its use to browsers.

In July, we filed an interoperability request with Apple to expand the scope of JIT compilation to non-browser apps. A full copy of the request we submitted can be viewed here. Unfortunately, after two months of deliberation Apple has denied our request. This means we cannot provide JIT support in iSH to our users in the EU at this time.

This decision is regrettable, as we feel that JIT compilation would be unambiguously positive for iSH. This feature is a major win for performance and battery life. In addition, we’ve already done a security review of our proposed implementation and found that it meets or exceeds Apple’s requirements for access to this feature. And, ultimately, we believe that Apple’s rejection ties the hands of third party developers looking to ship interesting software for their platforms. We get a lot of feedback from our users and it’s been our top feature request since we started working on the app.

Ripping out the asbestos

Of course, most of these requests do not actually arrive in the form of “please add a just-in-time compilation tier” (though, perhaps surprisingly, some do). Instead, we get many variants–using more colorful language than others–along the lines of “Why is iSH so slow? Can you make it faster?” The concerns aren’t misplaced: iSH typically benchmarks at 5-100x slower than native code, depending on workload. This performance hit is inherent to the environment iSH operates in. JIT compilation would unlock powerful ways to cut down on overhead, but to see why, it’s important to understand how iSH works today.

The premise of iSH is a simple one: it provides access to a command-line shell on your iPhone, one that runs locally and supports all the software you expect to use. While conceptually straightforward, its implementation is anything but. On a personal computer the shell has broad privileges by design. As an extension of the user, it can split itself into multiple processes, read and write directly to the filesystem, and communicate with other applications. Most importantly, it can execute code: not just some prepared commands, but real code, written by the user or downloaded from the internet or generated from yet another program. This is, to put it plainly, not how phones work. An app runs in a strict sandbox, limiting its capabilities. On iOS, an app cannot spawn multiple processes; it is assigned a narrow slice of the disk with which to work with. And critically, it cannot execute new native code: the OS prohibits it.

iSH, of course, works anyway. It leans into the restrictions, offering a convenient “Linux in an app” that is isolated from the rest of the device, so it cannot break anything, but still accessible enough to get real work done. It achieves this by turning to a standard technique in computer science: emulation. Two layers of it, in fact, each working around different limitations on iOS.

The first layer of emulation is the kernel layer. Apps cannot spawn new processes; they cannot schedule them, nor can they offer or gate access to resources. This is traditionally the job of an operating system’s kernel, and iOS already has its own. In iSH, we emulate our own kernel, translating the needs of the programs we run and wrapping them into requests for the real iOS kernel to service for us. Our translation chooses to implement a Linux interface, which is vaguely similar to the XNU kernel that iOS uses. This lets us map Linux processes to groups of iOS threads and forward file operations to a location inside the app sandbox. The process is very similar to Wine, or WSL1.

Our second emulation layer works around iOS restrictions on executing native code. We can’t run the code directly, so we don’t: we interpret it, as a script, using a custom interpreter called Asbestos. It’s largely written in assembly and uses techniques such as direct threading and branch prediction for speed. For the interpreter, the architecture being emulated is largely irrelevant, so 32-bit x86 was chosen to balance ease of implementation with software compatibility.

Emulation is an effective way to bridge dissimilar systems. It’s a wonder iSH works at all, really, but it does, in a way that is useful to many thousands of people. But it’s not free. This translation has overhead, and in the case of iSH the overhead is quite severe. There are still some ways we can speed up the interpreter, which we will continue to implement (as time permits–making fast interpreters is a billion dollar problem, and we’re just hobbyists). But those improvements are nebulous and have diminishing returns.

Just-in-time compilers substantially reduce emulation overhead by generating native code that can run at nearly full speed. There is still some extra work required to do this translation, but the gains are typically quite substantial for most applications. This is the reason why all major browsers on every modern platform have a JIT compiler for interpreting JavaScript code, or why so much research is put into optimizations applying just-in-time compilers to other languages. Apple uses it for accelerating not only JavaScript but WebAssembly, CSS, and regular expressions in their Safari browser on their mobile and desktop OSes. It also forms a critical part of their Rosetta 2 emulator on macOS, which is very similar in design to iSH’s interpreter. And Swift Playgrounds for iPadOS uses a limited form of native code generation to ensure the performance of apps it builds–a capability no other app on the platform enjoys.

We can’t speak to exact numbers because it’s not easy to test a JIT on iOS. However, our tests show easy wins of 2-5x on almost all tasks, even with very simple code generation. Depending on the implementation–we took Apple’s BrowerEngineKit as an example–we can improve performance even further by optimizing address translation and instruction lowering. The ability to generate native code just-in-time is unquestionably a major performance win for iSH, and essential if it is to compete with other scripting runtimes that have access to these features.

Industry-leading security

The DMA provides gatekeepers broad latitude to prioritize platform security over interoperability requests. Apple has repeatedly claimed that they believe the changes they are being asked to make in the EU compromise the safety and privacy of their users. Answering the question “is [x] secure” is never straightforward, and evaluating just-in-time compilers is no exception. Still, we have thoroughly reviewed iSH’s proposed implementation and we believe we can say that it incontrovertibly matches or surpasses comparable efforts, including Apple’s own.

Evaluation of the security properties of native code is a complex subject that is out of scope for this post, but Saagar has written a few words on the topic of to threat modeling against code generation. We will instead focus here on the specific security measures iSH would employ.

iSH, like all third-party apps on iOS, is subject to the platform sandbox. This serves as the first line of defense against malicious software, and it’s a very effective one: barring exploits or the occasional oversight, apps cannot strew cruft all over your device, trivially access your private data, or hog system resources. As far as we are aware, there are no serious calls to remove this sandbox, nor do we think it should go anywhere. iSH remains sandboxed today and we fully intend for it to be sandboxed forever. This is a valuable security feature which would be foolish to squander.

Apple’s architecture for just-in-time compilers limits code generation to heavily locked-down, isolated subprocesses of the main app. We welcome the change and wholly support this design. It confers useful security properties while also simplifying bookkeeping. In fact, we would suggest the following changes to the model to make it even more secure: the new processes actually have access to too much, as they are essentially a copy of Safari’s profile for its web content process. Third party browsers have no need to talk to as many things as Safari does, and iSH has fewer needs still. Locking down the surface area would increase the security of this design. In addition, while not quite related to just-in-time compilation, it seems reasonable to offer the process isolation model to any app, not just those needing code generation. Many apps handle untrusted content and perform complicated parsing that can open the door to dangerously easy compromises if done in the same address space as critical user data. Apple already splits off these tasks into specialized “quarantined” processes for its own software, and it’s overdue to make this functionality available to third parties as well. In fact, iSH is less likely to need such functionality, because it is less likely to be used in targeted attacks but also because it processes less untrusted content. Of course, we plan to use this design anyway because it’s good security hygiene.

Unlike today’s major browsers, iSH is a small project. It’s a lot simpler, and more amenable to adopting best practices for writing secure software. We aim for the iSH JIT to be written in a memory safe language, which would put it well ahead of attempts by the industry in this area. This would entirely eliminate or greatly eliminate a large source of vulnerabilities in our software.

The anti-Sherlock

In late September, we finally received a reply to our interoperability request. The response was terse but the actual rationale was shorter still:

After review, we have determined that it does not fall in scope of article 6(7) DMA. Apple does not itself offer emulation functionalities on iOS and it does not offer JIT compilation for non-browser apps on iOS.

There is a term in the Apple community that refers to the all-too-common practice of an indie app suddenly finding that their software has become a bullet point in the latest OS release: Sherlocking. Despite likely holding the dubious honor of the app that its users most want to be Sherlocked, iSH’s core value proposition has never truly been integrated into the platform. iSH uses iOS technologies but it does not compete with what iOS offers, because Apple does not provide comparable functionality. It’s unique. Or at least as unique as it can be in a market with a dozen terminal emulator apps with different tradeoffs of their own.

The rejection rationale is truly an anti-Sherlock: instead of Apple deciding to compete with our product, they believe that they do not have to offer their technology to us because they don’t compete with our app. They say, plain as day, that they do not offer emulation facilities on iOS. They don’t use JIT except for browsers and so nobody may use JITs except for browsers. The technology is there. The hardware for it is present, the software for it is already written. The only thing that prevents its use in iSH is that Apple will not grant us an entitlement to use the API. And as they have said, they do not grant this to any app that is not a browser, because they do not use it for anything that is not a browser.

Whether this is a valid response in light of article 6(7) of the DMA, we do not know. That is for the European Commission to decide. Our request was obviously submitted in good faith and is, to our understanding, valid. On the other hand, Apple feels that they are not required to expose this API because they do not compete in this area.

Perhaps this is true. In any case, however, it is somewhat disheartening to see this response. Apple does not think they have to give us access to this API because they do not compete in this area. But perhaps Apple does not feel they have to compete in this area because we do not have access to this API. We’ve poured a lot of love and sweat into making iSH. If what we can make is only “ok”, and never great, does that mean this market can be held permanently depressed by the gatekeeper never entering it? Not needing to enter, because everything can be limited to never being worth competing against?

We love working on iSH, because we love working on cool things. iSH is unassailably, indubitably cool. But it would certainly be cooler if it was great.