ExtensionKit makes extensive use of XPC. There are XPC connections all over the place! There’s a per-extension XPC interface. But, there’s also a per-view XPC interface. This can make for a lot of APIs to manage. It’s all critical too, as this will make up the foundation of an ExtensionKit system. So, to be successful with ExtensionKit, you have to work really closely with XPC. Let’s take a look at how we can do that.
This is part two of our series on ExtensionKit.
ExtensionKit uses XPC for all communication, via NSXPCConnection. Let’s start out by looking at an XPC interface that we’ll use as a basis for all our examples.
typealias XPCGreetingParameters = Data
typealias XPCGreeting = Data
@objc protocol ExtensionXPCProtocol {
func greet(_ greeting: XPCGreetingParameters, reply: @escaping (XPCGreeting?, Error?) -> Void)
}
Hmm, well this is technically Swift code, but it looks terrible. It turns out, NSXPCConnection doesn’t work nicely with Swift. Before we address this, let’s get into why it looks the way it does.
NSXPCConnection imposes two big limitations on our interface protocol. First, it can only use types that are representable in Objective-C. This means no enums, no structs, and optionals only for classes, to name some of the most problematic cases. At least these are compiler-enforced, so we’ll know what’s allowed right up front.
The second constraint is that all types must conform to NSSecureCoding. XPC must serialize all the parameters and return values for a method call, and it uses NSSecureCoding to do that. You absolutely can make a Swift class conform to NSSecureCoding. However, it requires a lot of boilerplate, it’s error-prone, and honestly, it’s just a pain.
It would be much nicer if we could just use Swift’s Codable instead. It makes for much easier seralization. It also means we don’t have to be constrained to ObjC-compatible types, because we’re just passing data over the wire. But, because we’ve lost our types, I’ve used typealiases. You could just use Data, but I like some clue as to what is actually expected on the other end.
Oh, and just in case you were wondering, I did do some testing, and NSSecureCoding offered no significant performance benefits over this approach.
You might not realize this, but Swift can automatically translate certain closure-based APIs into async-compatible versions. This works for XPC too, and at first, seems like it could be a huge win. Take a look - this version will work with NSXPCConnection.
@objc protocol ExtensionXPCProtocol {
func greet(_ greeting: XPCGreetingParameters) async throws -> XPCGreeting
}
When I first discovered this, I was pretty amazed. It looks like it gets us a lot closer to the Swift API we’d like. However, there is a serious problem with this approach.
Being a communication system, all XPC calls can fail. They can fail even if the method does not return an error. And, because of how they can fail, XPC methods do not guarantee that their reply callback will be called. This is extremely important, because that behavior violates the Swift concurrency runtime requirements. XPC calls will hang your tasks when they fail. Because of this, it is unsafe to use this technique in your XPC interfaces.
Let’s now try to address all of this so we can have a nice Swift API.
This poor fit between XPC and Swift has bothered many others. There are two libraries that look pretty nice for dealing with all this nonsense: SwiftyXPC and SecureXPC. They both offer async/await support, and use Codable for serializing data. Unfortunately, they also both use their own custom communication primitives. That doesn’t work well for us - ExtensionKit requires NSXPCConnection instances.
I have a feeling that it would be possible to adapt these libraries for use with ExtensionKit, but I didn’t try. I did play around with making a custom NSProxy to ease some of the burden here, but I didn’t get too far. In the end, I just accepted that some boilerplate is necessary. I very much look forward to a third- or even first-party solution. And, given how prominently XPC is used here, I expect something from Apple has to be forthcoming.
Ok, rant over. What does a Swift solution look like? Here’s a much more desirable version:
struct GreetingParameters: Codable {
let name: String
}
struct Greeting: Codable {
let value: String
}
protocol ExtensionProtocol {
func greet(_ greeting: GreetingParameters) async throws -> Greeting
}
Getting from our ExtensionXPCProtocol to this is going to take a bit of code, so let’s jump in.
Here’s a first pass of a wrapper for the host-end interface. There’s a lot of code here, and we’ve only got one method! But, it highlights the important problems we have to solve. Namely:
remoteObjectProxyWithErrorHandler to handle failurespublic final class RemoteExtension {
private let connection: NSXPCConnection
public init(connection: NSXPCConnection) {
self.connection = connection
precondition(connection.remoteObjectInterface == nil)
connection.remoteObjectInterface = NSXPCInterface(with: ExtensionXPCProtocol.self)
}
}
extension RemoteExtension: ExtensionProtocol {
func greet(_ greeting: GreetingParameters) async throws -> Greeting {
let xpcGreeting = try JSONEncoder().encode(greeting)
return try await withCheckedThrowingContinuation(function: function) { continuation in
let proxy = connection.remoteObjectProxyWithErrorHandler { error in
continuation.resume(throwing: error)
}
guard let service = proxy as? ExtensionXPCProtocol else {
continuation.resume(throwing: ConnectionContinuationError.serviceTypeMismatch)
return
}
service.greet(xpcGreeting, reply: { (data, error) in
switch (data, error) {
case (_, let error?):
resume(throwing: error)
case (nil, nil):
resume(throwing: ConnectionContinuationError.missingBothValueAndError)
case (let encodedValue?, nil):
let result = Result(catching: { try JSONDecoder().decode(Greeting.self, from: encodedValue) })
resume(with: result)
}
})
}
}
}
I think you can now really appreciate why there have been efforts to help improve XPC-Swift compatibility. This is just awful.
Fortunately, it is possible to factor out quite a lot of the common code. I did this, and put them into a very small package that is fully compatible with NSXPCConnection. Check out this version using ConcurrencyPlus.
import ConcurrencyPlus
public final class RemoteExtension {
private let connection: NSXPCConnection
public init(connection: NSXPCConnection) {
self.connection = connection
precondition(connection.remoteObjectInterface == nil)
connection.remoteObjectInterface = NSXPCInterface(with: ExtensionXPCProtocol.self)
}
}
extension RemoteExtension: ExtensionProtocol {
func greet(_ greeting: GreetingParameters) async throws -> Greeting {
let xpcGreeting = try JSONEncoder().encode(greeting)
return connection.withContinuation { (service: ExtensionXPCProtocol, continuation) in
service.greet(xpcGreeting, reply: continuation.resumingHandler)
}
}
This is a lot more tolerable. I admit, it’s still a huge pain. But, at least we now have a pattern we can follow that gives us the error behavior we need and the Swift API we want.
This RemoteExtension class would be used by the host talking to an extension. But, it’s likely that we’ll also need to go in the opposite direction. To do that, we can use a similar approach. First, we need a protocol used for XPC:
@objc protocol HostXPCProtocol {
func acknowledgeGreeting(reply: @escaping (Error?) -> Void)
}
Next, we’ll define a protocol for the actual API we’d like to use:
protocol HostProtocol {
func acknowledgeGreeting() async throws
}
And then, we’ll make a wrapper class that handles adapting the two:
public final class ExportedHost<Host: HostProtocol>: HostXPCProtocol {
let host: Host
func acknowledgeGreeting(reply: @escaping (Error?) -> Void) {
Task {
do {
try host.acknowledgeGreeting()
} catch {
reply(error)
}
}
}
}
Again, lots of boilerplate. In this case, I’ve omitted the need to deal with coding and decoding. But, you may still have to do it, and it will definitely still be annoying.
One thing that I do want to point out is the use of a standalone Task. This is a very typical way of getting an async context within a non-async function. It works. But, the ordering of Task execution is out of your control. If that host object was stateful, this could matter a lot.
In fact, I struggled terribly with this exact thing while introducing async APIs into my codebase. I was coming from the expectation that I could transparently move from closure-based functions to async. That only works when the underlying system isn’t stateful. And, I would imagine that is a special-case. The more I use Swift Concurrency, the more I view a standalone Task as a red flag. When you see it, it demands very careful thought about the ordering requirements.
To help address this, I ended up building a simple queue for Tasks. It works nearly the same as the standalone Task invocation, but guarantees ordering. I’ve also put that into ConcurrencyPlus. Check it out, if such a thing sounds like it could be useful.
Here, we have the building blocks we need to actually communicate over an NSXPCConnection with reasonable APIs. But, we haven’t yet actually used these with a real NSXPCConnection. To do that, we need to work out two parts - what local object we’ll export and the remote object proxy we’ll interact with.
To export our local Host object, I like to add a little extension to encapsulate the details. Here’s how that looks:
extension HostProtocol {
public func export(over connection: NSXPCConnection) {
precondition(connection.exportedInterface == nil)
connection.exportedInterface = NSXPCInterface(with: HostXPCProtocol.self)
connection.exportedObject = ExportedHost(self)
}
}
With this, and our RemoteExtension, we can configure our connection with just a few lines.
// given host and connection objects
host.export(over: connection)
// and here's the remote we can interact with
let remote = RemoteExtension(connection: connection)
Note that NSXPCConnection will retain its exportedObject. But, it will be up to you to manage the lifecycle of the connection itself.
Don’t forget, what we’ve covered here is just one side of the communication. Both the host and the extensions will need these wrappers. So, a finished version would contain:
HostXPCProtocolHostProtocolExportedHost for the host sideRemoteHost for the extension sideExtensionXPCProtocolExtensionProtocolExportedExtension for the extension sideRemoteExtension for the host sideThat’s a total of four protocols and four concrete types! And, all this is just for one connection! Remember that ExtensionKit can use more than one, depending on how your views are managed. I went through quite a lot of iterations trying to come up with a reasonable way to manage communication APIs. The patterns I’ve settled on here work ok. But, I really feel like this situation is untenable. We need better first-party Swift support for XPC, and we need it yesterday.
The things we’ve covered here aren’t actually that ExtensionKit-specific. But, XPC is a critical aspect of building a system with ExtensionKit, and it isn’t a simple thing to work with. Hopefully I’ve either presented some useful techniques, or horrified you and you are now inspired to build a library that makes this nicer. And if you do, please get in touch with me.
In the next part of this series, we’ll cover how to use these XPC primitives with ExtensionKit APIs.
Mon, Aug 22, 2022 - Matt Massicotte