Compiling XNU Kernel on OS X 10.10 Yosemite

The source for the OS X Yosemite kernel is out, and this release brings a few new interesting changes. There are already a few instructions floating around the internet for compiling it but I think they’re slightly lacking in explaining things. I’ll try and detail as much as I can.

First things first – dependencies. All the dependencies (Apart from Xcode) are available from the Apple Open Source site. All the open source packages can conveniently be found at http://opensource.apple.com/tarballs/.

1. Xcode 6.1 – Available for free on the Mac App Store. The kernel isn’t an Xcode project so it won’t open in the Xcode IDE but it does use the backend, xcodebuild. You need to accept the license before you can start using it so if you haven’t opened Xcode before, then open terminal and type:$ sudo xcodebuild -licenseAnd agree to it. Before you ask, no you can’t just use the command line tools package instead of the whole Xcode package. The command line tools doesn’t include the full SDK required.
2. CTF Tools – Part of the dtrace package. As of today the latest version is dtrace-118.1. Download, extract, and cd into the directory from a terminal. To make and install:$ mkdir obj sym dst
   $ xcodebuild install -target ctfconvert -target ctfdump -target ctfmerge ARCHS=”x86_64″ SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dstYou should see “** INSTALL SUCCEEDED **” at the bottom. However It’s only installed to the dst folder you created earlier. Thats what the “DSTROOT=$PWD/dst” bit means. Finish installing it by executing:

   $ sudo ditto dst/usr/local /usr/local

3. AvailabilityVersions – The latest version is AvailabilityVersions-9. Same as CTF Tools, extract and cd into the directory. The installation is only very slightly different for this one, as instead of installing to /usr/local your installing to the /usr/local inside the macosx SDK, that is inside the Xcode.app package.$ mkdir dst
   $ make install SRCROOT=$PWD DSTROOT=$PWD/dst
   $ sudo ditto $PWD/dst/usr/local `xcrun -sdk macosx -show-sdk-path`/usr/localA successful make/install should give no output.
4. XNU – the latest version is xnu-2782.1.97. If you open up terminal and execute$ uname -a

   then you will see this is the same version as the release kernel supplied by apple.
   
   So download, extract, and cd into the directory. This is where things get interesting. I suggest you have a look around the source and familiarise yourself with the way it works. The makedefs folder is a good place to start. Inside you will find 6 different makefiles. MakeInc.top is the top-most makefile for the whole XNU build system. If you have a flick through, you’ll see it defines the supported platforms. While Apple has stripped out all the ARM code from the kernel, they haven’t touched many of the makefiles. This is true of MakeInc.top. It includes references to iPhoneOS and iPhoneSimulator platforms. These are obviously the platforms for iOS and iOS Simulator respectively. New to this kernel, however, is the iPhoneOSNano platform. What is it, you ask? I don’t know. However It’s pretty likely it’s the Apple Watch platform. Sweet.
   Anyway, back to something more useful, you’ll notice there’s a new architecture called X86_64H. This is for Haswell Macs. If your Mac is of the Haswell variety you should use this instead of the standard X86_64.

   Compiling? right. So back to terminal compiling is pretty simple.

   $ make TARGET_CONFIGS=”RELEASE X86_64 NONE” SDKROOT=macosx all

   In one of the makefiles Apple has changed the default SDK to macosx.internal. This is, as the name suggests, an internal SDK for Apple’s eyes only. No matter, XNU compiles just fine with the macosx SDK, which is the one included in Xcode.app. You could change the reference in the makefile (makedefs/MakeInc.cmd) but I prefer to just set an environment variable. The build will fail if you don’t set $SDKROOT. As for $TARGET_CONFIGS, this is a three part variable that replaces setting $ARCH_CONFIGS, $KERNEL_CONFIGS, and respective machine configs. It can be upper or lowercase, it doesn’t make a difference as its converted to uppercase in one of the scripts. The first part is the kernel config. It can be debug, development, or release. The second part is architecture. If your building for Haswell this is where you use X86_64H instead of X86_64. Just note that i386 won’t work anymore. Apple dropped it a couple of releases ago as all Macs are 64bit. The third part is the machine config. If you look at the config files it sets “DEFAULT” as the same as “NONE” so it doesn’t make a difference what you use. So for example you could use the following to build a debug kernel for a Haswell Mac.

   $ make TARGET_CONFIGS=”debug x86_64h default” SDKROOT=macosx all

   I’m building this on my late 2009 MacBook, which is pretty slow. Depending on your machine you may want to have a coffee break. I usually have to wait about 10 minutes.
   When compiling the Mavericks kernel you’ll notice a lot of errors passing by. These have been silenced in this release.

5. Installation – Once its compiled, head to BUILD/obj/RELEASE_X86_64/ (the last directory will differ. I’m sure you can figure it out) and you’ll notice your kernel is now called “kernel” instead of the old “mach_kernel”. This is due to the location change of the kernel, which is now located at /System/Library/Kernels/kernel instead of /mach_kernel. To install your new kernel you could do what almost everyone else seems to do and backup the original kernel, and replace it with yours, or you could do it the way Apple suggests:
   From within BUILD/obj/RELEASE_X86_64/:$ sudo cp kernel /System/Library/Kernels/kernel.local

   You can choose your own suffix it doesn’t really matter. Just as long as it starts with “kernel”. Then, you need to tell your Mac to boot using a kernel ending in “.local”. To do this we’ll write it to NVRAM. You can also add any other boot arguments you like at this point. I tend to add “-v” as well so I can watch my shiny new kernel boot in all its verbose glory.

   $ sudo nvram boot-args=”kcsuffix=local”

   If you like double check the change with the following:

   $ nvram boot-args

   

6. Restart into your new kernel! If you left out verbose boot then you won’t notice any difference. To check your new kernel open up terminal and execute:$ uname -a

   

   You’ll notice the date and time has changed to reflect when the kernel was built. Success!
   To go back to your old kernel, just remove “kcsuffix=local” from your boot-args and reboot.

Enjoy! I’ll be coming back to XNU soon when my Intel Edison arrives. I’ll be back porting it to the Atoms architecture so if you happen to be interested in XNU on Atoms in general then you might find it fun.

Advertisements

‹ Hey there

Posted in Darwin