Fun and games with OS X Lion Server

With the recent release of Lion I decided to upgrade my desktop from Snow Leopard client to Lion Server and at the same time migrate the Snow Leopard mac mini server over to it.

According to the combination of a knowledge base article and the migration instructions Chapter 3 page 24 this should be easy. It was not.

The knowledge base instructions say in step 2-7

Open “Install Mac OS X Lion”. Note: In order to install Lion Server, the Server app must be in the same folder as the Install Mac OS X app. If you just purchased Lion Server, these items should be in the Applications folder on the startup disk.

Click continue and agree to the software license agreement if you agree.

Click the “Show All Disks…” button.

Select a blank disk on which to install.

Click the Customize button.

Select the Server Software package.

Unfortunately that isn’t true. It only works if you are already running Lion Server!!! Here is what you need to do.

Assuming you have a disposable Snow Leopard client on volume A and have a blank volume B.

  1. Boot into Snow Leopard client.
  2. Use disk utility to erase volume B
  3. Download both the Lion Client app and the Lion Server app from the app store.
  4. They will be place in you /Applications folder
  5. Run the Lion Client app, when asked press “Show all disks” then select install on Volume B.
  6. Unlike what Apple say the customise button will stay greyed out and it will ignore the Server app.
  7. During the install ignore the option to migrate (This is a client only migration).
  8. Once Lion Client is installed run the Server App to install the server component
  9. Copy the Lion Client app and the Lion Server app from the volume A /Applications to the new Lion Server /Applications directory.
  10. Use disk utility to erase volume A
  11. Run the Lion Client app, when asked press “Show all disks” then select install on Volume A.
  12. This time what Apple say will be true. The customise button will be enabled and you will be able to select the Server component to install.
  13. Connect the mac mini in target disk mode and during the install you will be invited to migrate from another server.

This guide to hosting Lion software updates on a Snow Leopard Server is correct and useful.

Installing grub2 on GNU/Linux Software Raid

I have been setting up a two machines with software raid to use as Xen virtual environments. The two machines have 5 and 4 500GB disks. I decided to run Gentoo as the Dom0  OS because there is lots of good documentation for it and it has always been my favorite GNU/Linux distribution. The install of the OS and xen is easy. I opted for the latest Xen 4.0.1rc3-pre code. Just emerge xen and xen-tools to get all the prerequisites and the unmerge them and build the latest source. Creating the RAID arrays was also easy. I did the following.

Netgear ReadyNAS Pro

I bought one of these because I keep running out of disk space. With the ability to store 12Tb, I thought it might keep me going for some time to come.

It allows you to build raid 0,1 and 5 arrays and Netgear’s own X raid something or other. Unfortunately, it doent allow striping and mirroring. It  is compatible with OS X and supports AFP shares and Timemachine as well as iscsi as I mentioned earlier. However, the GUI is a bit flaky and didn’t seem to like Initiator iqn’s at all. If you download the root ssh plugin you can access the box as root over ssh and look at what it is actually doing.

iscsi config is held in /etc/ietd.conf

 Lun 0 Path=/e/calendarserver,Type=fileio,ScsiSN=RN293R60037B-003,IOMode=wb
 HeaderDigest CRC32,None
 DataDigest CRC32,None
 IncomingUser user xxxxxxxxxxxx

 Lun 0 Path=/e/collaboration,Type=fileio,ScsiSN=RN293R60037B-001,IOMode=wb
 HeaderDigest CRC32,None
 DataDigest CRC32,None
 IncomingUser user xxxxxxxxxxxx

Firstly their Target IQNs don’t look like the spec described on wikipedia – I don’t know if it is Netgear or Wikipedia that is wrong here and don’t care as this doen’t seem to break anything.

The real problem in the Initiator IQN – I had to add this by hand and it gets stripped out of every entry in the file every time a new iscsi target is created and at other random times. According to the Netgear web GUI theses are needed for persistent reservation support.

However, once it is working it seems nice and stable. If your disks don’t mount just go and check those Initiator IQNs.

OS X Server

I have recently bought one of the new Mac Mini Servers, partly because I like everything mac and partly because I wanted to try OS X Server for my business.

First impressions are great. Apple have taken the best (or very good) OSS and created a nice, if somewhat basic GUI to sit on top. You can of course still go and edit the configuration by hand if need be.

Unfortunately, the graphics card failed in the first unit, however Apple were very good and I had a new unit in a couple of days. This got me thinking. Normally if I wanted to send a computer back for repair or replacement I would remove the disks but with a mac mini you have to prise it open with a paint scraper. Not something I really want to be doing to a box which is still under warranty.

So, when the new one arrived, I decided to take advantage of my Netgear ReadyNAS Pro NAS box’s (of which more later) ability to do iscsi and along with the globalSAN iSCSI Initiator added disks for each service that could contain sensitive data.  This took some time to get working, mostly due to issues with the Netgear but I now have all data on hot swap mirrored disks and the mac mini could go back to apple with no sign of sensitive data.

Developing an engine for OpenSSL

For fun I thought I would see how hard it is to write an engine for OpenSSL. There are several existing ones that you can look at. I started by seeing how the opensc engine worked. This code shows the first step.

#include <stdio.h>
#include <string.h>
#include <openssl/crypto.h>
#include <openssl/objects.h>
#include <openssl/engine.h>

static int bind_fn(ENGINE * e, const char *id)
  if (!ENGINE_set_id(e, "simple") ||
      !ENGINE_set_name(e, "simple engine")) {
    return 0;
  } else {
    return 1;


Compile it like this

gcc -c -fpic simple_engine.c
gcc -shared -o simple_engine.o

Make openssl.cnf look like this

openssl_conf            = openssl_def

engines = engine_section

simple = simple_section

engine_id = simple
dynamic_path = /path/to/
init = 0

distinguished_name = req_distinguished_name


Run OpenSSL and see your results

$ openssl engine
(padlock) VIA PadLock (no-RNG, no-ACE)
(dynamic) Dynamic engine loading support
(simple) simple engine

Of course it doesn’t do anything useful yet. But it is a start.

DNS Spoofing

At IETF 72 in Dublin I gave a demonstration of DNS spoofing based on the attack on DNS described by Dan Kaminsky. I was able to successfully inject a fake DNS record in to the cache of a name server with a fixed port in a few seconds and sometimes in well under a second.

Bert Hubert published a description of the math behind this attack on namedroppers and I have been playing with the spoofer to see how close I can get the experiment and theory.

I ran my spoofer on a network consisting of three machines linked by a cheap gigabit switch. The attacker was on a Mac Pro, the target nameserver was on a Mac Book Pro and the authoritative server, that the attacker was pretending to be, was on a old FreeBSD box (100Mb). I used DUMMYNET to simulate a longer link to the authoritative server (delay = 30ms).

I ran the spoofer 1000 times and plotted a histogram of the frequency of success against time.

The pink bar shows the median of all the times recorded. If I recall my A level maths correctly, this should coincide with the 50% chance of success predicted by the math.

The math presented by Bert Hubert considers the expansion of the binomial

Ps = probability of success on a single attempt

Pf = probability of failure on a single attempt

( Ps + Pf )^n = 1

Expanding this and remembering that the sum of all the terms containing success = (1- the term for always failing) leads to the probability of combined success

Pcs(n) = 1 – (1 – Ps)^(n)

We know that n = T/W so we get

Pcs(t) = 1 – (1 – Ps)^(t/W)

Bert Hubert tells us that Ps = (D*R*W)/(N*P*I) where

I: Number distinct IDs available (maximum 65536)

P: Number of ports used (maximum around 64000 as ports under 1024
are not always available, but often 1)

N: Number of authoritative nameservers for a domain (averages
around 2.5)

R: Number of packets sent per second by the attacker

W: Window of opportunity, in seconds.  Bounded by the response
time of the authoritative servers (often 0.1s)

D: Average number of identical outstanding queries of a resolver
(typically 1, see Section 5)

I used the following values
R=36000 – From looking at the traffic I was sending
W=0.030 – From the settings I gave DUMMYNET
N=1.0 (I fixed this)
P=1 (I fixed this)
Plotting this on the same graph as the histogram gives:

The blue circles are the predicted probability of combined success (Their y axis runs from 0 to 1 and is not shown). As you can see the predicted 50% chance (black cross lines) occurs slightly before the median but it is fairly close.

In order to improve things I added an extra term to the equation to account for the time that the window is closed (This is due to the spoofer taking a bit of time to notice that it has been unsuccessful and to try again). So:

n = T/(W+Wc)

Ps = (D*R*W))/(N*P*I)

where Wc is measured to be about 0.003 seconds. The graph now looks like

That seems like good agreement to me. The median in this case is 1386ms.

BTW: The graphs were plotted using R.  This is the code I used

#Plot a histogram of frequency of success against time
mydata <- read.table("/tmp/speed-test-30ms",header=TRUE)
#Plot both on a single graph
h <- hist(mydata$time,breaks=100,plot=FALSE)
plot(h,freq=FALSE, xlim=range(h$mids),ylim=range(h$density),
    sub="Histogram showing time to success of real spoofer (pink line shows median)",
    main="DNS Spoofer Performance",
    ylab="Density", xlab="Time/ms")
#Plot Bert Hubert's math
Ps <- ((D*R*W)/(N*P*I))
Pcs <- function(t){1 - (1 - Ps)**((t/1000)/(W+Wc))}
nx <- sample(h$mids)
#Scale plot to same as histogram
plot(nx,ny, xlim=range(h$mids),ylim=range(h$density),col="blue", ann=FALSE)
#Calculate time for 0.5 chance
time5 = 1000*(W+Wc)*(log10(0.5)/log10(1 - Ps))