Well I got the PCB and assembled it, and first off when I powered the board up it caused my PSU to go into over current protection. After a little investigation I realised I had grounded the left and right dots, which are not logic inputs like the rest of the chip for some reason, they are the cathodes of the LEDs. So I was basically putting a whole bunch of LEDs between 5V and 0V, oops. So having fixed that, it does indeed light up.
The next problem is that I forgot to include the enable line for the memory so that it only displays memory data/addresses rather than anything, which means that as the serial port is mapped as a device with an address range of 17777560-17777566 you pretty much only ever see that.
So a 2nd revision of the PCB is needed, a fix for the dot issue and a fix for the memory decoding issue.
In addition this board pulls around 1.2A and the displays get toasty warm, so I will also add a simple oscillator to change the brightness and also reduce the current/heat from the displays.