An experimental U-Boot port for the iPod Nano 5G lives at https://github.com/freemyipod/u-boot/tree/n5g-wip .
It can be started using wInd3x and will start up a CDC-ACM serial console over USB for debugging purposes. Currently it has no storage driver.
The current port expects to be loaded in place of OSOS, eg. by packaging it into an IMG1 and sending it to WTF. This is what wInd3x does. This way, U-Boot does not have to do any of the 'annoying' early boot stuff like bringing up DRAM.
make nano5g_defconfig make CROSS_COMPILE=arm-none-eabi-
./wInd3x cfw run u-boot.bin
When successfully started, U-Boot will then enumerate as yet another DFU device, this time ready to receive an U-Boot compatible image. You can load an image using dfu-util:
dfu-util -d 05ac:2137 -D foo -R
-R: this is needed to get U-Boot to exit DFU mode and actually execute the image.