Here at OSR we have been proponents of 64-bit Windows for quite some time - first as an important future platform for driver developers to consider and now as an actual platform to use when deploying server and workstation systems - most of our technical staff is using 64-bit in one form or another and it is rapidly moving to the desktop.
Of course, like any new use of technology there are bound to be bumps along the way. As we find them, we've been trying to capture our experiences ans share them with the rest of the community - hopefully we can spare people some of our pain as a result, and foster adoption of the 64-bit platform.
We recently had to install 32-bit Windows along side 64-bit Windows on one of four development boxes because we needed to run a specialized product (data recovery, actually) that didn't quite work right on 32-bit Windows. The installation went fine, we recovered our data and then tried to reboot back into our 64-bit environment - only to be told that "<windows>\system32\ntoskrnl.exe is corrupt or nonexistant". The advice was to replace the file.
Booting into the recovery console of 64-bit Windows is a surreal experience to begin with - you are presented with a menu of options with seemingly random drive letters (the first time through this the presented choices were e:\windows, f:\windows, g:\windows). There's no real insight into which version of Windows is which. So choosing one it prompts for a password (the Administrator password). It turns out the first guess wasn't the right guess - it was a 4 year old installation on a drive we moved over to the system (anyone remember the admin password on a 4 year old Windows installation?)
The second boot/reboot cycle went better. This time the drive letters came up e:, c:, d:, each with the "\windows" directory attached. We chose "c" and found that this was indeed our 64-bit Windows directory.
A quick "cd \windows\system32" allowed us to peek at the ntoskrnl.exe stored therein - looked fine. So we headed over to the x64 installation disk. The compressed version (ntoskrnl.ex_) was there so we tried to decompress it onto the hard drive - "access denied". Strange. Try a different directory, same results.
A heavy sigh at this point, a reboot into 32-bit Windows and browse to the relevant directory on the CD-ROM. "The program expand.exe is not a valid Windows executable image" and then "access denied". Ta da! Turns out that the shell was grabbing expand.exe from the current directory - and that is a 64-bit version of the utility. Of course, this doesn't work with 32-bit Windows. A little jiggling with current directory and finally the expansion works. The images are different, so we rename the old image and rename the newly expanded image into place. We also checked the HAL (since kernel and HAL really do need to be a matched set, at least according to Windows folklore!)
Reboot, hold our breaths.. and the same error.
After several hours of this we finally resolved to "repair the installation". That proceeded reasonably smoothly - the usual scramble under the desk to find the sticker with the Windows license key on it, realizing that you can't read it because for some inexplicable reason nobody installs lights under their desk, so you take that torchiere lamp (fire hazard that it is) sitting in the corner, lay it on the floor and shine 300 watts of brilliance at the label.
Subsequently we were discussing this and one of our young bright engineers suggested the bootstrap loader might have been the problem. After a bit of digging it turns out that of course he was right. The 64-bit bootstrap loader is capable of loading the 32-bit OS image, but the 32-bit bootstrap loader looks at the 64-bit OS image, concludes that it is corrupt and spews out the relevant warning message.
Of course the lesson then is that if you are going to live with both 32 and 64 bit OS versions, you should make sure that you either install your 32 bit system first or after installing the 32 bit system you restore the 64-bit bootstrap loader.
In spite of it all, 64-bits is definitely worth it. We'll have to co-exist with 32-bit systems for at least the next couple of years - so it is definitely worth learning how to "get along" with both of them.