Can't install 2.5.4 release after uninstalling prior beta

I uninstalled 2.5.4 beta (last one before the 2.5.4 release) and it didn’t seem to actually uninstall. It left the launch shortcuts on the desktop and the uninstaller flagged a bunch of files and folders for deletion, and nothing abnormal there.

I manually deleted the shortcuts and stopped the MCEBuddy service.

Now when I install, the installer tells me that MCEBuddy is still installed and exits. However, MCEBuddy is gone from the application listing in both the control panel and my uninstaller.

What can I look for to find out what is blocking the install?

See the install/uninstall FAQ’s table of contents depending on the error

I finally deleted enough registry entries and files to get the install to complete.
Important tip: I made backup copies of my profiles and settings in another folder to make sure I could put things back the way they were before. Particularly the recording profiles and rules.