Duh! I thought I tested with an external power supply yesterday, when rnixon asked about the power supply, but I should have pulled out the USB and then I would have realized you have to move the jumper to actually get power externally. What can I say, I'm a software guy.
OK running off a 9V brick (with JP1 bridging 2-3) works much better.
I also didn't realize I was not running directly from a desktop USB port. I moved to a direct port and now power from USB is working.
Thanks to everyone, that will teach me to trust that little red power led, or my DVM.