You are correct.
Currently I'm using a modified heavy duty Thule bicycle carrier, and yep - it flips down. It actually has a shock absorber on it, so the tire swings down at a controlled rate. And you are correct about a secondary attachment point. I use a quick release and a turnbuckle on a chain, which stabilizes everything, along with a hitch stabilizer.
Its quick and easy access to the tailgate - you just pop the quick release and press down on an aluminum release lever - and the tire moves out of the way. To put it back into position, you just push the tire back into position and it latches automatically. Reconnect the quick release and your good to go.
I would like a traditional tire carrier, but this setup works well in the mean time.