I have a similar problem, and I suspect it's not the brakes at all, since my rotors are only a couple of years old, new front pads, and I rebuilt my drum brakes for this problem and still have it. At first I suspected the praking brake, but it seems to be working fine pre- and post- rebuild...