Thanks for the feedback. Sounds like how mine went (with the ghost rectangle). You were removing nine-year-old labels; my BMW was about 7 when I removed them. When I did it to my Jetta the car was brand new, and the decals never had a chance to set in so they came off cleanly.
As far as the white specks, I wonder if Magic Eraser might be able to do anything about that.