My old man has had 2 A4's a 1.8T B6 and a 2ltr B7. They've been great in terms of reliability. The CVT transmission seems to be their Achilles heel though. The B6 went through a tranny in 200,000kms with Audi quoting 8K to fix it. At least 60% of the kilometres were on a highway sitting at 100/110kmh so not a lot of load on the transmission. The current one has 240,000kms on it and is still going strong but I reckon it feels like it's been slipping a bit for the last year or so. Besides an O2 sensor nothing else has gone wrong with it. One of the radiator fans is on it's way out but that's it.
They have been really good on fuel as well.
It just depends how you drive though, as a daily they're fine but a CVT transmission and FWD is nothing to get excited about. Quattros are probably more prone to problems, drink more fuel and are louder.
RS4s are nice though but probably a bit out of the budget..