The largest determining factor is weight and gearing. If you add heavier wheels, it will decrease your acceleration as well as handling response.
Think of it as your car being a human sprinter, running shoes are optimal, Timberland boots are the worst. So choose your wheel weight accordingly.
As far as gearing goes, the larger the wheels, the taller your final drive ratio, a taller ratio will net you softer acceleration and a higher (theoretical) top speed.
So those are your basic rules of thumb. Good luck with the purchase