Yes, in theory a slight decrease in acceleration, but increase in stopping power and safety. Just like adding wider tires.
If you need bigger brakes, you probably already know it. Not necessary for a casually driven car. If you are glazing or burning up front pads at the track or want the extra weight transfer for autox, it's a no brainer.
There are lighter rotors available too.