The same is true for me. A little bit more vibes on the downside, but many things on the upside. Better grip, better feeling of the bike, more safe (I do also stand up in town traffic), and appr. 1 cm lower footpegs.

Just try it. It takes no more than 1 minute to remove the rubbers