Much to my shame I have done the same thing. Didn't fit the cap/dipstick properly, got two miles down the road and pulled over in a cloud of oil smoke. I refitted the cap, rode back the two miles, cleaned everything and topped the oil up. That was three years and 12,000 miles ago and the bike is absolutely fine!
So I'd say, from first hand experience, don't worry about it. Top the oil up, ride and enjoy!