Seriously? That's it? Thought it would be more complicated.....Thanks

The subject speaks for itself. Someone sent me a guild invite while I was in combat. Just to get the message out of the way I tried to click no, but thanks to my scroll speed i hit the yes tab on accident and joined into his guild. I dont want to be in it because I'm a tad of a loner and there's only two people in it that are of a high enough level to help on any of my quests (Most are still in the single digits and low doubles) so yeah, can I leave it or no?