I don't have this problem. I usually just say in /1 LFG "Questname". This will usually give me either a group invite or a whisper from someone within a couple of minutes. I NEVER accept random invites from people, if you can't even bother to whisper me, the you can find someone else to group with..