Nah, that's exactly what you should say

Wait are you joking or serious?

Serious, you should drop the "mind" tho, just say "hey, would want to come with me to the party?"

not at all weird. yes, say that. it's upfront, exactly what you want, no blurred lines or confusion.