I know this sounds weird but, am I going to be able to have sex?

Ok, so I'm not really sure what's wrong with me. I kinda had sex once. I put my penis inside of the girl, and it was just so uncomfortable that I lost my erection. I told this to my close friend, who has had sex with lots of girls, and he told me it was because it was the first time I put on a condom. I'm not sure I just feel like I'll never be able to keep it up with a girl, and I'm only 18. I am definitely sure I'm not gay, but I just feel weird about this. I masturbate a few times a week, and I never use my bare hand, I always wrap my penis in a blanket and put my hands around that, because I like the way it feels. Am I just so used to this that a vagina won't feel good? Haha I know this all sounds really weird, and I just don't wont any answers ridiculing me. Serious answers only please.