Prenatal vitamins related question...?

I didn't realize I was pregnant until 12 weeks so I wasn't taking prenatal vitamins during my first trimester. Does this increase my baby's risk for birth defects? I am now taking them in my second trimester. I'm really worried. Thank you.