Hair loss question – help?

I looked in the mirror this morning and noticed that my hairline was pretty far back… not sure if it has always been like this but it’s freaking me out. Also, if I grab a handful of hair, 3-4 hairs will fall out. Should I be worried? I’m only 18, and all the men in my family (father, grandfathers) have a full head of hair (even my mom’s dad who is 70). Any help would be appreciated.