Friends of my 25 year old son show up at my house to play pool and drink w/o asking. Husband don't mind. Hel?

And when they show up, they are loud and stay late into the night. I am looked at like the bad "guy" for wanting them over.