Headshops started in the mid 1960’s and sold everything to consume and produce marijuana, like waterpipes, grinders, scales, and products to contain it. They became cultural hubs, selling counterculture newspapers, underground comic books music posters and other sundries.
Next to the “drug paraphernalia”, as this is called, they you will find merchandising like Bob Marley t-shirts, flags, stickers and a lot of other things. Everything except drugs.

Where does the name “headshop” come from?

It is a little unclear where the name comes from, some claim it is the abbreviation of “He Eats Acid Daily” but more likely it comes from the idea ‘all things for the head” or is it more likely it were shops for the “acid-heads and the pot-heads”?

What is the difference with a dispensary?

In a dispensary you can actually buy marijuana or cannabis related products. In some dispensaries they have a headshop located inside the dispensary.

Be aware it is not legal to carry “drug paraphernalia” with you in the whole of the US, you have to check what the legislation per state is.