Scenario: You're strolling about a mall, minding your own business. In front of you, ten yards away, a person in the crowd produces a handgun from his pocket, and places it in a holster in his waistband. Said person walk away, no incident ensues.
How would you feel about that?
It's probably a citizens who is (legally or otherwise) carrying a concealed weapon, and places it in a more comfortable spot. It shouldn't have been in the wrong place, and he definitely shouldn't have taken it out in public, but he didn't do any harm, or did he?
He doesn't harm me, I don't really like to be confronted with guns outside of a range or cleaning table, as that usually results from criminal or irresponsible behavior, but guns don't scare me or make me uncomfortable. But I do understand that other people might be scared. I wouldn't say "take offense", but brandishing a gun can be considered an offensive gesture.
Same situation, different person:
-The person is accompanied by a uniformed police officer, and is obviously a plain clothes officer.
-The person is wearing a business suit, and places the firearm in his briefcase
-The person is wearing a hoody and sunglasses, he's shifting his head, surveying his surroundings. You're outside a pharmacy.
In the last case, you'd find me behind the nearest pillar with a cellphone ready to call 911, whereas I'd walk right by the first two. Don't tell me I'm thinking black and white, I know how I function in a stressful situation, can you honestly say that the image of a stereotypical criminal with a gun wouldn't startle you more than a police officer handling a handgun?
It's not the gun, it's the person holding it. Does he have good intention or bad ones, can he be considered a threat? Are you in danger? Guns scare few people, but a threat to their lives does.
I think that it's important to never lose sight of that, it scares people that they're vulnerable, this is only natural, but guns shouldn't scare you, people willing to abuse guns should scare you.