I live in an area with a very active garrison and in fact have other active garrison members just a mile away, and I still got plenty of weird looks from the people at the craft and leather store when I explained what I was making. My neighbors are mixed with how they feel about it, some laugh and think it's silly, others admire me for doing charity work, and yes, the hunting and fishing guy next door pretends to act interested, but I can tell he's not digging it!
That being said, I'm guessing your dislike for your present location existed before you got into assembling your armor, this project has just magnified the feelings you already had.