Do you have any kids, or is it just you and the wife?
If it's just you two, and you don't feel comfortable where you are, I'd say go find a new job somewhere closer to the bright center of the universe With kids this might be a little more problematic, as they probably have a stable social network there that they rely on.
Anyway, I can absolutely understand why your situation would be really, really depressing.
I'm blessed to live in a very urban region here in Germany, with millions of people living close by, so I get my fair share of nerd friends here.
I can only imagine how bad it must be being the "freak" living in an otherwise boring, conservative surrounding - if finding a job someplace else isn't an issue, I'd say do it! This hobby gets ten times better once you have people to share it with, and from what you've written it doesn't look like you're going to lose interest anytime soon