If you're looking for local hacking groups/meetups.. you might possibly have a few options
1) Local Defcon groups
As stated, most defcon groups are shit, and if they aren't shit, then they're dead.
2) Local 2600 chapters
they typically have the same problem as the defcon groups/meetups. They're shit and/or dead.
3) Check and see if there is a hackerspace/makerspace in your area. The closer you are to major cities, the more likely there will be a hackerspace for you to hang at.
bsides conferences are cheap, all over the fucking world, and generally speaking, the attendees aren't giant egocentric douchebags like you see at Defcon.
Other conferences I can recommend (note: these are all based in the US):
CircleCityCon - Indiana: Hotel and airfare there was cheap. The trainings are by and far, the most affordable training sessions you will find anywhere. The organizers are great, the content is good and most the speakers are more than willing to shoot the shit with you after their presentations. IIRC, the tickets are well under 100bux, which is a bargain compared to most conferences.
tickets tend to sell out really fucking faster for this. Training sessions are top-notch, but training tickets are 1k a pop. Still, compared to shit like blackhat (average is around 2-3k) and SANS (5k if you want to be in an actual class with an instructor), its a bargain. Standard tickets are 175/ea
You'd expect a con in DC to be filled with nothing but suits and shitty "cyber" talks. shmoocon is decent, but not unlike derby, the tickets sell out really goddamn fast. they're 175/ea, I think.
other conferences I've heard good things about, but have never attended:
ShowMeCon (St. Louis)
Layer 1 (Los Angeles)
>What if I don't have money to attend a conference?
Sometimes conferences like Defcon, Blackhat, Shmoo and some others will offer a "scholarship" for students or people who really want to go and can't afford it. Try applying. You never know.
also, if you just want to see the presentations, just go to irongeek.com. If the videos aren't there, go to the internet archive and search there. Some conference organizers are easily triggered and don't appreciate irongeek's sense of humor, so they drop their shit on archive.org instead.
With regards to "infosec, twitter, and drama" yeah. There's a lot of drama and bullshit, but its a really good source of information. Don't discount it so easily.