I was not getting the BBT hate here, so I put some thought to it. I think the difference for me is that I grew up with a kid that basically is Sheldon, through and through. He has aspergers and is brilliant, more brilliant than anyone I have ever known, be he absolutely cannot function in normal society. The episode where Sheldon loans Penny some money hit it spot on. My friend has no concept of the value of money realistically, even though he has made as much as $300k per year designing robotics systems (started with Toyota, then consulting, now he is basically retired as his aspergers has progressed far enough that he cannot function in the workplace consistently enough to work). He has enough money, knows exactly how much he has and where he keeps it, and is willing to loan it out as long as the amount doesn't exceed what he views as "extra" with no thought to it afterward other than he does expect it to be paid back.
My friend entered college and got advanced degrees in engineering disciplines at a very early age (college at 16, finished before he was 22, etc.). He also is totally clueless in terms of romantic relationships and cannot fathom the desire to have that kind of relationship at all. So I was in awe of how well the Sheldon character hits that kind of person, exactly correctly every time. Even down to the little idiosyncracies like twitching if he isn't allowed to finish a thought or having his "spot", his schedules, all of it really. I also find their digs at the nerd culture and poking fun at society interesting and funny (probably helps that I grew up playing D&D, and have played Magic somewhat competitively in the past, as well as online games, so I get that culture).
But I can also see how if you don't have an eye for that kind of stuff it would be just too unrealistic and dumb, even though a lot of it is spot on. I remember when Napoleon Dynamite came out. I just didn't get it and found it ridiculously stupid. I watched it once, actually sat through the whole thing, and figured, well that was dumb, and I never planned to watch it again. Until I went to my sister's house and found my bro in law watching it and laughing his head off. For him it was the funniest thing ever. I found out that he actually started his teaching career in that exact town, in that exact high school in Idaho and so for him it all rang so true he found it the greatest movie he had ever seen, as it struck that nerve.