There is more to it than that. I have expressed some of the problems in terms of the fact that most of the religious world is made up of cults that make a show of Torah, but are really demonic cults. So there are a few problems that got me to thinking the religious world really is very far from Torah, and perhaps could considered as a real kelipa and a Dark force of evil. [See L.M. in his reference to Torah from the Dark Side which is what I think is the root of the problem with these cults.]
This however should not be considered as a critique on the authentic great yeshivas that learn Torah for its own sake along the lines of the Gra and Reb Israel Salanter and Rav Shach. And of course there are the Bnei Akiva places that combine Torah with work and serving in the IDF that I think are very great places.