The idea of Rav Nahman not to be מחמיר חומרות יתירות [not to seek extra restrictions] shows what it is about the religious world tat is "off". For the religious world is always seeking new restrictions.

 If people ae not familiar with the Gemara, this might seems to in accord with Torah. But the extra restrictions of the religious are very often not necessary and sometimes made up fanaticisms. However the world of the Litvak yeshiva is much better in thus regard.