You can see the idea of the Gra that lack of knowledge in the Seven Wisdoms causes a decrease in understanding of Torah a hundred fold. [Seven wisdoms was the Trivium and the Quadrennium.]
But these do include the classification of the Rambam that the categories of the work of Creation and Divine chariot refer to Physics and Metaphysics. [The Rambam says this openly in the introduction to the Guide.]
Most secular subjects are not at all in the categories of positive things but rather mind destroying. So one does need a certain degree of intuition and common sense in deciding into what to devote ones time. Myself I strive for the approach of my parents. Which was in action highly secular but they still had a sense of the need of balance between Torah and derech ertez [the way of the Earth.] So in my approach I went more into the Torah aspect. But still I realize the need for this kind of balance.
