Idolatry is the attempt to draw down into a physical object the spirit of some spirit or being besides God.
In the Torah we find spirits besides God --serafim, ofanim, refaim, seirim etc. To believe that these forces exit is not idolatry. To pray to them is. Pantheism is not the faith of the Torah. The faith of the Torah is Monotheism. And considering God separate from the world is not idolatry. Monotheism does not deal with the question of God's physical location.