The religious clothing has always seemed a little "off " to me. For one thing, you are not supposed to advertise how religious you are. מה השם אלוהיך שואל ממך ... והצנע לכת עם אלוהיך
"What does the Lord your God ask from you ... and to walk modesty with your God".
The skull cap itself has seemed to me to be even worse. There is no record of Jews wearing such a thing until long after it was the custom of popes and bishops. [Take a look at all paintings and drawings of Jews from the Middle Ages. There is not one skull cap.]
If the issue is a head covering, there is no requirement of such a thing except for during the public reading of Torah as brought in Mesechet Sofrim [not from the Mishna but one of the books on halacha from chazal/the sages. The Taz wants to add that not wearing a head covering is the custom of gentiles. That is odd, since the skull cap itself is the custom of popes and bishops. So the Kipa itself is minhag hagoyim (custom of the gentiles).
