(1) There is such a thing as סוף הוראה (The end of the period when people could make a legal decision) the end of פסק הלכה. As the Talmud says רבינא ורב אשי הם סוף הוראה. Ravina and Rav Ashi were the end of the period when people could make a legal decision.
(2) The Rambam says during the time of the Sanhedrin, there was no argument because you went by the majority of the Sanhedrin. After that time the Oral Law in its entirety was complied into the two Talmuds and Midrashim. And the end of that period was the "end of decision."
(3) What is strange about the Shulchan Aruch is that it changes this decision making process. Instead of the give and take of people in a Sanhedrin, you have the decisions made over a thousand year period all considered equal and you collect them all to take a vote. רוב ראשונים.
Can you image all the small courts sending their decisions to the Sanhedrin and then the Sanhedrin going by a majority?
(4) And the cliche you hear at Shabbat Table Judaism, "We go by Shulchan Aruch," is false. There is almost no where n Halachah that the Shulchan Aruch is the way people decide halachah. They always go by some later authority that they happen to like. It is a game of קטנות המוחין small mindedness. What determines Halachah is in reality the pecking order. Who is the Alpha Male? That is how Halacha is decided.
(5) The trouble does not stop with the Shulchan Aruch. It gets compounded in every generation.
Once people get to the Halacha books of the Achronim the whole concept of סוף הוראה the end of decision is gone. Then the Rishonim become like the tenaim, and achronim like Amoraim. The entire idea of the Gemara [which the Rambam brings] that the Gemara is the "end of decision" is gone.
(6) What is the pecking order that is the deciding factor in Halacha? It is this: When on.e decides to start keeping Torah, he thinks he is giving up "this world." All the possibilities that were open to him will be closed. He can't any more get a Noble Prize in Physics, he can't become millionaire, he can't make it big in Hollywood. Why? Because now he is devoting himself to Torah. So you can imagine his surprise when he enters into the world that he thinks is following Torah, and he discovers they are just as much into money and honor and sex as the world he left. And who is the big shot gets all the money and good shiduchim. So he tries to become one of the חשובה אברכים (important people). And then he discovers Breslov. And he sees some kind of light. So he drops what he is doing and joins that group. And even what h might had had for sure he loses instantly. And then he finds out that there too is the pecking order the determining factor.
Torah is irrelevant to the people there just as much as in the previous groups. Because only the pecking order matters. Who is the Alpha Male? And after him everyone has their place. The Baal Teshuva is at the lowest rung, and given the scraps. And usually not even that.
(7) Like what Reb Chaim from Voloshin wrote "אין לנו אלא דינא דגמרא" We only have the decision of the Gemara.
(8) So why was the Shulchan Aruch written? The author, Joseph Karo had decided that there was a problem. In much of Spain, Jews had been deciding issues based on the Rambam. But in one area they were deciding like the Rosh. After the expulsion this state of affairs continued. So Rav Joseph Karo decided that this was a problem.So he set out to correct it by making a book that would be the one proper decision that everyone would follow. However sometimes something become a problem only after someone decides that it is a problem. And most often the solution is worse than the problem. For example at one time if someone was poor and decided they needed more money there was immediately available a simple solution. Get a job. The Lyndon Jonson came along and decided that Poverty is a problem. and everyone started agreeing and saying to each other you know what? Poverty is a problem. Da. And the obvious solution was tell people to work? No. It was to give away free money. Now we have seen the horrible results of this. So in our case. Why should everyone have to do the exact same thing? And let us say that in one community the light their Chanuka Menora from right to left and in other they light it from left to right? Why should that be problem?
(9) The problem with this situation is this: What happens when the idealistic newcomer does not want to accept his or her proper place at the bottom of the pecking order? What if they discover that they are not any less bright or moral than the big shots? Or perhaps they discover that teh big shots that are touted as Tzadikim and geniuses are in fact stupid, dull, barely sane, and morally challenged?