(Much of this is a popularization of things already posted in this category, originally posted to Facebook.)
As I see it, halachic decision-making involves the weighing of numerous items — the strength of the legal logic, the authority of one’s sources, the breadth of acceptance (in both time and population) of the particular ruling, and which ruling better enables the asker to fulfill their obligations to “be holy for I am Holy” and “you shall do the upright and the good”.
And then different communities give each domain different weights. Rav Ovadia Yosef zt”l focused on the authority of the Shulchan Arukh and on numbers of preceding decisors and then on legal weight. Overturning precedent wasn’t an issue for him, and often that precedent was the Ben Ish Chai’s putting practice in line with Qabbalah — his framework for looking at the pursuit of holiness. German Jews look at the age of the longevity of existing pesaq (“authentic Minhag Ashkenaz”), being loathe to overturn precedent, in a way few other communities do. The Gra and then later the Briskers and others in the Lithuanian Yeshiva world, focused on strength of the halachic logic. Etc…
Halakhah decision-making is an art, not an algorithm. It requires comparing apples and oranges for their importance, and thus can’t be reduced to clear numbers and programmable results. The result is that the rules of pesaq cannot be articulated. To cast it in terms used in Computer Science, in particular the field of Artificial Intelligence, we would say that the halachic process is a heuristic, not an algorithm. While a poet who isn’t writing coherent clauses doesn’t make sense, the person who takes poetic license is not following the rules of grammar as formalized in a textbooks for immigrants learning English as a Second Language. (H/T Dr. Moshe Koppel, who composed that metaphor.) And like a language, to fully learn the way it’s used one must be immersed in its culture. An aspiring halachic decisor needs shimush, apprenticeship, under a skilled poseiq to truly learn the craft. Studying the texts in a formal teaching setting is insufficient.
The process of identifying expertise and bequeathing halachic authority might seem circular. Solomon Schechter’s system was prone to this flaw: He considered “halakhah” to be the practices accepted by Catholic Israel, and Catholic Israel the community of Jews who observe halakhah. The hole in this definition is that nothing prevents a group of Reform Jews to decide that they are part of Catholic Israel, that thus their practices are halachic — and because they’re following halakhah, their claims to being within Catholic Israel are justified. Schechter failed to articulate anything about “Constitutional Law”, something that anchors current process to defining elements existing halakhah. To provide rules for knowing which changes in Catholic Israel’s norms are valid, and which not.
I would say it’s more a cycle than a circle. Those with the expertise to understand what halakhah IS are those who decide what halakhah WILL BE. The skill has to reside among the poets who mastered the feel of the system from existing law; they are the ones we empower to make future law. And as R Moshe Feinstein told the NY Times, halachic authority comes from public acclimation. R’ Moshe told the interviewer something like “Someone asked me a question, he and those he interacted with liked my answer, so more people started sending me questions…”
Going back a moment to the factors being weighed themselves,two of them take halakhah beyond black-letter law. The first is that precedent has legal import. To give more strength to this idea: the Oral Torah is a dialog down the ages. If we allow too frequent overriding of the precedent, we rob the past of its voice in that dialog. Mesorah loses its continuity, and contemporary practice is cut off from the Sinai moment. We give the past authority because they were closer to Sinai — both in culture and simply as links in the chain we ourselves depend on.
The second is that being that holiness and being upright and good are themselves calls to go beyond black letter law, and to not be — as the Ramban put it — “disgusting with the ‘permission’ of the Torah”.) There are halakhos (Hilkhos Dei’os and parts of Hilkhos Teshuvah in the Rambam) that obligate us to pursue the Spirit of the Law to the best extent we can understand it in addition to its specific letter.
This does mean that decisions can at times be ends driven. For example we seek ways to free agunos from being locked into dead marriages. But those ends must be Torah values, and not convenience. And only as one weighting factor among many — the importance of the goal does not mean that we can find “solutions” that violate halachic process. Not every time that there is Rabbinic Will can we necessarily find a Halachic Way, and often when a mechanism does exist, that Way can only be found after generations of searching.
Thus all of Orthodoxy’s defining practices are halakhah. In fact, I would say that’s a tautology; a definition of Orthodoxy. However, some are specific laws, and some are the laws about trying to remake oneself into a holy person who pursues the Torah’s ideals. And unlike Catholic Israel, Orthodox subgroups can accept practices that violate the feel of the process and thus to a greater or lesser extent they could cease being Orthodox.