MBA 611 Week 3 Share: Ockham's Razor
Ockham and his razor
Frequently, scientists cite loosely a principle known as Ockham’s razor: models with fewer assumptions are to be preferred.
This a principle some call Parsimony. Another view is that the canon of parsimony, forbids the empirical scientist from affirming what, as an empirical scientist, the scientist does not know.
Here is Richard McElreath, Statistical Rethinking (2020, p. 191), to help us with our thoughts and understanding (wisdom might come later).
"Mikołaj Kopernik (also known as Nicolaus Copernicus, 1473–1543): Polish astronomer ... Famous for his heliocentric model of the solar system, Kopernik argued for replacing the geocentric model, because the heliocentric model was more “harmonious.” But Kopernik’s justification looks poor to us now ...
"There are two problems: The model was neither particularly harmonious nor more accurate than the geocentric model. The Copernican model was very complicated. In fact, it had similar epicycle clutter as the Ptolemaic model (https://www.britannica.com/science/Ptolemaic-system). Kopernik had moved the Sun to the center, but since he still used perfect circles for orbits, he still needed epicycles. And so “harmony” doesn’t quite describe the model’s appearance. Just like the Ptolemaic model, the Kopernikan model was effectively a Fourier series(https://www.britannica.com/science/Fourier-series), a means of approximating periodic functions.
"This leads to the second problem: The heliocentric model made exactly the same predictions as the geocentric model. Equivalent approximations can be constructed whether the Earth is stationary or rather moving. So there was no reason to prefer it on the basis of accuracy alone. Kopernik didn’t appeal just to some vague harmony, though. He also argued for the superiority of his model on the basis of needing fewer causes.“On the contrary, we should rather heed the wisdom of nature. Just as it especially avoids producing anything superfluous or useless, so it frequently prefers to endow a single thing with many effects.”Copernicus, De revolutionibus orbium coelestium, p. 23 ( https://www.geo.utexas.edu/courses/302d/Fall_2011/Full%20text%20-%20Nicholas%20Copernicus,%20_De%20Revolutionibus%20(On%20the%20Revolutions),_%201.pdf). He also mentioned "Let no one untrained in geometry enter here." And it was true that a heliocentric model required fewer circles and epicycles to make the same predictions as a geocentric model. In this sense, it was simpler.
"Scholars often prefer simpler theories. This preference is sometimes vague—a kind of aesthetic preference. Other times we retreat to pragmatism, preferring simpler theories because their simpler models are easier to work with. Frequently, scientists cite a loose principle known as Ockham’s razor: Models with fewer assumptions are to be preferred. In the case of Kopernik and Ptolemy, the razor makes a clear recommendation. It cannot guarantee that Kopernik was right (he wasn’t, after all), but since the heliocentric and geocentric models make the same predictions, at least the razor offers a clear resolution to the dilemma. But the razor can be hard to use more generally, because usually we must choose among models that differ in both their accuracy and their simplicity. How are we to trade these different criteria against one another? The razor offers no guidance."
The question before the house
In building models, is Ockham's razor or even the canon of parsimony dead? After all we are in the age of AI, Machine Learning, etc. Is there something better?
Search your souls, minds, business, for an example and argue it and please keep it short and sweet unlike this post.
Without parsimony we will probably overfit and misfit most of the models we would propose. The Razor is another issue. Non multiplicanda sine necessitate (no additional anything without necessity) is one version. How would we define necessity? Here's Lewis and Short, Latin Dictionary for parcere, the root of parsimony:
ReplyDeleteparco, pĕperci, etc., [for sparco; Gr. σπαρνός, rare; cf. Engl. spare; but v. also paucus, parvus], to act sparingly, be sparing with respect to a thing, to spare.
Yes, to spare in the sense of "be merciful." But it also has the further meaning of "use carefully, not to injure." Let's not injure our client with a non-parsimonious analysis. The only downfall of this approach is that if we left something out of our analysis, might we overlook something else it is connected to and which is also important, though seemingly insignificant now?
Since models with fewer assumptions are preferred, it gives us little room to explore different and unique possibilities. The parsimony is necessary in building certain models since can lead to us overvaluing or undervaluing the model due to probable overfitting. The only issue with this approach is that models with fewer assumptions can lead to us overlooking an important variable within our analysis. This can give a wrong perception of data and give inaccurate assumptions about the model. An example of Ockham’s razor in the business world would be in the gaming console industry. When a consumer decides on which console to prefer for instance PlayStation or Xbox. Ockham’s razor model would take simple answers rather than more complicated answers. A simple answer would be design, graphics, and performance are better than others. With more complicated answer could be how the console is programmed and how performance allows it to run big high-definition games for hours with minimal lag time than others. Or that one offers a better overall customer experience with more available games to play and has great customer support. Choosing the simple answer gives us a narrow scope within our analysis to see why customers prefer one console over the other. This loss of information can impact the business, not allowing them to improve upon their product in the further by giving them what customers desire.
ReplyDeleteGreat example Tyler. This is the stuff of complex decisions. Your point about overlooking assumptions is key to the insurgent threat they might pose, even though they are out of current scope, are seeming of less importance, or just plain meddlesome.
DeleteWith Ockham's razor and the concept of parsimony, I believe that it is definitely applicable in the era of artificial intelligence and machine learning, but they must be applied cautiously. Simplicity is great and idealistic, but principles of this are not very applicable to today's technology. I believe that there are foundational ideas which parsimony is integrated however.
ReplyDeleteAn example can be older and pre-established coding. There was extreme simplicity. Making the gamer pacman was very simple and it required simple coding. Cost of this game in today's world is what a top college coder would be able to accomplish. Now that technology has increased exponentially, there is better and better competition to catch the audience's attention and end goal, money. This requires a higher amount of cost.
Q: In building models, is Ockham's razor or even the canon of parsimony dead?
ReplyDeleteI do not believe Ockham's razor / parsimony are "dead" because it appears to me, like Copernicus said, our universe seems to prioritize conservation of energy, and as such, the simplest explanation, or in other words, the one that demands the least energy, seems to have the upper hand in explaining reality. On the other hand, universal ideas such as conservation of energy and especially entropy, do not really seem to apply as strictly to humans and our society. There appears to be a completely different set of laws that governs life, compared to the natural laws which govern all the inanimate material in the universe. And so, while Ockham's razor does seem to be upheld among the natural laws, I'd say it's applied to a lesser and more vague sense in respect to life, and especially human psychology and society at large.
Is there something better?
I'm not very knowledgeable on the subject, but it appears that AI is being built upon "neural networks" which mimic our own brains. I'm not sure if this technology goes against the principles of parsimony, but I do think the future of computers and models is based on this technology where nodes are communicating in parallel as opposed to in series like a string of binary numbers. That might not make much sense. I suppose what I'm trying to say is that there seems to be some magic in the design of our minds, which we are now creating in computers, which allows us to go beyond the universal laws of simplicity and conservation of energy.
To me, Ockham's Razor is still an important life principle that can guide humans philosophically to utilizing AI as a tool as opposed to using it as the ultimate truth of all situations. While AI's capacities far outweigh anything the human brain could conjure up, the emotional responses that humans possess (which AI does not currently) allow humans to have an understanding physiologically of what is right for them and others in a given moment: a conscience. This means that it is important to utilize Ockham's Razor as humans to crosscheck the results of complicated models: as humans tend to value emotion in the ways that they live their lives.
ReplyDeleteOckham's Razor and the principle of parsimony still hold significant value in the age of AI and Machine Learning, despite the increasing complexity of models in these fields. In many applications, especially in critical fields like healthcare or finance, understanding how a model makes decisions is crucial. Simpler models, which are often more interpretable, can be more easily explained and trusted by end-users. For example, a simple logistic regression model used for predicting patient outcomes can be more transparent and thus preferable to a more complex neural network, especially when the performance improvement of the more complex model is marginal
ReplyDelete1. In the context of projects, various myths, mindsets, paradigms, and worldviews can significantly impact outcomes, both positively and negatively. I have encountered the myth of perfectionism. On its best days this can drive high-quality results, but on bad days it can cause frustration and burnout. I would say it could be kept because of its desire for excellence but with reasonable time measures. As far as mindsets, there is always the one individual who thinks they can single-handedly drive the project, which I would say is the individual heroic. On good days this can create innovation and progress but on the bad days it can lead to bottlenecks and overreliance one the one individual. This is not necessary in a group environment, unlike perfectionism, it does not pose a big enough benefit. The classic paradigm projects always have at the beginning is the paradigm of linear progression which assumes there will be no set backs or need for iteration. This paradigm can provide a sense of security but on bad days can cause frustration and missed milestones. Although this paradigm is comforting, I would say to topple it because it is necessary that you know there will be a need for iteration and adjustments. Knowing this will cause less stress and confusion when they arrive.
ReplyDelete2. The most vulnerable participants are the ones with the paradigm of linear progression. It's crucial to know challenges will come during a project. overestimating and overreliance on this paradigm leads to being unprepared for conflicts in the process. I think a good leverage point would be to use it as a guide as the project progresses but do not become reliant on it. Based on the application of System Archetypes, there are several traps or patterns that can erode goals or lead to undesirable outcomes. Initially, goals are set high with ambitious targets, but as challenges arise or initial successes are achieved, there can be a tendency to lower expectations or settle for less. This erodes the original vision and can lead to mediocrity or missed opportunities for innovation. We can solve this by reaffirming goals and ensure they stay aligned with the overall vision. Addiction cycles are an archetype that involves a repetitive pattern of behavior that provides short-term relief or satisfaction but leads to long-term negative consequences. Teams or organizations may become addicted to short-term wins or solutions that provide immediate gratification but are unsustainable or detrimental in the long run. This will lead to a cycle of dependency on short fixes which is ignoring the underlying issue. A solution to this can be to focus on the strategies that promote resilience and lasting value. There is another archetype of limiting your success. This archetype involves self-imposed constraints or beliefs that limit the potential for achieving greater success. This can stifle innovation and prevent breakthroughs that could lead to significant growth or improvement. A solution to this could be to foster a growth mindset that encourages exploration of new ideas and approaches.
4. The project called this course is going pretty good. The assignments take time but I'm getting comfortable with the sketch drawings which in turn are helping me with the Vensim model. The new variables/ stocks are a little confusing like work flow (not sure what units) at first but the the arrows we input in the model help visualize where things are going and coming from. There have definitely been some hiccups but overall working through it.
1. Given my limited work experience I find this question difficult to answer. Regardless of how hard some school projects may have been during my engineering undergrad, I did not find myself reaching deep enough to find any of these 4 terms in my endeavors. However, in my minimal construction experience, I have experienced many myths and worldviews as I speak with the construction laborers and operators who build the very roads and buildings all around us. The offer a unique perspective on many things I wouldn’t have thought of in certain ways which is great for opening your perspective, but it also teaches you to not believe everything you hear and to be relentlessly questioning everything you read, hear, and see in today’s world of information. In terms of paradigms, I really can only think of religion, which I don’t follow too closely. Outside of that, I think that you could say technology and construction should soon experience a paradigm shift. The blue-collar way is to avoid tech and do without it as it is untrustworthy and unreliable, but the future is coming and many of the things I have seen being implemented point to a future where the blue-collar worked is exponentially improved with technology to save costs, improve efficiency, and improved overall performance.
ReplyDelete2. My worst-case project experience is unclear to me currently, but if I were to hypothesize I would say the most vulnerable participants I would say it is those who are viewed or categorized as non-essential. There are many personnel added to projects nowadays to accentuate the project team and they view themselves as necessary but those directing the project understand their true value to the project and at any moment they could be dropped off of the team. In terms of a leverage point, I would say the point would be what determines non-essential and essential. If a non-essential employee can train themselves into the essential category then they have used this leverage point to leverage further employment and possible personal benefits.
3. When it comes to both of those, Addiction Cycles occur when a system relies on quick fixes that ultimately worsen the problem. For example, if a company continuously resorts to temporary layoffs to cut costs, it may become trapped in a cycle of instability and reduced productivity, as these short-term solutions lead to long-term issues. Eroding Goals happens when initial high standards gradually decline due to ongoing challenges. For example, a student who starts aiming for an A but settles for lower grades over time. Similarly, in systems, this trap leads to diminished performance and missed opportunities as goals become less ambitious.
4. This project called this course is going well for me. I need to review some of the class materials further to gain a better understanding of some of the topics we are working on but in terms of the models and the Wall posts, I have been doing great. I have found a good system and workflow to get my work done on time and effectively. I had some frustrating moments early on in the class with the VenSimPLE models, but I have since resolved these issues and learned to approach them differently. The time it takes to complete my work varies each week with the topic but I find that reasonable given the varying course content each week. I have enjoyed the discovery of rework through fixing my mistakes in the models and improving my understanding of each of the questions asked each week.