Stake logo

Share

Prove It

The Replication Crisis. More and more research is coming out to show just how difficult it is to accurately replicate experiments. How many things we accept as fact are just wrong.

A climate crisis, a virus crisis, a food scarcity crisis…we’re going through a bit right now. Underpinning all of these threats is the thorn in science’s side right now…the replication crisis. In short, more and more research is coming out to show just how difficult it is to accurately replicate experiments. Sure, a study may prove some theory with statistical significance but it’s incredibly difficult to repeat the experiment and get the same results.

Research found 40 of 100 of the top economic research papers could not be replicated. Another study found that most conclusions in economic research are exaggerated by at least a factor of two, and a third are exaggerated by 4x. That being said, economics is fairly accurate. Other fields like psychology see immense difficulty in replicated results. 

The obvious question arises, what are we accepting as fact that simply isn’t true? Let’s run through some commonly held beliefs that may not be so statistically significant. 

The first is the signature “marshmallow test”. You’ve seen it before. Kids are left in a room with a marshmallow and told not to eat it. The original conclusion was that those kids who abstained from eating the marshmallow had greater willpower and were more likely to succeed. 2 decades later, the experiment was more rigorously performed and no correlation was found between high school success and willpower at 5-years-old. In fact, it was found that family wealth was the biggest explanatory variable; poorer children were more likely to eat the marshmallow given the instability and less certainty in their day to day lives.

 

More directly related to the markets, an incredibly influential paper in 2010 found high government debt was correlated with low growth. The findings affected government decision making until it too could not be replicated. In 2013, researchers found coding errors, data discrepancies and an unsound method rendered the results worthless. The revised results found high levels of government debt encouraged growth. 

 

Let’s be very clear, this isn’t anti-research or science. The purpose of this piece is to encourage critical thinking. Especially when investing, questioning what everyone else accepts as fact creates opportunity. On top of that, it highlights just how much more powerful research which has been replicated is compared to single-study findings. 


Want more?

You know what to do

Insights, trends and company deep dives delivered straight to your inbox.


Stake logo
Over 7,000 5-star reviews
App Store logoGoogle Play logo

Subscribe to our free newsletters

By subscribing, you agree to our Privacy Policy.

Stakeshop Pty Ltd, trading as Stake, ACN 610105505, is an authorised representative (Authorised Representative No. 1241398) of Sanlam Private Wealth Pty Ltd (Australian Financial Services Licence No. 337927) ('Sanlam') and an authorised representative (Authorised Representative No. 1241398) of Airwallex Pty Ltd (Australian Financial Services Licence No. 487221) ('Airwallex'). Stake is not authorised by Airwallex under Airwallex’s AFSL to arrange for clients to be issued with securities as Airwallex is not authorised under its AFSL for this purpose. Stake is not authorised by Sanlam under Sanlam’s AFSL to arrange for clients to be issued with a non-cash payment facility as Sanlam is not authorised under its AFSL for this purpose. Stake SMSF Pty Ltd (‘Stake Super’) is not licensed to provide financial product advice under the Corporations Act. This specifically applies to any financial products which are established if you instruct Stake Super to set up a self managed super fund (‘SMSF’). When you sign up to Stake Super, you are contracting with Stake SMSF Pty Ltd who will assist in the establishment of a SMSF under a ‘no advice model’. You will also be referred to Stakeshop Pty Ltd to enable your trading account and bank account to be set up in order to use the Stake Website and/or App. Stakeshop Pty Ltd will also run marketing and promotions to you under. For more information about SMSFs, see our SMSF Risks page.The information on our website or our mobile application is not intended to be an inducement, offer or solicitation to anyone in any jurisdiction in which Stake is not regulated or able to market its services. At Stake and Stake Super, we’re focused on giving you a better investing experience but we don’t take into account your personal objectives, circumstances or financial needs. Any advice given by Stake is of a general nature only. As investments carry risk, before making any investment decision, please consider if it’s right for you and seek appropriate taxation and legal advice. Please view our Financial Services GuideTerms & ConditionsPrivacy Policy and Disclaimers  before deciding to invest on or use Stake or Stake Super. By using our website or service in any way, you agree to our Privacy Policy and Terms & Conditions. All financial products involve risk and you should ensure you understand the risks involved as certain financial products may not be suitable to everyone. Past performance of any product described on this website is not a reliable indication of future performance. Stake and Stake Super are registered trademarks in Australia.

Copyright © 2023 Stake. All rights reserved.