When you start looking for evidence, when no one is asking for any, you will be surprised at what you find. Your view of the world, and your ego will be lying in pieces on the ground by the end of it. I'm not talking about confirmation bias research where you ignore all evidence, which runs contrary to your opinion. The evidence of the effectiveness of karate is that it is terrible, but most martial arts are terrible. The evidence which suggests that it is effective is that it survived being passed down in Okinawa. If it's use got you killed then you're not around to pass on your crappy tradition. It's positively selected as opposed to negatively selected. It's survivor bias in a good way. The fact is that adrenal decay, lack of stress inoculation, and lack of experience in violent situations counts for more than your training. Training is the compromise between lack of preparation and being a violent jerk. Rory Miller says it's luck and instinct, which gets you through most violent encounters, until you've hit about 20 force encounters and then it's luck, instinct and training. We've all heard of the people who survive violence by doing X, Y and Z, but we don't hear about those who did X, Y, and Z who died screaming for someone to save them. Unfortunately, there is this thing called reality and it doesn't cooperate the way we wished it would. We as people tend to attribute our successes to skill, and our failures to bad luck. Hardly ever do we attribute success to luck, and failure to lack of skill, even though this is more likely.
I remember reading an article about UFC training, where the trainer was explaining that fighters could achieve the same results by training much less than what they currently were doing. The training had nothing to do with power, speed or skill. It might have even been slightly detrimental. It had to do with mental toughness, and anxiety management. Do you want to go into a fight confident that you did everything you possibly could to prepare, or do you want to go into a fight just as prepared, but with a mental nagging doubt? When you're only fighting pain and fatigue, mental toughness counts for a whole lot.
Here's where this can get you hurt. If you train to fight your anxiety and become less effective than you're not doing yourself any favors. You will be over confident, and less skilled, which is a bad combination.
What does this have to do with research?
It means we need to know the aim of research, and what people are trying to achieve. If you train like a UFC fighter who is trying to manage their anxiety, you might be less effective in a life or death confrontation. You might be over confident, because you did super hard training and die, rather than being under confident, running away and surviving. Research might be comparing very minute battlefield differences, which made a difference tactically, but not much of a difference to the individual. The Thompson-Legerde tests on caliber, which is usually dismissed, was testing the wounding capabilities of different cartridges not how lethal they were. They already knew that a bullet through your head, spine, or heart was lethal. They wanted to know what the differences were when the bullets hit non-essential organs and tissue. Their conclusions were based on certain rounds allowing a person to bleed out faster than others, but this time is counted in minutes not seconds, because regardless of the round our flesh is elastic and closes the wound and our body starts trying to repair itself and stop the bleeding. A wounded enemy soldier will be taken out of the fight eventually, but this doesn't help those in his immediate vicinity. It only means soldiers who show up minutes later won't have to worry about him. The person who gut shot him is probably just as dead. It doesn't help you individually, but it helps your comrades in arms later on down the line. It has a tactical advantage in war, but has no tactical advantage in self defense where a person needs to be taken out immediately. You don't have the ammunition, time nor backup to lay down suppressive fire while you wait for the wounded to bleed out.
It was also thought that non-jacketed lead projectiles would not function reliably in an auto-loader, which is false, but has nonetheless changed the trajectory of firearms development regardless of this being a more than 100 year old myth. I just fired some non jacketed lead rounds out of one of my rifles just a few months ago, and it worked just fine. They have been relegated to the dust bin of history without even a retest to see if the original reasons and conclusions still hold true.
In the Napoleonic war the British sabre caused horrific wounds, but seldom killed. The French sword killed, but did not produce grizzly wounds. The difference between cut and thrust. A thrust kills, a cut maims. The British sword was feared because of the terrible wounds, which is a huge psychological advantage in war. Great for warfare, but not that great for a duel against a determined foe. The British sword was thought to be more effective. Was it? Maybe for fear factor, but not for killing though fear counts for a whole lot.
In the world of the stock market, research has shown that actively managed portfolios do no better than broad market passively managed portfolios. Throwing a dart at the Wall Street Journal is about as effective as the most advanced stock market analysis, because regardless of method you can't predict the future. Regardless of this there are still many companies that advertise the virtues of their management, and people are happy to hand over their money in fees.
What does this all mean?
It means things are seldom how they appear, and if you want to get to the bottom of things you need to dig deep and prepare for your ego to be destroyed, your anxiety to hit the roof, and for no one to ever listen to you because you're going against what amounts to tradition. They use anecdotal evidence, research out of context and ignore conflicting information, because it makes their tummy feel funny to face reality. If you're tummy feels funny, it means you're learning something great. It doesn't mean it's true, or false it just means you're testing your assumptions.
If you want to succeed, you need to ignore that funny feeling and look at the hard real-world evidence. You need to be okay with the fact that luck might play a bigger factor in success or failure than you are comfortable with and that people might be selling you snake oil, so you can manage your anxiety. You also have to be okay with people vehemently and even violently disagreeing with you, because it calls their fantasy world into question. Even with all of this there is freedom in facing reality. It means you can keep your head down, train, prepare, plan and know that focusing on the aspects of life that you can control directly is the only thing that matters even if you are falling through the rabbit hole.
