Ep 347 - New Retraction (and What It Means For "Evidence-Based" Fitness)

Interactive Transcript

spk_0 Hello everybody welcome back to Iron Culture. It is me, Eric Trexler, alone, afraid and

spk_0 alone. It's just going to be me this episode. Helms is off doing whatever it is. Helms

spk_0 does, who can say. So I'm holding down the fort here. I already have extremely negative

spk_0 chatter in the live chat. We've got Expanda asking where is my jazz music. That was one of

spk_0 the best things about the podcast being the live jazz music. I tell you what, Expanda,

spk_0 since you were the first person in the live chat here over on the Iron Culture YouTube

spk_0 channel, I'm going to commit to you. I will bring back the intro music. I've made many

spk_0 mistakes in my life. One of them was veering away from that music intro. I am on it. I will

spk_0 make it happen. So folks, as you are settling into the live chat, once again welcome to the

spk_0 show. This is going to be a little bit different of an episode. What I'm going to do today is

spk_0 take you through a recent happening in the world of evidence-based fitness slash nutrition

spk_0 stuff. It was a really interesting back and forth, a really interesting saga, I guess you

spk_0 would say, that took almost two years to kind of happen. So I want to walk through what went down

spk_0 and more importantly after that, what it kind of tells us about the state of science in the

spk_0 world of exercise and nutrition and kind of what it means in terms of how we might move

spk_0 forward in attempt to be someone who is evidence-based. There's all these content creators who make

spk_0 kind of quote-unquote evidence-based fitness content. A lot of people claim to be science-based

spk_0 lifters or take a science-based approach to their nutrition. I think in order for any of those

spk_0 who really carry any weight or mean anything, it means very seriously and rigorously engaging

spk_0 with the evidence and the research that exists and also being critical of that where it's warranted,

spk_0 not only the research itself, but also the processes that are intended to maintain the integrity

spk_0 of that research. So we're going to talk about an interesting saga where basically this paper was

spk_0 published about apple cider vinegar as a weight loss tool. Almost immediately it caught my eye,

spk_0 teamed up with a couple fantastic people. I'll talk about as we go here. We wrote a letter to

spk_0 the editor and the whole thing kind of snowballed from there. And just within the last week or so,

spk_0 that paper was actually retracted. And so we're going to talk through how that process went down

spk_0 the investigation that unfolded what the investigation found. And then again, talking big picture,

spk_0 what does this mean for people who regularly try to translate evidence into practice?

spk_0 But before we get into that, a couple things to get out of the way here. If you enjoy the show and

spk_0 you want to support it, there's many ways to do it. Make sure that you like, make sure that you

spk_0 rate, make sure you subscribe. You know all the good stuff. Leave a review if you like the show.

spk_0 If you don't like the show, maybe don't leave a review. Maybe skip that part. But do make sure

spk_0 that you're subscribed. And of course, if you want to be a massive top tier ultra supporter,

spk_0 make sure you head over to massresearchreview.com and consider becoming a member of the mass

spk_0 community. We put out a research review the first of the month. Every single month we've been doing

spk_0 it since 2017. So we've got a new issue coming out in just a couple days here on October first.

spk_0 And it is a doozy. Here's my sales pitch. If you were a member of the mass community,

spk_0 then you knew about this apple cider vinegar paper essentially as soon as it came out. And you

spk_0 probably did not take it particularly seriously. And you certainly, if you're a member of the mass

spk_0 community, did not actually act upon the information in it. So if you want to be approximately two

spk_0 years ahead of the research curve, become a member of the mass community. One other thing I want to

spk_0 mention, I want to give a shout out to our dear friends at Elite FTS. If you have any needs

spk_0 related to gear or apparel of the lifting variety, make sure you head over to EliteFTS.com.

spk_0 You can use our discount code MRR10. It stands for massresearchreview10 to get 10% off most items

spk_0 in their online store. Elite FTS, if you don't know, it's Dave Tates operation over there in

spk_0 central Ohio. They have been a fixture in the iron game for a very long time making extremely

spk_0 high quality products. And as we've talked about in previous episodes, I've been using stuff

spk_0 from EliteFTS since about, I don't even have access to the email that I would have used to

spk_0 to look up my order confirmation. I think that I've been using their stuff since like 2010,

spk_0 maybe, wrist wraps, straps, you name it. So if you're in the market for any of, you know,

spk_0 your lifting related stuff, bands, I'm sure they've got all sorts of knee wraps and sleeves and

spk_0 you name it, I'm sure they've got it over there. So head over to EliteFTS.com, use the code MRR10.

spk_0 All right, folks, let's go ahead and dive in here. As I mentioned, it was a couple years ago,

spk_0 almost, that this paper came across my desk. It was called Apple Cider Vinegar for weight management

spk_0 and Lebanese adolescents and young adults with overweight and obesity, a randomized double-blind

spk_0 placebo controlled study. And this paper, in short, really caught my attention. It was published in

spk_0 BMJ Nutrition Prevention and Health. And the easiest way to summarize the findings in a nut shell

spk_0 is this paper came out and reported very substantial fat loss effects. Whether you're talking

spk_0 about the change in fat mass or the change in BMI or the change in body weight, the effects were

spk_0 very, very striking for a relatively short-term intervention. And basically, we crunch the numbers.

spk_0 And the effects of Apple Cider Vinegar in terms of weight loss were comparable. And in some

spk_0 cases, depending on what trial you're looking at, better than the effects of ozempic on weight loss.

spk_0 And so that's going to catch my eye 10 times out of 10. Because if you tell me that I can take 5 or

spk_0 10 or 15 milliliters of Apple Cider Vinegar a day, and that's going to give me ozempic-like

spk_0 results without ozempic-like potential side effects. Now, I'm not a person who bashes

spk_0 these GLP-1 weight loss drugs. I actually think they're quite a marvel, pretty incredible

spk_0 as it, you know, as, you know, new medications go. I've been really amazed by, you know,

spk_0 what we've seen in that space in terms of the research and the clinical applications. But,

spk_0 nonetheless, they do, of course, like any other medication have a side effect profile. Some of

spk_0 those side effects, although rare, can be substantial. So if you tell me, hey, we've got this thing,

spk_0 it works as well. The side effects are not nearly as, you know, even though the low probability

spk_0 ones are not nearly as severe. And also, you know, if that's true, then we could instead of having

spk_0 this giant, like, $100 billion industry or whatever it's at these days, we can just use Vinegar.

spk_0 So, for example, the last time I looked into the actual pricing for what it would take to be on

spk_0 like a GLP-1 drug, I saw various things, but, you know, over the last couple of years,

spk_0 depending on when you looked, it might be somewhere in the ballpark of 500 or even up to $1,000 a month.

spk_0 And so, you know, basically, the value proposition is, what if I gave you this Apple cider vinegar,

spk_0 and instead of 500 to $1,000 a month, we're talking about, like, I don't know, $6 a month,

spk_0 I haven't priced out, haven't priced out vinegar lately, but a fundamentally negligible cost

spk_0 with a lower side effect profile. Not only is that a huge game changer, if true, at the personal

spk_0 level, but it also essentially wipes out an enormous, what has essentially become a sub-industry

spk_0 of the pharmaceutical industry. So, it was very surprising to me and to my colleagues that nobody

spk_0 was looking at this with the level of skepticism and scrutiny and saying, why are we still doing

spk_0 ozemic and similar GLP-1 drugs if we are to take this seriously? And I've never gotten a good answer

spk_0 for that. Again, under the assumption that we're taking the findings seriously and literally.

spk_0 So, again, we crunch the numbers and basically what we found is in terms of weight loss,

spk_0 as reported, we were looking at something that was similar, if not in some cases,

spk_0 better than ozemic for weight loss. And essentially, there's two levers you could pull mechanistically

spk_0 to make that happen. One would be a reduction in energy intake that, again, rivals what we see with

spk_0 these potent GLP-1 drugs. The other potential mechanism was, well, maybe this is increasing energy

spk_0 expenditure by far a less probable mechanism. But if so, if you actually crunch out the numbers,

spk_0 then essentially what we're looking at is something that is increasing energy expenditure

spk_0 at rest to a level that we haven't seen since looking at DNP. And the problem with DNP is that

spk_0 sometimes it literally kills people if they take too much because of this dangerous increase in

spk_0 energy expenditure. So DNP is what they call an un-coupling agent as it pertains to proton leak

spk_0 within the mitochondria. And so what that means is basically it causes extra proton leakage

spk_0 within the mitochondria, which decouples that flux of protons from the actual process of generating ATP.

spk_0 So under normal circumstances, that whole process is linked and the movement of those protons

spk_0 within the mitochondria ultimately facilitates the production of ATP. But if there's leakage,

spk_0 if there's excess leakage, then what happens is your body is kind of going through and doing

spk_0 metabolism. It's doing all the preliminary steps, including the stuff that actually increases

spk_0 thermogenesis. So as a byproduct of just doing metabolism to create ATP, our body burns calories

spk_0 and generates heat. But the payoff is supposed to be at the last step, we generate all this ATP

spk_0 to fuel body functions. And so therefore, there's this really nice feedback loop where we burn these

spk_0 calories. We increase this, you know, we have this thermogenic effect where we're generating heat.

spk_0 But because we're creating ATP, then we stop doing so much metabolism because we say,

spk_0 cool, we've got all the ATP we're trying to create. Now we can chill a little bit. But if you have

spk_0 all this extra proton leak, then you're going through all the calorie burning and the heat generation,

spk_0 but you're not generating as much ATP. And so therefore, you do, as I'm used to talking in a cadence to

spk_0 inspire my students to shout out or participate, but I don't hear anyone shouting into my headphones

spk_0 right now. But basically, you know, if you are doing this whole process, these protons are leaking,

spk_0 you're not creating all that ATP, you're going to do more metabolism. And so you're churning through

spk_0 even more calories, generating even more heat. And so what we find is when people take, especially

spk_0 very high doses of DNP, there's this kind of runaway metabolism that causes this massive

spk_0 expenditure of calories, but more problematically, this massive increase in metabolic heat generation.

spk_0 And in some cases, there have been situations where that kind of runaway metabolism and heat

spk_0 generation has led to fatal instances of hyperthermia, even in thermonutral conditions. So people that

spk_0 are just sitting in a normal temperature room who are doing metabolism at such a rate and without a

spk_0 functioning feedback loop, such that they generate a tremendous amount of body heat and ultimately die.

spk_0 So that is kind of the premise of why we had the initial premise of why there was so much scrutiny

spk_0 and skepticism of this paper is, what are you telling me? Are you telling me that this reduces appetite

spk_0 on the level of ozemic or that it increases expenditure on the dangerous level of DNP? And also,

spk_0 side effects reported or if they're where they were extremely minor in nature. So

spk_0 yeah, there were just things that in terms of face validity just didn't seem to make much sense.

spk_0 So that leads to a letter to the editor teamed up with VaHeed Malbooby. I could be mispronouncing

spk_0 his name. We've emailed a million times, but I don't think we've actually chatted or I've certainly

spk_0 never heard his name spoken in the course of conversation, but VaHeed is the lead author.

spk_0 And of course, we teamed up with none other than the James Hethers, who has really

spk_0 carved out a bit of a niche in this space of identifying research for which there is great skepticism

spk_0 and ultimately shining the light on that research. In some cases, it may be warranted to follow

spk_0 up with an investigation. In some cases, James's work has led to retractions of papers. So

spk_0 James is a seasoned veteran in this pursuit of following up on papers that lead to tremendous

spk_0 skepticism and getting to the root of what might be causing the findings that have been observed.

spk_0 So we basically wrote this letter to the editor. It is published on BMJ nutrition prevention and

spk_0 health where we talk about a variety of things. We talk about the magnitude of effects, which

spk_0 certainly caught our eye and raised concerns about biological plausibility in terms of identifying

spk_0 a mechanism that could actually make that happen and not be dangerous. We talked about some data

spk_0 characteristics using a procedure called sprite that gave us some cause for concern in terms of

spk_0 just the way the data were reported. So we presented some graphs saying, you know, if these

spk_0 data in these tables are what they say they are, then these must be some very unusual data

spk_0 distributions that are hiding in the actual data set. And our question was, how would the data

spk_0 come to be distributed that way? It would be very much statistical improbability if not a statistical

spk_0 anomaly in some cases. Of course, we talked about some issues we had with the actual statistical

spk_0 approach that was taken. And again, at the end kind of laid out, you know, our concerns about if

spk_0 we compare these effects against things like GLP1 drugs and DNP or 2, 4, dyneitrophenol, things just

spk_0 don't seem to be adding up here. And you know, the final thing that we kind of mentioned or alluded

spk_0 to in this letter is in the original paper, the authors basically had a data disclosure statement

spk_0 at the end of it that said, all the data for this study have been uploaded as a supplementary file.

spk_0 And so when I first read this paper, I thought, cool, I'm going to dive into the supplementary file

spk_0 and get to the bottom of resolving all of my concern, skepticism, etc. If they uploaded the data,

spk_0 then I'll just take a look and see why these table values look the way they do.

spk_0 But when you go to the supplementary file or when you did originally, when the paper was published,

spk_0 it would just take you to a press release by the journal, which was basically just hyping up the

spk_0 paper itself. And so that is not actually technically data to include a press release that

spk_0 just hypes up the findings. So we also included that. We said, hey, all of our concerns could be

spk_0 alleviated if we could just take a peek at the data, which was allegedly supposed to be made available.

spk_0 So if you could make that available, that would sure be terrific. And then of course, before we

spk_0 wrote this, I should mention we emailed the corresponding author of the paper multiple times asking for

spk_0 the data set, which took as far as we could tell was by accident, essentially not uploaded with

spk_0 the original paper. And so we're just like, hey, it says it's uploaded as a supplementary file.

spk_0 If you could send us the data, we'd love to take a look at it. And none of our emails, we tried

spk_0 multiple times. None of them were eventually answered. So that led to this process of, so

spk_0 I'm going to go through the timeline in a bit. But let's pretend that time went in a linear

spk_0 fashion here. So we published the letter to the editor. And then eventually there was a response.

spk_0 And to be totally honest, the response to the letter annoyed me more than the original paper.

spk_0 So basically they kind of went through and almost addressed some of our concerns, but not really.

spk_0 So for the statistical methodology, they basically defended using T tests instead of analysis of

spk_0 variants with no adjustment for multiplicity, which without getting too technical, basically

spk_0 that perspective lives in a world or some kind of alternate universe where we didn't decide in

spk_0 about the 1930s. It's been nearly a hundred years since we decided that's probably not a good idea.

spk_0 And we should probably at the bare minimum do some kind of multiplicity adjustment.

spk_0 Of course, with this particular study, it goes beyond that. It should have also included a baseline

spk_0 a baseline adjustment, some type of ANCOVA type approach or linear mixed model that has a

spk_0 baseline adjustment. There were a lot of opportunities to do a better approach statistically that is

spk_0 frankly more valid. And they just kind of said, no, we're not going to do any of that. No thank you.

spk_0 For the biological plausibility, they basically said like there's evidence that apple cider vinegar

spk_0 can significantly influence body weight, fat mass, and metabolic markers such as blood glucose,

spk_0 insulin sensitivity, and triglyceride levels. They mentioned that the acetic acid component can

spk_0 enhance fat oxidation or inhibit fat synthesis by increasing mitochondrial activity. But again,

spk_0 none of this really makes any sense when you apply it to the kind of magnitude effect that we're

spk_0 talking about. So if you're talking about without impacting energy expenditure, promoting fat

spk_0 oxidation, well then, okay, we might see a modest increase in fat oxidation at the expense of

spk_0 carbohydrate or protein oxidation. But we're in the world at all these calories go. That is not

spk_0 something that would be answered by that kind of mechanistic justification. They took issue with

spk_0 our use of the sprite analysis and said these kinds of reconstruction techniques in their words are

spk_0 sensitive to assumptions and should not be viewed as definitive evidence of data irregularities.

spk_0 In fact, data simulation models often result in variations that may not necessarily mirror the

spk_0 original data set, which is interesting because if we highlight like hey, these data distributions

spk_0 look odd. The easiest way to rebut that type of comment is to simply plot the data distributions and

spk_0 say nope, here they are. They are not odd. They are very normal. They are very typical. And so that

spk_0 did not happen. And so it was basically like, their response was like, have you ever considered that

spk_0 you might be wrong without giving any evidence to make us believe that we were wrong.

spk_0 Data availability, they basically said like, well, this is more of like an ethics and privacy thing,

spk_0 which I would be open to if the original paper didn't say we are happy to upload our data. Here

spk_0 it is attached to this submission. So if from the jump, they basically said hey, our university is

spk_0 really particular about this stuff for liability reasons. And therefore they're very particular about

spk_0 how, how, when, where, and why we make data available. I could live with that on the initial

spk_0 initial submission to the journal. But if you tell me, hey, it's just attached. Make sure you

spk_0 download it. And then you say, no, I'm definitely not going to let you see it. To me, that's a bit

spk_0 of a problem. And so I think one of the things that made me really annoyed about, about this

spk_0 particular letter is like, so for example, you see here this paper by Daniel Lackins. This was

spk_0 just egregiously misscited. They basically cited it while making a claim that is not at all

spk_0 supported by the paper itself. So that to me was very annoying because it's just like, here's

spk_0 a statement that is not true and not in this paper. But we're going to cite this paper. The reason

spk_0 that gets under my skin is because it's a great paper. I have like a personal level, like fondness

spk_0 for this paper. And to see it misused in that way was particularly grating to me. And then,

spk_0 just making statements that don't hold water whatsoever statistically. It's one thing to

spk_0 misscite that paper. And also the other thing that drove me nuts was we're just pretending that

spk_0 we don't need any kind of post-hoc corrections for multiple T-tests, which again, that was old news

spk_0 in 1941. So unless you're going to substantially rewrite the last nine decades of statistical

spk_0 literature, you kind of have to give me more than like, what if I didn't adhere to like basic

spk_0 best practices? So it bothered me that this letter was written, but it bothered me way more that it

spk_0 was accepted. And so when this was accepted, I wrote a letter, not a letter, I wrote an email saying

spk_0 like, what in the world are we doing here? But I think, oh, my light went off on me. There we go.

spk_0 I think my dear friend, James Heather's, wrote a very funny blog post over on his

spk_0 substack. It's called JamesClaims.substack.com. And he kind of walks through

spk_0 how this all went down. And I love the way that he laid out the timeline. Because I said previously,

spk_0 like for the time being in the interest of just being concise, we'll pretend that time advanced

spk_0 in a linear way. But I can promise you, this stuff all happened super out of order in a very

spk_0 strange way. So as James recounts it in May, 2024, he sees this paper in the way he said, the way

spk_0 he puts it, it was full of unlikely features, contradictory data and generalized silliness.

spk_0 The good way to put it. So then, you know, me and the heed have been kind of talking about this

spk_0 paper. We see that he's talking about this paper. We kind of team up and decide, let's,

spk_0 let's do this the right way. The right way to do this is first, you email the corresponding author

spk_0 for the data, that is procedurally the correct thing to do. And with multiple attempts and ample

spk_0 time, if you never hear a response, then you write a letter to the editor and you let the journal

spk_0 handle it. So the, like I said, the paper was published in May. We sent them the letter to the

spk_0 editor in June. And it was published in October. Or I'm sorry, it was accepted in October.

spk_0 Now, in November or December of 2024, I forget which one, that was when they published their

spk_0 rebuttal to our letter to the editor. But major issue, our correspondent, our letter to the editor

spk_0 was not published. So basically, the journal published their response to our letter to the editor,

spk_0 which to readers did not exist. So they didn't cite our letter to the editor. And they were basically

spk_0 arguing with ghosts. They were, they were arguing against a thing that in the eye of the reader

spk_0 did not exist and could not be accessed. So then we told the journal, like, hey, you, you kind of

spk_0 have to like publish our letter as well. So people can see what's going on here. What was really

spk_0 crazy about it, James lays out the timeline numerically. And so basically, this paper got retracted

spk_0 559 days after it was published, 504 days after James first published his criticisms, 464 days,

spk_0 after we submitted our letter to them. And what's really weird and was annoying was that at first,

spk_0 they were going to charge us over $2,600 US dollars to submit that letter to the editor,

spk_0 which I don't think people know how this works. But like, with open access journals,

spk_0 you have to pay to publish your paper, which is a whole conversation unto itself that I could

spk_0 gripe about forever. But what was particularly crazy about that was we were annoyed because this

spk_0 paper got published in our view shouldn't have. We didn't think it received a proper and appropriate

spk_0 level of scrutiny in the peer review process. So now we're going through the trouble of writing,

spk_0 you know, compiling this evidence, doing these analyses to inform the journal, like, hey, for your

spk_0 sake, to the extent you value your reputation, you should act upon this. And at first, like, I had

spk_0 to work really hard to get them to not charge us $2,600, 20, 26, 52 for doing what they should have

spk_0 done at the beginning for them, which was just extremely annoying. So I think James went back

spk_0 and counted. And by his count, I'll take his word for it. There were 12 emails back and forth to

spk_0 try to figure out whether or not we should have to pay $2,600 to correct what they had mistakenly

spk_0 done. James estimates that this process took approximately 100 emails in total. Zero is,

spk_0 by the time they retracted it, they actually didn't tell us first. They just did it. And then I

spk_0 think like two days later, I finally got an email saying, oh, by the way, thanks for sending 100

spk_0 emails back and forth. We decided to retract that a few days ago. So I kind of buried the lead,

spk_0 but eventually at the end of the day, we submitted this original letter to the editor.

spk_0 There was all the shenanigans about for some reason it was accepted but not published. And then

spk_0 they responded with this letter to the ghost remnants of what our letter was. Then eventually, they

spk_0 got our letter up. But all of this essentially plus the emails that I sent after this rebuttal to

spk_0 our letter where I was like, none of these claims are appropriate and some of them are just flat

spk_0 wrong. Basically, none of them were sufficient and some of them were just flatly not true.

spk_0 Basically, what they did was initiate some kind of statistical

spk_0 investigation that we were not a part of. I have no idea who did that, the investigation itself.

spk_0 So after that investigation on the 23rd of September, they go ahead and submit this

spk_0 retraction. And in the supplementary material, that's where I got a few laughs because they

spk_0 uplet up. They included a supplementary file that kind of talks through how they did this,

spk_0 like they did a very formal write up that whoever did the statistical review of what they did

spk_0 and what they found. And going through that, I got some serious kicks out of that. Because at the

spk_0 end of the day, we essentially claimed in our letter we're like, hey, these data distributions look

spk_0 really weird if you reconstruct what they would be based on the summary data that we're seeing in

spk_0 these tables. And then in their rebuttal, they were just like, no, sometimes people mess up when

spk_0 they try to reconstruct this stuff and you're making assumptions and stuff like that. So just

spk_0 leave it alone, basically what it came down to. So according to the statistical review,

spk_0 they looked at the results reported in table three of the article. This was the anthropometric

spk_0 outcomes, some of the key variables. And they said of the 128 values reported in table three,

spk_0 none of them agreed with the supplied data set. So the researchers gave these statisticians a

spk_0 data set that general basically made them the statisticians tried to reconstruct the tables in

spk_0 the paper. And they went zero out of 128. We're trying to reconstruct the values in table three.

spk_0 For table four, there were 128 values in table four. And they went zero out of 128 for trying to

spk_0 reconstruct those values. So to me, that makes it all the more rich that in the rebuttal to our

spk_0 letter that basically said, like, hey, we have some concerns about like what these data must look

spk_0 like based on the summary data in these tables. Basically, the outcome is, yeah, well, that's because

spk_0 none of the values in the tables actually match the underlying data sheet. So it turns out that

spk_0 the assumption driven reconstructions that we tried actually had some veracity. And indeed,

spk_0 there was more to get to the bottom of in terms of why those table values look so odd. It turns out

spk_0 they were not reflective of the data, which is an interesting turn of events. So I'm not going to get

spk_0 too much further into the weeds. All this stuff is documented. You can read about it if you like.

spk_0 But I think James did a great job kind of writing about how annoying this whole process was,

spk_0 how long it took. And the last thing that I want to highlight is one of the reasons that this

spk_0 matters so much. I can't really see at one point in time, I remember checking and seeing that even

spk_0 though this was a fairly new paper when I checked, it was, I believe, the most red paper in the

spk_0 history of this journal, which means it just caught on like wildfire and was immediately kind of

spk_0 circulating and making the rounds. So there's something called alt metrics, which basically seeks to

spk_0 quantify how much a research paper gets shared posted on the internet commented about things like

spk_0 that. And so the alt metric, basically, if an alt metric value is high, basically means this thing

spk_0 went viral. People saw it, people talked about it, people were interested in it. And at the end of

spk_0 the day, that means essentially it had a big impact. It got to a lot of people who saw here are the

spk_0 findings of a peer reviewed paper. And in theory, this is science. This is what you can take to the

spk_0 bank and believe and apply. So when you look at the alt metrics, the details here say this paper was

spk_0 in the top 5% of all research outputs scored by this alt metric system. So of all research, we're

spk_0 talking about top 5% tile. It says this is one of the highest scoring outputs from this source.

spk_0 It says this had a high attention score compared to outputs of the same age, put it in the 99th

spk_0 percent tile, basically meaning among newer studies, this thing really caught fire and lit pretty viral.

spk_0 And so it was mentioned by 193 news outlets. And you know, all down the list, this got a lot of

spk_0 attention and was shared widely. Not, and I don't care about the clicks, I care about the message.

spk_0 Right. This paper was shared broadly as scientific evidence for what you ought to do.

spk_0 And the reason that I bring up those numbers is I've written a previously I wrote a mass article

spk_0 where I basically said that retractions aren't enough. Basically, retractions don't work in the way

spk_0 we hope for them to work. And basically what I meant by that was, you know, there's all these

spk_0 paper. I think I was writing specifically in the context of a really widely read paper about,

spk_0 it was about ice baths, right. And so there's this paper about ice baths. It got spread all over

spk_0 and people were reading it like crazy. It was wildly popular and eventually got retracted because

spk_0 it was rubbish. But it got retracted after it reached like millions and millions and millions of

spk_0 people. I think at one point I tabulated that the results were out there conveyed as scientific

spk_0 truth to I think I estimated about five six million people. And so then you publish this little

spk_0 retraction notice and like 15,000 people hear about it. And so it's like, well, six million people

spk_0 heard this was true. And then 15,000 said, actually, a year later, never mind, forget about all that.

spk_0 There are so many people impacted by these findings who hear them, accept them as true. They're not like

spk_0 chronically online. And even if they are, they're not chronically tapped into, you know, the British

spk_0 medical journal, family of journals. And so the reality is when a study like this gets published

spk_0 and it goes viral and it gets shared and people talk about it on podcasts and people write articles

spk_0 about it. That message gets out to people and the retraction message statistically just doesn't.

spk_0 It doesn't get out to people. And therefore, the impact of saying, oh, well, don't worry about it.

spk_0 If we mess up, we'll just retract it. That's not good enough. It simply does not actually do what

spk_0 we needed to do, which is go out and reach the people who were misled by this work, whether intentional

spk_0 or unintentional, doesn't really matter. We need to convey the message to people that, hey, this

spk_0 is not actually a representation of what we know scientifically on the topic. And that is a huge

spk_0 issue that is really hard to rectify. It's really difficult to address, what are we going to

spk_0 ultimately do about that? How do we have a paper like this that is like super viral and reaches

spk_0 all these people? What is the mechanism we have to actually reach out and correct the record and

spk_0 give people a better understanding of what actually is true and what actually they should be doing

spk_0 in terms of their health, their fitness, their nutrition, whatever. And right now, I think the

spk_0 reason that it's important to highlight the timeline and the procedure is like, ultimately,

spk_0 at the end of the day, this just doesn't work very well as a system. It takes way too long. It should

spk_0 not take the better part of two years to rectify something that, as we talked about in our timeline,

spk_0 it took a matter of days to say, hey, this stuff doesn't add up. And so if you want to actually

spk_0 try to get corrected information out there, even though you noticed it in a couple days,

spk_0 put together your very robust airtight justification, which ends up being more than vindicated on

spk_0 a statistical review. You need to, under the current system, settle in for what is essentially going

spk_0 to be about two years. It's going to take about 100 emails. It's going to be sometimes tense,

spk_0 back and forth. Sometimes researchers won't answer you at all, which is actually a reasonably

spk_0 pleasant way it could go. Sometimes they get very defensive and there have been notable situations

spk_0 in the past, not in this paper, where people start getting litigious, threatening to sue you for

spk_0 poking around, or a variety of unpleasant things you have to deal with. Sometimes the journals will

spk_0 act like they're going to charge you two or three grand to have the privilege of helping them correct

spk_0 the record. So it's just overall a really, really bad system. And I don't bring this up just to

spk_0 complain, although it may be warranted in this case. But I bring this up because it brings me to a

spk_0 few conclusions that I think are worth addressing to the iron culture audience who, by and large,

spk_0 I think views itself as being science-based or evidence-based. First and foremost, a conclusion

spk_0 to walk home to take away or to take home from this whole saga is, turns out, apple cider vinegar

spk_0 does not work better than ozempic. It does not increase energy expenditure more than DNP.

spk_0 Who knew? Who knew? But it turns out that that's not the case. Another conclusion that I think should

spk_0 be an absolute, an absolute travesty and a scandal is that this whole saga that I just explained

spk_0 is actually an example of the system working. So I'm sure James won't mind me sharing this,

spk_0 but James has tapped into that world of people who kind of do this and kind of shine a light on

spk_0 research that they believe may be potentially unreliable and then investigations follow up and

spk_0 sometimes there's retractions and things of that nature. And believe it or not, as tedious and

spk_0 protracted and annoying is that whole process sounded, this is actually a smooth and fairly

spk_0 expeditious run through the process. In many cases, this process will take way longer.

spk_0 The people who are just trying to get the scientific record corrected often end up getting in

spk_0 situations where people threaten to sue them or actually sue them. And so as crazy as it sounds,

spk_0 this process, if you had no context for it, you would look at it and say, well, that was a mess and

spk_0 took way too long and was way too hard on the people who were just trying to flag research that

spk_0 was clearly unreliable and proved to be unreliable. But at the end of the day, that went well,

spk_0 relative to the norm, which is a problem for not just for researchers, but for people who use

spk_0 research because who knows how many papers in the next 600 days are going to get retracted that

spk_0 are currently being believed and put into practice, put into application. Another conclusion that's

spk_0 really important to highlight is this is going to get a lot worse, I think. I am not making any

spk_0 claim about why there's a massive degree of disagreement between the data spreadsheet and the

spk_0 tables in the paper that we've talked about up to this point. I don't know how that came to be,

spk_0 and therefore I won't comment on how that came to be. But I will say this, getting away from that

spk_0 paper and getting into the broader context, okay, I want to be very clear about that.

spk_0 It has never been easier with AI tools to just generate a fake data set that doesn't look

spk_0 quite as silly as what people may have done in the past. If a person wanted to fabricate a data set,

spk_0 they could do that using AI tools in a way that is not at this point, I don't think it would be

spk_0 full proof, but it would be a lot more convincing than in the past. Now people who actually know how to

spk_0 use numbers and stuff could actually, even without AI tools, they could be up to far more

spk_0 malicious things in a much more convincing way, but even people who aren't that good with numbers,

spk_0 I think if they've felt inclined to fabricate a data set or embellish a data set, I think that's

spk_0 easier than it's ever been. And on top of that, it's easier than it's ever been to very quickly

spk_0 generate a paper using large language models. And then you get reviews in from the peer reviewers,

spk_0 and it's never been easier or quicker to quickly generate a response to those reviewers using

spk_0 large language models. So what I'm getting at is clearly we've never been good as a world of

spk_0 publishing scientists at actually doing this or trying to stay on top of doing it in a prompt

spk_0 or expeditious way, but in a system that is likely to get flooded with a much higher number of

spk_0 total submissions and by extension unreliable submissions in the years that follow, I expect that

spk_0 this is going to get a lot worse and a lot harder to, a lot harder to, to ultimately correct

spk_0 or police. So yeah, I'm very concerned about just the sheer volume of kind of flooding a system

spk_0 that already seems to be working at a rate and with a success rate that seems to indicate that

spk_0 it's already not doing so well. I think flooding it with more papers more quickly is just going to

spk_0 stress and severely challenge an already very challenged system. So now getting into stuff that's

spk_0 actually practical here, what do we do with this? Why does this matter? When we say that we're going

spk_0 to be evidence-based or science-based as it pertains to our training, our exercise, our nutrition,

spk_0 our health, whatever, when you say I'm evidence-based or I'm science-based, at the end of the day,

spk_0 that is really a declaration of your perspective. That's a statement of epistemology, meaning I am

spk_0 declaring what kind of evidence I find to be valuable, what kind of evidence I find to be useful,

spk_0 I'm implying a certain hierarchy of different types of evidence and how I wish to apply them.

spk_0 I am signaling and declaring that I think the scientific method is a good idea. I think that

spk_0 objective scientific research is generally a good way to learn as much as we can about the world

spk_0 around us and what we observe in this world. So that's pretty much what you're saying. It is not

spk_0 saying once something hits PubMed, I'm locked in, whether I like it or not, whether I agree with it

spk_0 or not, whether I think it's reliable or valid or not, once it hits PubMed, it is officially

spk_0 science and I am bound to it. That is not what being evidence-based is and frankly, that is a very

spk_0 poor approach to trying to be evidence-based or science-based. In reality, a person who is

spk_0 evidence-based realizes that once a paper hits PubMed, once a paper hits that journal's website,

spk_0 that is where the actual evaluation and implementation of evidence begins, not where it ends.

spk_0 I know this has been said a million times, so I won't belabor the point. But being evidence-based

spk_0 or being science-based does not mean I am forever required to have a PubMed link to justify

spk_0 everything I do. Someone who's evidence-based, as it pertains to nutrition or training, is someone who

spk_0 says, I have my own experiences and those are valuable. I have an understanding of broad

spk_0 recommendations and guidelines that are widely accepted by authorities in the space

spk_0 that is also a source of information. Aside from those, also the experiences of people who have

spk_0 accomplished a lot in the endeavors that I'm interested in, that is also valuable information

spk_0 and in addition, peer-reviewed, well-controlled, scientific studies are a very, very good source of

spk_0 information as well. An evidence-based practice is combining all of those things.

spk_0 But what's really important is that for every type of information you wish to combine

spk_0 into your evidence-based practice or your science-based practice, there needs to be a level of

spk_0 evaluation that happens before it's all entered into this big mixing bowl of information we're

spk_0 going to act upon. So what that means is, with your own observations and anecdotes, you should be

spk_0 constantly challenging yourself on those. A good example is when I was in college, I remember my

spk_0 freshman year, I started taking an herbal dietary supplement that was marketed for muscle growth.

spk_0 I really believed in it because the marketing was good and I was a more trusting person at that

spk_0 stage in my life and a much more optimistic person as well. So I started taking this herbal supplement

spk_0 and I grew like a weed. I got very big, very strong, very quickly and I had convinced myself,

spk_0 very strongly, man, this supplement is incredible. I can't believe the gains that it is giving me.

spk_0 In hindsight, and I've recognized this shortly after this kind of honeymoon period believing

spk_0 that this supplement was just incredible, I looked back at it and I said, you know what, I'm a

spk_0 never had this level. I lived like a hundred yards from my gym if that. It was open all the time.

spk_0 There was even a gym in the dorm basement that was, I mean, I don't think the university knew it existed.

spk_0 So it was unofficially open 24 hours a day and you can just go in there and it was actually

spk_0 like reasonably well equipped. So all that is to say, I had never enjoyed this unfettered level of

spk_0 access to really solid training opportunities. I never had this much free time because I probably

spk_0 should have been going to more classes than I was. And I also was on a meal plan. So I was steps

spk_0 away from a dining facility, which was all you can eat. And honestly, if you followed my content

spk_0 in any capacity, you know that I am not a food snob. So I was in heaven, like all the food as far

spk_0 as I could tell was incredible. So in hindsight, I looked back and I said, was it really the supplement?

spk_0 Or is it possible that you just really like lifting all day, sometimes multiple times a day,

spk_0 and eating unlimited amounts of reasonably high quality food? Could also that have had something

spk_0 to do with these gains you were making? So all that is to say, when you're leaning on your own

spk_0 experiences and your anecdotes, you still have to really challenge and interrogate those before you

spk_0 kind of put those in the mix in terms of dictating decision making. And we see this all the time as

spk_0 well when people are trying out different programs and they convince themselves, you know, this program

spk_0 works for me. This one doesn't therefore, I guess I only respond to low volume stuff or I only

spk_0 respond to high volume stuff. You start digging deeper. And in many cases, you'll challenge

spk_0 the initial assumptions that you make as you're kind of making these preliminary observations.

spk_0 Same thing goes when we're incorporating anecdotes from other people. If our personal anecdotes

spk_0 sometimes end up being unreliable, then certainly we have to question anecdotes from people for whom

spk_0 they're talking about observations, experiences that we didn't even see with our own eyes, right? So

spk_0 of course, we need to take those anecdotes and observations with a grain of salt because we couldn't

spk_0 possibly, we could not possibly have the whole story when someone is kind of walking us through it.

spk_0 We don't know if they say, oh, I did this program. I had the best gains ever. We don't really know

spk_0 what their diet was like day to day, their stress levels, you know, was it just this incredible

spk_0 opportunity like I was talking about to have low stress, low work obligations, just train in all day,

spk_0 eating all day. Who knows if they were using other things to supplement their gains. So other

spk_0 people's anecdotes, we have to obviously scrutinize very carefully. Expert consensus is usually

spk_0 measured, but it oftentimes represents a pretty good snapshot of where the evidence is at a

spk_0 given point in time. But those types of consensus statements and papers are certainly not infallible.

spk_0 Sometimes they will be led by people who have a lot of sway on the final report or the final

spk_0 publication. And they may have some kind of pet theories that get treated with an inappropriate

spk_0 lack of scrutiny. And so you will see some unreliable things sneak in from time to time. And then,

spk_0 of course, those things tend to expire at the end of the day. You know, sometimes years pass,

spk_0 we learn more about the topic and ultimately things that were believed to be true are either

spk_0 severely updated or in some cases even overturned depending on where the research leads. So

spk_0 all these sources of information require a great deal of scrutiny and skepticism. And at the end

spk_0 of the day, published peer-reviewed scientific research is no different. Now, obviously,

spk_0 I live a life of science. I'm in the lab every morning dealing with my urine samples,

spk_0 getting the machine going for the day for our doubly labeled water analysis. I publish frequently

spk_0 in scientific journals. So I don't want to give the impression that I have become some kind of

spk_0 cynical, nihilistic person who believes that there's no point to all this. If that were the case,

spk_0 I would probably be having a severe identity crisis and or I wouldn't say a midlife crisis,

spk_0 but I also wouldn't say a quarter life crisis. So a third life crisis maybe. I'd be going through it

spk_0 if that was really where I was at mentally. That's not where I'm at mentally.

spk_0 However, I do have a high level of concern that a lot of stuff is getting published that

spk_0 shouldn't. And when the process of rectifying that takes 600 days and maybe a couple thousand

spk_0 dollars and maybe 100 emails and you're treated like a nuisance the whole time you're doing it,

spk_0 which dissuades you from doing it in the future. It's a very thankless thing. It's a very

spk_0 inefficient process. I'm very concerned that if we continue as scientists to pretend that

spk_0 that system is adequate, we're going to continue to publish first and ask questions later.

spk_0 We're going to continue to publish unreliable information. That unreliable information is going

spk_0 to reach way more people than the eventual retraction notice. And ultimately we will be contributing

spk_0 as scientists to a system where people will start to correctly take note of the fact that

spk_0 I guess now with published science, I'll say now even though it's always been true, but

spk_0 you have to take it with a big grain of salt. Now that's always going to be true, but I think

spk_0 now more than ever as a scientific community, we have to be really cautious about how hard we're

spk_0 working to earn and preserve the trust of the non-scientific community. I have concerns about

spk_0 that and then when it comes specifically to the fitness space, I just think it's really critical that

spk_0 we have to be very, very skeptical of research that gets published. We have to scrutinize it very

spk_0 carefully. It's okay to really dig into the research, reach out to multiple content creators in

spk_0 the space that you trust and say, hey, this seems weird to me. What do you think? This seems odd to

spk_0 me. What do you think? These two papers seem like they are fully contradictory. What do I make of this?

spk_0 At the end of the day, I think that that dialogue is going to be really critical. If we want to have

spk_0 like a metaphorically healthy, robust science-based or evidence-based lifting community that's using

spk_0 research to inform our training and our nutrition, unfortunately we cannot rely on the journals themselves

spk_0 to play defense against some of these unreliable papers and prevent them from getting published. We

spk_0 need to have what is often called secondary or post-publication peer review happening within the

spk_0 lifting community. We have to have chatter and discussion and debate and sometimes that debate

spk_0 may get animated, may get tense depending on people have their pet theories. This paper supports

spk_0 that paper rejects it. People get into some heated debates over this stuff and of course we should

spk_0 be able to do it in a very measured, reasonable way. But that more than ever, I think that debate,

spk_0 disagreement, conversation, that is really critical for making sure that the evidence is ultimately

spk_0 bolstering what we're trying to do with our training and nutrition rather than hindering it or

spk_0 getting in the way. We looked to research for increased clarity more so than we looked to it for

spk_0 confusion. Sometimes confusion is an intermediate step we have to take. Sometimes research findings

spk_0 at first don't make sense. Then we replicate them and replicate them and replicate them and we

spk_0 realize, oh, we had it all wrong from the start. We shouldn't be quick to outright refute everything

spk_0 that initially doesn't really make a ton of sense. But we have to remember to keep those good

spk_0 practices in mind, which is one paper or a couple papers with some indicators that they could be

spk_0 unreliable, could be shady. Those things come out. We don't need to immediately rush to application.

spk_0 Sometimes we do have to remember to slow our horses a little bit to kind of take a slow measured

spk_0 approach and say, let's wait to see if this actually gets replicated. Let's try to make sense

spk_0 of this in the context about other literature and sometimes being the slow one to move ultimately

spk_0 pays off because you don't end up chasing these random things that proved to be unreliable.

spk_0 So just within the last couple of years, I've kind of written pretty extensively about these two

spk_0 papers that have been ultimately retracted. One of them was on ice baths. The other was on apple cider

spk_0 vinegar. Now, what is the probability of absolute tragedy if you took those papers at face value?

spk_0 At the end of the day, it pretty low probability, right? There have been rare but very severe

spk_0 adverse effects associated with some people doing ice baths that really shouldn't have been.

spk_0 They were not in the cardiovascular condition to do it or they were doing it dangerously out in

spk_0 water that was too deep. You'll see rare instances where people will have cardiac events or

spk_0 accidental drownings from certain approaches to cold water like ice baths or cold water immersion.

spk_0 But by and large, usually the downside of doing cold water immersion is that you wasted 15 minutes

spk_0 of extreme discomfort for no tangible output, which is annoying but again, not a tragedy.

spk_0 With apple cider vinegar, maybe you had to take a shot for a while of something that you didn't

spk_0 really enjoy. If you're taking it as a shot non-dioluted, maybe you did a modest level of negative,

spk_0 a modest level of harm to your tooth enamel because of how extremely acidic it is.

spk_0 But again, we're not talking about outright tragedies here. But I think the real downside of

spk_0 especially with these kind of hot topic papers that keep getting published and then retracted is

spk_0 at a certain point, we're going to lose as a scientific community the trust of people.

spk_0 And at a certain point, it's like, well, if we keep doing this, do we deserve it in the first place?

spk_0 And the answer is probably no. So that's why not to turn this into some kind of advertisement,

spk_0 but that's why we, the folks of the Mass Research Review, and many research reviews like us,

spk_0 we find it so important to, and it's the reason we do this podcasting, it's the reason we do research

spk_0 reviews is unfortunately relying on the journals to do their own business and do their work effectively.

spk_0 It's not the best approach these days unless you're willing to be wrong for a couple of years and

spk_0 then say, oh, okay, never mind. So I want to extend this as a challenge to the science-based or

spk_0 evidence-based lifting community talk. You already do, but keep it up. Talk early, talk often,

spk_0 talk regularly with zeal when it comes to new research that comes out. It is important that we

spk_0 kind of work as a whole community to stress test these new findings that catch our attention.

spk_0 Approach them with open-mindedness, but also with skepticism and scrutiny. Have those conversations

spk_0 about what we think we can or cannot take away from this research and ultimately, you know,

spk_0 put out a variety of perspectives about where the research currently stands, which paper seem

spk_0 to be reliable and which ones seem not to be. That's not to make it some kind of huge witch hunt

spk_0 where every paper is presumed to be guilty until proven innocent with this extreme level of skepticism,

spk_0 but we got to keep our eyes open and we got to, whatever new paper comes out, if it absolutely

spk_0 stuns us, those are important conversations to have because, you know, one of the things I'm

spk_0 really happy about, one of the things I'm not going to lie, I'm really proud about it is

spk_0 with that ice bath paper and with this paper, people near me were not taking it seriously,

spk_0 and they save themselves the trouble and the confusion about a year or two before there was

spk_0 any kind of indication coming back from the journals. What frustrates me is that even with our

spk_0 best efforts, we know that, like I said with the ice bath paper, that reached 6 million people,

spk_0 and I'm not reaching 6 million people to remind them, hey, actually that wasn't legit. With the

spk_0 Apple cider vinegar paper, again, we can look at those alt metric scores and say, okay, we saw what

spk_0 193 news outlets share the original paper. Are we going to have a 193 that share the

spk_0 retraction notice? Of course not. We'd be lucky to get to 30, I think. I'd love to be wrong about

spk_0 that, but I've seen a handful and these kinds of, you know, a retraction notice does not have a

spk_0 long tail in terms of the news picking it up. Basically, you get a little flash in the pan,

spk_0 and it's probably going to dry up pretty soon here. So if I had to guess, by the time it's all said

spk_0 and done, the ratio of news outlets to cover it versus the original paper versus news outlets to

spk_0 cover the retraction notice, it'll probably end up being like 193 to 25-ish. And usually the news

spk_0 outlets that cover the retractions are not, they're not the ones that people actually,

spk_0 they're not the ones that have massive readership. You know, you get there like really big,

spk_0 the really big kind of lay press sites that will say, hey, apple cider vinegar to lose 15 pounds

spk_0 in 12 weeks, but then it's like some some blog that's like dedicated to like science, literacy,

spk_0 and statistics that'll eventually cover the retraction. And like what they do is very important

spk_0 work and they do it, you know, a lot of those outlets do it quite well, but their readership is like

spk_0 100,000 of what you get in some of the on lay press outlets that will cover the flashy news story

spk_0 when it appears to work, but they never really quite get around to putting out a new article saying,

spk_0 by the way, never mind, forget all that. So briefly, I'm going to check in on the chat here.

spk_0 Like I said earlier, my mistake, my personal apology to the Iron Cult, I will bring back the jazz

spk_0 music. It will be, I don't know if it will be for next episode, but it will be coming, bear with me,

spk_0 folks. See, chimed in. First of all, the comment, flaky helms. I don't know any other way to put it.

spk_0 This is a flaky man. He cannot be relied upon, but we do wish him well in whatever he's up to as he

spk_0 skips yet another episode of Iron Culture. Another comment here from C says, I just lost 20 pounds

spk_0 the hard way if only it were this easy. So yeah, I mean, it would be great if again, instead of spending 500,

spk_0 thousand dollars on ozemic and, you know, running the risk of the low probability, but, you know,

spk_0 substantive adverse effects that could occur. Yeah, the shot of Apple cider vinegar sounds pretty good

spk_0 compared to that, but unfortunately, it doesn't work out. A question was, where did Apple cider vinegar

spk_0 get such a cult following to begin with? That I honestly do not know. All I know is it's been bouncing

spk_0 around the kind of fitness wellness world for quite some time. I do believe that it varies

spk_0 regionally. There are some regions of the world where it has a lot more cultural purchase than

spk_0 others, but in any case, it is big. And I want something kind of gets that that reputation for being

spk_0 this kind of like wellness-oriented remedy for lack of a better term. It really takes a long time

spk_0 for that to go away. So I think Apple cider vinegar is going to be around for a while. I wouldn't

spk_0 bother using it for any outcome, really, but I don't expect it to go quietly into the night just

spk_0 because this one paper was retracted. Got a comment about AI paper mills. Yeah, paper mills have always

spk_0 been a thing. These are basically entities, you know, small collections of people who kind of just

spk_0 churn out these academic papers that are very low quality, sometimes nonsensical, but they just kind

spk_0 of churn them out and get them submitted to journals and just try to push them through the publication

spk_0 process. You might be wondering why, but there's all sorts of weird incentive processes or incentive

spk_0 structures that would lead one to pursue that process. But back in the day, if you were going to do

spk_0 a paper mill, someone had to actually sit down and write the thing. Even though it was rubbish, mostly,

spk_0 they still had to do it. But like I was saying with my concerns, you know, it seems that

spk_0 it's never been easier to run a paper mill. That's for sure. And frankly, you probably don't even

spk_0 need as much of a paper mill these days if your whole goal is to just get something out the door

spk_0 with AI tools. It's never been easier. And I do fear the short-term impact that will have on journals.

spk_0 Now, and not just journals, but the scientific literature is really what I care about.

spk_0 Having said that, some optimism, theoretically, these journal publishing companies who are worth

spk_0 very large amounts of money should be able to find creative ways to leverage AI tools for the good

spk_0 of the journals' rigor and processes. So I certainly wouldn't discount that, but I definitely think

spk_0 it's a game of cat and mouse. And I think the mouse is winning right now relative to the cat.

spk_0 And I hope that the cat in this metaphor, the publishing companies, the journal publishers,

spk_0 I hope that they will catch up in a meaningful way. And I do believe that there is promise for

spk_0 those tools to be developed, but they're probably already developed. They need to be implemented at

spk_0 scale in a way that's actually useful and effective. Another comment here, a question I should say,

spk_0 how are the incentives about doing replications versus novel research structured in academia?

spk_0 It's getting better, but there's much more fanfare and prestige and therefore incentive

spk_0 to publish totally novel original research more so than trying to replicate something that has been

spk_0 found previously. I think that compared to all other times in human history, it's never been

spk_0 more appreciated to do replication research, but we still certainly have not gotten to the point

spk_0 where that is viewed as an equal contribution to the literature by any means. And that's ultimately

spk_0 a problem. If we are of the belief that the value of science is adding knowledge, improving the

spk_0 knowledge base of the published literature, in theory, we would incentivize this kind of work

spk_0 of identifying papers that that were at retraction. In theory, removing an unreliable paper from the

spk_0 literature, I think should be just as valuable as adding a new reliable paper to the literature.

spk_0 Replicating a finding should be viewed as just as valuable as an original finding. At the end of the

spk_0 day, they're all going to be little data points on a meta analysis. And frankly, when you look at that

spk_0 forest plot or that funnel plot, once the meta analysis is done in 20 years, you're never really

spk_0 going to look at it and say, yeah, but who came out first? And so I think we are broadly adopting that

spk_0 that perspective of really embracing meta science and looking at the big picture rather than

spk_0 the whole idea that it was so much more valuable to publish something novel. I think that ultimately

spk_0 comes down to like this old school view of the career of a scientist or academic more so than what is

spk_0 good for learning and the literature itself. So for example, you could really make a name for yourself

spk_0 by identifying the trexular theory of blank with this novel finding, the first to do it.

spk_0 And of course, it's valuable to have people seeking to be innovative and to learn new things

spk_0 about the world. But the idea that being the first to the punch is necessarily advantageous or good

spk_0 or valuable. That is from my view exclusively a career slash ego oriented perspective. I want to be

spk_0 the first because then it's my discovery. Whereas if the main concern really was just making sure

spk_0 that we had robust scientific literature to lean on, the first to find it is no more valuable than

spk_0 the third to find it. And frankly, not until we have the second third and fourth to find it, do we

spk_0 actually, what we actually want to really hang our hat on that set of observations. So I think that we

spk_0 are broadly slowly moving away from that model of everyone trying to be the first to something.

spk_0 I think we're understanding the value more as a collective scientific community of I think we're

spk_0 more appreciating the value of replication. And so we still have a lot of work to do in terms of

spk_0 rectifying some of those incentive structures, but they're definitely to be optimistic. They're

spk_0 definitely moving in the right direction for sure. So great comments and questions in the chat over

spk_0 there in the iron culture chat. I want to make sure that I'm answering both chats. So let me pop in

spk_0 here. Yeah, I think I got to everything that I could see. So hopefully people in both chats feel

spk_0 adequately listened to that they've gotten adequate attention during this live recording of the

spk_0 iron culture podcast. I'm just going to zoom out real quick. Yeah. So everyone, I want to thank you

spk_0 once again, as my light turns off for joining in on a live recording of iron culture. I really appreciate

spk_0 all the comments, all the questions, all the support as always. Once again, if you like the show and

spk_0 you want to do what you can to support many ways to do it, make sure that you like, rate, subscribe,

spk_0 review, make sure you go over to massresearchreview.com and consider becoming a member of the mass community.

spk_0 Definitely make sure that you're supporting our dear, dear friends over at Elite FTS. Fantastic

spk_0 folks over there. Like I said, I've been using their products for well over a decade. And finally,

spk_0 as I was killing time and saying the things that are burned into my memory, I navigated over to

spk_0 the other live chat. So I just want to make a quick, I want to quickly address some of the comments

spk_0 over there. I did the mass live chat now. I'm in the iron culture live chat. So Dylan said, Helms

spk_0 is going to be here to show his commitment to the cult, right? And the answer is no. As we've

spk_0 already seen regrettably, I don't know what his commitments are to, but it's definitely not the cult.

spk_0 Got a lovely good night message from Eduardo. Good night. Sleep well. Appreciate you popping into

spk_0 the chat. Steven says it's Dr. Trexler versus Big Vinegar. Absolutely it is. It is frightening to go

spk_0 up against a massive industry like this. We know that they have unlimited power, unlimited wealth.

spk_0 They could make a person like me go away very easily, but I'm willing to take on the vinegar lobby

spk_0 and the vinegar mafia, so to speak. So we'll see how it goes. But if anything negative were to

spk_0 happen to me, certainly the vinegar barons are the culprits for sure. And then Dylan said,

spk_0 it's really sad. My nutrition professor is an RD and once stated in our class, taking apple cider

spk_0 vinegar is one of the health, healthiest things you can do. Yikes, Dylan. Yikes. I am going to be

spk_0 charitable and pretend that there is some kind of context that I'm not understanding in which that

spk_0 would be a defensible thing to say, but my initial reaction is Yikes, Dylan. Yeah, so if you

spk_0 were going to talk about what are some of the healthiest things you could do to wrap up the show

spk_0 here. Apple cider vinegar, I don't believe it would make my top 1000 healthiest things to do.

spk_0 Number one is be physically active. Number one, and number two is eat a balance diet with adequate

spk_0 macro and micronutrition. Certainly things on the short list would be avoid smoking.

spk_0 If you're going to drink alcohol, do it in moderation, get adequate sleep, develop and maintain

spk_0 close social bonds and a good social network in your local community. Identify something that gives

spk_0 your day and in more broader terms your life a great sense of purpose such that you feel connected

spk_0 to things bigger than yourself. We could go on a very long list before we ever got to

spk_0 apple cider vinegar. So that is very troubling. And Dylan has challenged me in the 11th hour

spk_0 of the show metaphorically to do my whole top 1000. So I tell you what, Dylan, stay tuned.

spk_0 So the next issue of mass is coming out in a couple days, October 1st, November or December 1st.

spk_0 Make sure you tune in to see if I've decided to list as an article in mass my top 1000 things to do

spk_0 for your health. I'm not committing to it. I'm certainly not guaranteeing it. But if it keeps

spk_0 you on the edge of your seat, Dylan, you can think about the possibility of me listing out 1000 healthy

spk_0 things. And we can take bets on whether or not apple cider vinegar will show up somewhere in the

spk_0 980 to 1000 range time will tell. All right, folks. As always, thanks so much for joining me.

spk_0 I really appreciate you checking in. We will be back with another episode in seven days. But

spk_0 until then, everyone have a fantastic week and take care of yourself.

Ep 347 - New Retraction (and What It Means For "Evidence-Based" Fitness)

Interactive Transcript

Topics Covered

Related Episodes

Exploring Integrative Medicine and Holistic Health with Dr. Sandeep Gupta - Part One

The New Science of Women's Health with Tamsen Fadal & Dr. Amy Killen

216. Dr. Labib Ghulmiyyah: On How To Prepare Your Body for Pregnancy, Fertility Decline, and Preconception Health

Episode 5 - AI & Facebook to screen for depression

Share Episode