Making sense of lists of studies

peerreviewprotest
Wouldn’t it be great if these were the signs at protests? by IFeakingLoveScience

The global discussion of genetically engineered crops has been heating up, and people are looking for answers to their questions. Are they safe to eat? Do they harm the environment? Does it make the foods radically different from what they were like before? And is there independent research besides the industry-funded science? These are all good questions, and we’ve been doing what we can to help people understand the issues through blogging, interviewing, and hosting discussions on our site. The biotech industry has started up their own GMO Answers site to address the common questions that they get, but we at Biology Fortified have gotten a fair bit of attention for a little list we started years ago. That list, what it means, and what it is becoming, will be the subject of this exercise.
As scientists, we here are familiar with the scientific literature of our field, and know how to look up and understand information in the scientific literature. The peer-reviewed scientific literature is written by scientists for scientists, and are not generally intended to communicate the results to the general public, as the terms, methods, and results are complicated and require background knowledge to understand. For the public, the scientific literature is generally not accessible. So it comes as no surprise that in our journey to help explain the science to the public and engage on this topic, the claim that there is no science, hardly any science, or only industry science conducted on genetically engineered crops kept coming up. Some would say that the few independent studies always found problems.
Dr. David Tribe started to assemble a list of studies to refute these claims, and to show the studies that support the safety of these crops for consumption, the environment, etc. Over time, this started to climb into the low hundreds. Based on this list, we created our own here, and started adding more studies to the list as we found them. We also created a second sub-list that contained studies indicating that they were funded by sources independent of industry. In discussions, when people would make these various claims, we and others started plugging the links to these lists in to help people see that at least those basic claims were false.

The Failure of Mere Lists

listSuch lists as communication tools are not very useful. I recall one discussion where someone kept aggressively demanding for us to find out if there was any research on a particular genetically engineered crop. We suggested they take a look at the list and see if they can find anything. They declared that there was nothing of use in that list and no research on that crop at all. Again, we entreated them to just give it a try, and search for words that describe that crop and they will probably find something. Again, they howled that it was not in there and we were taking them for a ride.
The research they were looking for was number 4 in that list.
There have been similar instances that demonstrated to us that simply providing a list would not help people understand the science. Indeed, it can be a barrier to learning because of how intimidating it is. Lists like this are really only useful for scientists, so we started thinking about how we could turn this list into something far more useful for our target audience – everyone else. That’s when we conceived of making it into an actual database that told people about the research itself. And rather than fill it with only research that appeared to support one conclusion or another – we decided that it must include all of the relevant research to be a useful resource. We wanted to write plain-language summaries of each study, and list their outcomes and funding sources on the sidebar, and use crowd-sourcing to do it. The first concept of the GENetic Engineering Risk Atlas (GENERA) was born.

GENERA

With a little fanfare, we announced in 2010 that we were starting the GENERA project, complete with how-to guides for volunteers to help us organize and describe this research. But, this task of writing original summaries for what was then only 300 studies proved too daunting of a task to do on a voluntary basis. The next year, we applied for grant funding to make this possible, but didn’t get it. This started a process of thinking and reconsidering our approach.
Scientific papers have their own summaries already written – the abstract. You can get the general idea about the scope of a paper by reading this, and most people would probably not bother reading even the abstract if we had the simple sidebar study breakdown like we had envisioned. For most people, seeing the funding, results, and category of research at-a-glance would be enough. Last year, we re-applied for a grant with the American Society of Plant Biologists to fund the project, and we got it.
At the time, we then had about 350 studies in the list, but when we started to systematically search the literature for more studies, the number grew – and fast. Before we knew it, we had 600 citations in our list. Since we were busy turning this into a database, but people were still looking for the basic list in discussions online, we set it up to display the first 600 citations, organized alphabetically, and no longer edited it to add or remove citations so we could concentrate on the project at hand. Though 600 were listed, the number has grown considerably more in the past few months.
As discussed in this post, we searched long and hard for a system to make managing this project possible in the long term, and we settled on a reference manager that allowed us to keep track of and tag each PDF with useful information, and export it into a spreadsheet to instantly turn it into a database we could upload to the site. Our call for volunteers was answered by some of our readers and fans, who have been helping us track down, download, and enter the citation information for each and every study, and search the reference sections for additional studies to add.

Qiqqashot
Our growing library of studies. The number is real.

It has been working out great, and of the original 600 citations you can see online, our volunteers have already gone through and picked out all the publicly-available studies, and have moved on to combing other people’s lists of citations, including those on anti-GMO activist websites. They have helped us identify duplicate citations, as well as citations that were to conference abstracts instead of peer-reviewed studies. These sorts of things can happen when you cobble together multiple lists and search results, particularly when there are minor spelling differences between citations that make them harder to spot.
I am slowly crawling down our list to get to the hard-to-reach studies by contacting the authors, reaching through the toughest paywalls, and even scanning the studies from library collections if need be. I’m only as far down as the letter D for this process right now. There is a lot more to find, and indeed, as I will demonstrate, the number of studies that will be included may very well reach 1,000 before we’re done digging up past research.
Now comes the difficult task of assessing the outcomes and funding sources of each and every study. Each study may have experiments that specifically address questions of safety for consumption, safety for the environment, efficacy of using genetic engineering for a desired outcome, and analyses of the equivalence or differences between genetically engineered crops and their non-GE counterparts. We are not interpreting the quality of the methods, analysis, or conclusion of each study, but are simply rating the outcomes as reported by the study authors. Similarly for funding sources, we are categorizing them according to industry, government, competing industry, and various categories of NGOs, and for those without such information, we are contacting the authors.
The end result, we hope, will be the most comprehensive and useful database of research that focuses on the relative risks of genetically engineered crops.

To think like a Scientist

There is a bit of a mental shift that has occurred, which is akin to the change in perspective that you can get when you become a scientist. When talking about the safety of genetically engineered crops, it can be tempting to dismiss the studies that reach conclusions that you disagree with, or not include them for consideration on the basis of your assessment of the science or the scientists conducting it. People cherry-pick all the time to win arguments. The original list of studies was a response to cherry-picking from critics of genetic engineering who try to ignore the vast array of science that overshadow the few critical studies they can find. But just responding with only studies that support your own point of view – even if it is the consensus view, is presenting less than 100% of the science.

movealong
Officer Barbrady does not want you to read the NAS NRC consensus reports

I once heard a politician go on the radio, responding to me mentioning our GENERA project earlier in the program, and regarded it as merely saying “My science is bigger than your science.” He regarded the few cherry-picked studies he had in hand as being enough for him to ignore the rest of the literature. This is a fundamental misunderstanding of how science works. There is no “your science” and “my science” – there is just “science.” When you collect it all together, you derive a consensus view based on repeatable and predictable observations. There will be papers that dispute the consensus – this is predicted by statistics – but you only arrive at a complete view of what we know when you take it all into account.
This week, a group in Europe called ENSSER published a political signing statement that there is no consensus on the safety of genetically engineered crops. They list a few cherry-picked studies that find problems, and on this basis declare that there is no consensus. They also reference David Tribe’s version of our list of studies, but avoided mentioning it by name in the body of the text. What is funny is that they call it an “Internet website” – as opposed to non-internet websites? They totally ignore all the National Academy of Sciences consensus reports, and misrepresent studies as finding “Toxic effects” though the studies themselves do not claim to find them. Their intent is to dismiss a large body of research and focus on unrepeated studies with methodological problems that disagree with the rest of the literature. This is not thinking like scientists, but like ideologues and politicians.
When I see people link to their favorite unrepeated studies claiming harm from genetically engineered crops, instead of getting annoyed I get excited. “Excellent – more studies for GENERA!” This is a healthy perspective, one that reflects a truth-seeking attitude that I hope more will come to appreciate. So naturally, I combed through the references in the signing statement for any new studies they knew about that were not in our database. The only studies that were new to me actually had conclusions that were positive about GE crops. We’ve been grabbing lists of studies from groups claiming that they have research that shows problems with genetic engineering, and harvesting them like this for every last citation they have. All will be assimilated. Because Science – and GENERA – are not about cherry-picking to reach a predetermined conclusion. Science does not operate on the basis of political signing-statements, but on the weight of evidence. This is what it means to think like a scientist.

Unclear on the Concept

We’ve been very public and clear on the aims and scope of this project, however, communicating this to others has met some challenges. Claire Robinson of GM Watch, was unaware that we were including all research, not just one “side” as she had assumed. I talked to several activists and people skeptical of GE foods who seemed to also have difficulty with understanding this. Some of it could be a consequence of remembering only what the list used to be, but much of it I think reflects how people typically approach this topic. I have said many a time that “we’re including all of the relevant research” to hear people process at and respond: “so you’re only including what you agree with?” – at which point I have to repeat myself. But I have come out of conversations with anti-GMO activists who were really excited when they learned that we were going to be comprehensive with the Atlas. No other organization deeply embedded in this debate has ever tried to do this.
Some have expressed shock that we would include research that reaches negative conclusions in our project. How can a database of studies demonstrate a low risk for GE crops while there are a few studies that disagree with that conclusion? Simply by showing that they are a minority of the studies, and that their results are not found in studies done by other researchers. The same is true for one-off studies claiming that climate change isn’t real, when the vast majority of the literature shows that it is, and it is repeatable. This is a communication challenge that we’re hoping to address, because it cuts to the core about how science works, as I have explained above. Plus, by including all of these studies, we may end up finding patterns that point to risks that no one had considered. Science does that.
A more recent example of misunderstanding is a series of blog posts by Madeleine Love formerly of MADGE Australia, in which she prematurely reaches the conclusion that all the many studies in our GENERA project are irrelevant, misrepresented, or over-reported. I would normally not highlight this, however, several prominent activist sites, pundits, and one scientist who is a vocal critic of genetic engineering have been promoting it as if it was a cogent and insightful analysis of our project. It is not.
Ms. Love has rightly pointed out that there are duplicates within the list, and ironically at the very top of the list, so she started subtracting from the total of 600. Her goal is to try to prove that this project and the research outlined in it can be dismissed. She gives the impression that we’ve overstated how much research there is, or what its conclusions are. A twitter conversation with her revealed that she blames us for misunderstanding what our project was about, but places the blame on others, never once pointing out an error of ours, or taking responsibility for her own misunderstanding.
She is engaging in a citation-by-citation exploration of the studies in our list, which is a process that I wholly endorse. She is demonstrating through this process what we already know about the uselessness of such lists as a communication tool, and the contents of our own list that we have already figured out months ago.
GENERA-success-boy300However, she engages in a doctrinaire and unscientific approach in her posts right from the beginning. For instance, the first citation was a conference abstract for a paper that was later published. Caine et al. (2007) looked at pigs fed four diets: one containing roundup-ready canola, one with the nearest isoline, and two reference diets with different varieties of canola. The conclusions of the researchers was that the diets were nutritionally equivalent, and that there was no differences due to the addition of the genetically engineered trait. They looked mostly at parameters of meat and carcass quality, however, they also examined growth characteristics, organ weights, and other parameters that gave insight into development and metabolism. They found that the major differences between groups were due to the differences in glucosinolates between the varieties of canola – the consequences of breeding, and not between the genetically engineered canola and its similar parent.
Ms. Love tried to minimize their conclusion about the equivalence of the genetically engineered canola, avoided mentioning the measurement of non-meat-quality parameters, and yet somehow, twisted the study to the creative conclusion that any canola from Monsanto is bad to eat because the variety was different from the comparison varieties. This is also trying to have it both ways – calling it both a study irrelevant to health and trying to reach conclusions about the healthfulness of different varieties of canola based on it. Ms. Love claimed that the well-described issue with glucosinolates somehow “raise[s] doubt about the quality and relevance of the findings,” which it does not.
In other posts about studies in the list, she reaches similar dismissive conclusions apparently by only reading the abstract and not reviewing the full studies at all.
We are actually reading them all.
Her dismissal of this study as being in any way relevant for human health can be shown to be doctrinaire with a simple thought question. If the study had, in reverse, found that the pigs had enlarged or deformed organs, lower muscle mass, and altered metabolism as a result of eating GE canola, would Ms. Love still consider it to have no implications for human health? We would all agree in that case that it would be cause for concern, including her. So therefore in absence of such harms, it is reasonable to conclude that this study provides some assurance, albeit not very detailed, that GE canola would be unlikely to have a negative (or positive) impact compared to conventional canola.
Because of the duplication, Ms. Love gleefully announced that we had only 599 studies, not 600. As we go through the Atlas building process systematically, we identify studies that are missing as well as those that are duplicated or otherwise incorrect. In the process of reviewing Caine et al. (2007), we removed one abstract from our spreadsheet, however, we identified 4 additional studies that were not included. Our math disagrees with hers because we’re trying to build a resource, while she is trying to make people ignore it. We don’t just have 603 studies, either. There’s a lot more science to catalog than that.

Like Counting Stars in the Sky

443819_f496
Milky Way Galaxy appears over Ontario Credit: Kerry-Ann Lecky Hepburn

I said above that I would demonstrate why I think there could be well over 1,000 studies that will end up in GENERA. Here is the current status of how many studies we have as of today. We have downloaded 572 PDF files of peer-reviewed scientific studies, and entered them into our library. These are mostly the easily available ones, and our volunteers have identified 127 studies that are behind paywalls that academics with access will have to retrieve. There are another 51 studies newly added from references and searches that have not been examined yet for access, and 38 more that have been flagged for administrators to figure out where they are. If we just add up the ones that we know are peer-reviewed scientific studies, then we get 572 + 127 + 51 = 750.
This does not include the studies from other lists we are currently sifting through, the dozens being sent to us by other scientists, or new searches of journals, nor citations yet to be harvested from the reference sections of each PDF we currently have, and if only one out of three of these gave us a new one to include, we would exceed 1,000 studies easily. The rate at which we are adding new studies to our database suggests that this will be the case. It could always slow down as we get more duplicates, which is why I could not say it would definitely have 1,000 studies. That was just a projection that was spoken, but not written down. But earlier this month, something changed that will make this a more definite reality.
A review study was published in the peer-reviewed scientific journal, Critical Reviews in Biotechnology, which searched for and cataloged all the peer-reviewed research on genetically engineered crops in the last decade. The scope of Nicolia et al. is broader than GENERA, as it included research on coexistence and traceability in addition to research on the safety of consumption or off-target effects in the environment. Here are the category breakdowns and how many studies they found in each:

  • General: 167
  • Biodiversity: 580
  • Coexistence: 96
  • Wild Relatives: 111
  • Horizontal Gene Transfer in Soil: 59
  • Non-targeted assessment: 107
  • Equivalence: 46
  • Consumption: 313
  • Traceability: 305

If we were to add up just the general, biodiversity, non-targeted assessment, equivalence, and consumption studies together from this list alone, that would total 1,213. We will go through all of these to add relevant ones to our database, so maybe the number could go down, but it very likely could go up (keep in mind, this is only research from the past 10 years and we have studies dating back to 1994). 1,200+ would be a conservative estimate. Jon Entine at the Genetic Literacy Project nicely juxtaposes the sheer number of studies done on GE crops with statements from activists saying that there’s almost none at all. However, to say that there are 2,000 that confirm safety is optimistic. As you can see, about 400 of the studies in the Nicolia review are about coexistence and traceability, which isn’t a safety issue so much as a cultural and political one. But it would be entirely accurate to say that these ~2,000 studies all show together that genetic engineering is a highly scrutinized and well-studied technology.

Where we are right now

IwantU
Frank needs an army of GENERA volunteers. There are perks!

The review process for outcomes and funding is going on right now, and a fair pace. We’re still just in the double-digits on this part, but as we also work on how we will display the Atlas on our site, we will prepare the current collection of studies in our database to begin some alpha and beta testing, with an expected public display of the contents of GENERA by the end of this year. The sheer number of studies that we are finding means that we have an ever-present need for additional volunteers, and are searching for more sources of funding to finish the analysis of the study outcomes, which takes the most time. Please feel free to give us suggestions, submit studies, or help us out by volunteering for the project. When we get this information online and searchable for everyone, it will be a fantastic resource, and we can get beyond talking about how many studies there are and more about what information those studies contain.
GENERA will continue to evolve as we learn more about research there is out there, and we will be challenged by studies that don’t fit into the neat categories we try to put them in. We’re going to find errors in our approach and refine it to address them and improve the database. We’re also going to encounter criticisms, both constructive and dismissive. Constructive comments will help us identify errors we can fix, and information we can add to the database. Efforts to dismiss GENERA will ultimately be seen as a result of a narrow view of the world, threatened by the vast amounts of knowledge that science constantly generates. Science marches on, and its our duty to help people find and understand the discoveries and conclusions made by scientists around the world.
Editor’s note: I incorrectly indicated that Madeleine Love was currently with MADGE Australia, and she has informed me that she is no longer a part of the organization, so I have updated the post to reflect that.

31 thoughts on “Making sense of lists of studies

  1. Karl,
    What is telling is the lack of studies that show harm. If any of the classics, from Pusztai to Seralini, to Seralni, to Seralini to Carman and everything in between was real and solid, it would have been the basis for further hypothesis testing based on the phenomenon reported.
    One example. The famous “organ failure” paper from If these results were real, wouldn’t they nail down the molecular mechanism? Wouldn’t other labs lend their expertise and expand the findings? Wouldn’t today’s super sensitive molecular, genomic and metabolomic tools be employed to learn about this earthshaking story?
    Absolutely. IF it was real. IF other researchers felt merit in the findings.
    The fact that they die on the defective science vine and disappear slowly into history unadvanced speaks volumes.
    The flip side to a list demonstrating a lack of harm is the failure to expand the “paradigm shifting” results that say the technology is dangerous.

    Like

    1. And even the “organ failure” claim was never made in those papers. That was invented by bloggers and activists and not in any paper I have read. You are right, if it was possible to show this by repeating a study, it would happen. Heck, I’m tempted to apply for a grant to repeat Pusztai’s study and do it right.

      Like

    2. The fact that they die on the defective science vine and disappear slowly into history unadvanced speaks volumes.

      I wish! People that are opposed to GE or any other biotech advance are keeping those defective studies on life support from what I’ve read.

      Like

  2. Kevin,
    I recall a discovery that was made back in the late 90’s that resulted in some …interesting… research. Here are some old studies that are even more likely than the above to get their authors hate-mail if they are publicized!
    “Evidence for the shikimate pathway in apicomplexan parasites.”
    http://www.ncbi.nlm.nih.gov/pubmed/9655396/
    “Targeting the Shikimate Pathway in the Malaria Parasite Plasmodium falciparum”
    http://www.ncbi.nlm.nih.gov/pmc/articles/PMC89043/
    This may be a path to what I remembered first:
    http://www.bio.net/mm/plantbio/1998-June/017823.html
    But this study puts a favourite compound of the alt-med community into BAD company!

    Click to access P020111021323017185118.pdf

    But there are a zillion discussions. I wonder why I have never heard of “that chemical” advocated as a “supplement”?
    http://www.rsc.org/Education/EiC/issues/2005Mar/Soundbitemolecules.asp

    Like

  3. Oh. Yeah.
    As someone who has personally entered some of the scientifically discredited work into GENERA, I attest to the broad coverage of this effort.
    Sure, it made me wince. But it is in. That said, it’s such a small fraction of the corpus that it is really just laughable.
    Even more laughable: someone on a blog cited to me just the other day the hairy hamster HuffPo piece (still not published), and the withdrawn Austrian study (never peer-reviewed). These are not–needless to say–in GENERA. For good reason.
    I know some people really mean well and think their Google-U education is the same as a decade of education in the field and years of experience. But it really isn’t.

    Like

  4. It sounds like this is going to be a phenomenal resource. Kudos to you guys for making a non-ideological effort.
    On the communication front, I recommend reading Randy Olson (thebenshi.com) – we need to make sure we don’t fall into the same trap as climate scientists. This resource will be great, but we need to remember that convincing regular people is not just a matter of throwing more data at them. The response to GMO is emotional as much (or more) as it is mental.
    One other idea, any thought on using the new pubmed commons towards this project? You might leave comments on papers included in the index to solicit summaries – might pull in people that otherwise wouldn’t find out about the list.

    Like

      1. I can issue invites to anyone who has a paper in PubMed. If you want one, get me a Pubmed ID for the paper, and the email you use for your MyNCBI account, and so far it seems to have worked.
        So far we have been able to rate up AJ Stein’s comment on his Golden Rice paper, which is nice :).

        Like

  5. …we need to remember that convincing regular people is not just a matter of throwing more data at them. The response to GMO is emotional as much (or more) as it is mental.

    From my mere lay person’s perspective: The quote above is another reason why I recommend that non-experts stop citing “studies”: They don’t know what they are doing.
    The worse journalists and food writers do it all the time: They go to their favorite websites where all the studies have been “pre-cherry-picked” for them, then they parrot their favorite “study.” My hunch is that they’ve never read the study, they don’t have the tools to read OR evaluate the study, and they therefore have no idea whether it is a good study or a crappy study.
    In my own writing, I’ve stopped citing “studies,” and as Kevin points out, it hardly matters anyhow.
    I like Dr. Harriet Hall’s rules for evaluating the whole “studies show” thing, paraphrased as:

    1. Roosters don’t make the sun come up.
    2. Never believe one study.
    3. Always find out who disagrees and why.

    Source.
    If one hasn’t done that work, one should shut up about it.

    Like

    1. Mike, I agree in that simple pointing people to some SCIENCE! will not change many minds. There are many values wrapped up in the discussion which have the power to maintain opinions despite contrary knowledge. We’re not naive enough to think that providing this Atlas will reach everyone, but rather, it is intended as a tool to help people find information if they are so inclined. We’ve gotten a lot of interest from teachers and journalists, and I imagine many scientists will find value in it as well.

      Like

  6. Dear Karl: Not only teachers and journalists will find it interesting but FARMERS like me, thanks for all you are doing.

    Like

  7. This is a laudable and Herculean effort and I wish you well. To me, if the goal is to sort out the safety and benefit of GMOs in an independent way, it might be easier to just throw out all the industry-sponsored research in order to avoid the possibility of including studies that were influenced by pro-gmo factions. Likewise, studies that might be influenced without being directly funded might be suspect. We should only look at truly independent research because if we list all research, the volume of pro-gmo research will outweigh the independent simply due to the fact that pro-gmo research is extremely well-funded, while independent research on gmos is poorly funded (meaning – there’s not a lot of it). Also, historically at least, independent researchers have been prevented from having access to studies done within the developers labs in order to analyze them for validity or in order to duplicate them. Also, the types of studies that are done don’t seem to include much proteonomic analysis, which would help to assess how safe and truly “equivalent” gmo food is.
    Honestly, i can’t imagine the amount of work you’re going to have to do to put this together – it sounds daunting. Having worked long hours to sort out influence in various areas of the debate over gmos, I can tell you that it gets to be very convoluted very fast.

    Like

    1. Thanks, it is a lot of work! As for the idea of throwing out industry-funded studies, that would also be a form of cherry-picking and give us an incomplete view of the literature. We would be unable to make any comparisons between industry and non-industry-funded studies if these were removed. Also, some studies that are critical of GE crops, such as Seralini’s work, have received funding from competing industries and aligned nonprofits – should they be thrown out as well? We have to keep in mind who funds the study, but stating a funding source does not give one a reject-the-science-free card.
      I am finding a bunch of proteomic analyses in our library, and lots of stuff on nutritional equivalency.

      Like

  8. The problem with all this was discussed in the cover story of the Economist, entitled Haw Science Goes Wrong, which talks about how “half of published research cannot be replicated,” Amgen…could reproduce just six of 53 “landmark” studies in cancer research,” and “one in three researchers knows of a colleague who has pepped up a paper by, say, excluding inconvenient data from results ‘based on a gut feeling’”. The approach suggested here is built on the premise that the “quantity” of research with a particular conclusion is an indication of its reliability and this is false. I believe the way to get to the bottom of the debate is to follow the suggestions in the article. For instance, studies need to include COMPLETE information on how to replicate them. There needs to be a much bigger penalty for publishing results that are later discredited. There needs to be greater incentive to replicate studies. Etc.

    Like

  9. You’re right of course. As I said, it’s so hard to sort out influence in the literature. It would be interesting to learn how much money is spent on either side, just to get a “gestault” of how it’s weighted overall. I can’t even look at a list like GENERA anymore because to me there’s no way to sort it out and it’s too overwhelming. Like i said, Herulean. What I worry about is that people who can’t analyze the individual papers for their merit will simply say to themselves “wow, there really is a lot of research on the safety!” (or something like that for example) But we really can’t make broad statements like that right? Because it’s all about what you look at, and for how long, etc. It kinda seems like anyone who wants to support generalizations can find something somewhere to support their claim.
    Maybe I’m just tired 🙂

    Like

    1. It is a huge amount of information. I have also seen some people dismiss the list saying “oh it’s all studies on X or Y, not what I wanted” or that they’re all milk-production studies. When its just a list it cuts both ways and gets in the way of understanding. The sooner we can get it out of list form on the site the better!

      Like

    2. First time I’ve seen the problem posed as such. Normally one hears that there are no studies. Now there are too many. What’s a technology to do?

      Like

  10. My personal thoughts:
    In order to assess the research on any given topic, we have to first decide: what’s important to research with regards to that topic? Let’s take safety of GM foods for human consumption, for example. Right now we have the test of equivalency. Important and known components for nutrition are measured against the parent. It’s tested for toxicity. (perhaps others can tell me more about what’s tested)
    But, knowing that GM food can present novel molecules, (because the enzymes which synthesize them have a very low substrate specificity and plants generate thousands of nonessential molecules as part of their secondary metabolism) and knowing that the regulation of the synthesis and structure of those molecules remains largely mysterious, and knowing that the amounts and specificities of these molecules are unpredictably altered by the types of mutations and pleiotropic effects associated with transgenic technology – I would say that research on this topic is important.
    Now, for all i know it’s being done. But i can’t find it, and I’ve read that it’s not really being done. So, if this (to me important) research isn’t being done, there’s no way to notice it’s absence if, like me, you weren’t even aware of the concern to begin with.
    If we assess the important factors when it comes to safety for human consumption, then we’ll be able to decide better whether the research is valuable and what it says. That’ll allow us to make informed decisions about regulations, etc.
    The safety for humans is just one example. And the people at biofortified would be the best ones to try to formulate some kind of structure to evaluate what value various research has, and whether or not there are some areas that we just don’t have much valuable study at all because we’re not asking certain questions that should probably be asked.
    Hope this makes sense. I’d like to see a critical assessment of the nature of the research with regards to: what’s important to research, and how does the current body of research break down as far as how it addresses those areas.
    thanks

    Like

    1. “knowing that GM food can present novel molecules … pleiotropic effects associated with transgenic technology”
      That’s a lot of claims you have there! Can you please explain what these these novel molecules are? Do you have any citations where these have been discussed? I’m having a hard time understanding what you mean here.

      Like

  11. Anastasia, I’m not making any claims. I’m just repeating what scientists say. And I’m sure you have a far greater ability to understand it than I do.
    The main point of my comment is:
    I’d like to see a critical assessment of the nature of the research with regards to: what’s important to research, and how does the current body of research break down as far as how it addresses those areas.
    My examples, such as the pleiotropic effects associated with transgenic technology, are simply topics to be considered.

    Like

      1. I don’t know why it’s not good enough that I said it.
        I’m sure you have access to the research on this.
        If you think I said something inaccurate, why don’t you just explain? It would be a great opportunity to educate the public.
        again, my main point:
        “I’d like to see a critical assessment of the nature of the research with regards to: what’s important to research, and how does the current body of research break down as far as how it addresses those areas.”
        regarding your interest in the novel molecules and pleiotropic effects associated with transgenic technology, the following letter from a discussion about regulations discusses this:
        http://www.twnside.org.sg/title2/service221.htm
        which is somewhat related to my main point, as it suggests that metabolic profiling and AMES test be done as part of safety evaluation on GM food plants.
        And I found one of the references from the letter for you:
        http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2621.1995.tb01365.x/abstract

        Like

  12. I was excited to learn that someone was compiling all this data, it can be daunting to sift through and try to make an informed decision with. After being repeatedly being duped by the peer review process and finding out that studies were published in error or falsified, I am overly weary of sources and funding of those sources. After reading the mission here, I started looking around to find the purpose and source of funding. I ended up on this page where high schoolish catty remarks are being made about anyone who has objected to the validity of your project in what is best described as a juvenile attempt at public shaming. Then there’s a meme. How scientific. I read the entire “blog” though, and here in the comments I see that you are not eliminating studies that have been funded by the industry. So, what I have gathered is that this project intends to use information from biased sources, had a goal of proving with lists of those sources that GM foods are not harmful and makes fun of anyone who disagrees with their opinion or questions their process. I really wanted this to be an unbiased credible source but time and time again I have learned that people find exactly what they want to find and it looks like you guys are well on your way to doing that. Can you explain this?

    Like

    1. Hi Kelly,
      Of course we are not eliminating studies that receive funding from industry. To do so would actually introduce a bias into the database, as would eliminating any study funded by competing industries or aligned nonprofits. Instead, we are tracking down the funding sources for all of the studies, and including 100% of them in the database so that people can apply their own biases and perceptual filters if they want to. So you could use it to look at only the non-corporate-funded studies, while they are still available to others. Besides, you’ll also miss out on some stuff that you would probably agree with – one industry-funded study found issues and is highly cited – should we exclude that from the database?
      Sorry you don’t like my sense of humor with regard to the kinds of off-base criticisms we get. I like how when people find a scientist responding in a non-robotic “X = Y therefore Z.” way they get called unscientific. I see it on blogs all the time. Just because I make a meme to make fun of someone dismissing the science we include in our database doesn’t mean that there’s anything wrong with the database.

      Like

  13. Thank you so much for doing this. I am greatly troubled by the spread of anecdotal ‘evidence’ being shared by well-meaning, intelligent, and educated people. It is difficult to differentiate them from the FOX drones who permeate the climate debate. This meme ‘scientists disagree’ is laughable. So what? That’s science. Consider your efforts ‘shared’!!

    Like

Leave a comment