Science Video Publications for Increased Reproducibility in Research and Education: JoVE (Journal of Visualized Experiments)

Val Buntrock & Kira Henderson
val.buntrock@jove.com
kira.henderson@jove.com

October 28, 2013 - October 30, 2013
Abstract: 

Recent attempts by science research consortiums to validate results from contemporary publications have exposed a tremendously low percentage (~10%) of reproducible results. This astoundingly poor reproducibility can partly be attributed to flaws in static publishing format, namely lack of detail in describing complex methodologies. As new technologies emerge, science research and education have adapted to include these new offerings. However, there has been no accompanying evolution in publication format. The scientific publishing model initially introduced in the mid seventeenth century is the same model used to present today’s research findings. Articles are still published as text heavy tomes with the occasional scheme, diagram, or table to demonstrate a point. Why is the science community still employing ancient text standards? How should the publication template be modified to accurately communicate advanced techniques? This presentation describes a new trend in publication format, namely video publication, and its ability to accurately capture complex techniques, thereby increasing reproducibility of results. Recently conducted case studies will be shown in support of video publications as a valid communication venue in scientific publishing.

Article PDF: 

 

Science Video Publications for Increased Reproducibility in Research and Education: JoVE (Journal of Visualized Experiments)

 

 

State of Publishing

 Science research is presented using largely the same publication model introduced in the mid seventeenth century. With the exception of color printing and electronic dissemination (cataloguing and searching functions), not much has changed. While these advances have improved user accessibility, they have not improved the ease of application or utility of the presented findings to other areas of research.

Conversely, as new technologies emerge and are applied to various disciplines, research and education is advancing at a rapid rate. Despite these advances, both new research and education models are presented as text heavy documents with the occasional diagram or graph display of obtained results. Why hasn’t the presentation of these new reports evolved to include the concomitant advances in research and education?


Reproducibility on the Decline

A severe consequence of the current stagnate state of research communication presentation platforms, reproducibility in science is on the decline. Irreproducibility of research has surfaced as a serious concern in recent years with numerous studies dedicated to reproducing the findings of contemporary publications. Each of these studies has exposed an alarming percentage of scientific findings that cannot be reproduced.

Two years ago, Bayer Health called attention to the issue internally.1 Interested in quantifying reproducibility from their labs, they selected 67 published experiments and repeated them. Comparing their new results to those previously published, they found that a staggering 64.2% could not be replicated. Of the rest, a mere 20.9% were fully replicated, 11.9% results were partially replicated, and the remainder was not applicable.

A separate study conducted by C. Glenn Begly while he served as the head of global cancer research at Amgen Inc. calls attention to the reproducibility in cancer research.2 Begly’s research group identified 53 papers from high profile journals and tested their claims. Similar to Bayer’s findings, Begley’s group found that only 89% of these selected works could not be replicated.

 

Deleterious Effects of Irreproducibility

Obvious negative impacts of irreproducible published results are numerous: contamination of the literature with potentially false findings, assumptions based on these reports, and a backward trajectory of science advancement, among others. A more downstream consequence, research budgets allocated through government grants and independent industries are at risk. As the number of studies investigating the reproducibility of results in published research climbs, funding agencies are beginning to apply more stringent reproducibility guidelines to their applications.

Echoing this concern, an article in the Journal of American Medical Association provides a break-down of the biomedical research budget and funding.3 The report reveals a recent decline in budget trends despite the steady incline in the number of researchers. An updated study using recent fiscal calendars shows that research budgets continue to decline.4

The authors of the study posit high failure rate of new technologies as one possible cause for the budget decrease. Failings in technology can be traced back to the poor reproducibility of published research and the inability to translate basic research findings into advanced applications. Though some of these failings may be due in part to complications of scaling factors, lack of reproducibility certainly plays a large role.

Considering funding availability directly impacts science progress, science will suffer greatly if reproducibility is not addressed and remains an instigator in declining research budgets. To mitigate this serious and immanent threat, the science community must acknowledge the problem and alter the way in which information is shared for increased reproducibility.

 

Revolution in Science Publishing

The reproducibility epidemic has not gone without attention by the publishing groups responsible for distributing research reports. As the number of reproducibility investigations and published reports climbs, irreproducibility awareness has increased. Just this past spring, Nature published an editorial commentating on ways in which researchers can alter their published works to increase reproducibility.5 Among these suggestions was increased methodology details and specific reagent information, as the lack of sufficient detail in published reports was implicated as a main contributor of the irreproducibility problem.

What was not addressed in the Nature editorial was how researchers could provide greater level of detail when describing their recent findings. If publication methods or the ways in which information is presented to the community have not changed in centuries, how can scientists be expected to present their advanced findings with the required level of detail needed for replication? Publishers must adapt and provide new outlets for sharing methods and results to the science community. These new outlets must facilitate detail-oriented presentation.

Here at JoVE, the Journal of Visualized Experiments, we have approached this problem and developed a novel publication format, peer reviewed video publication. Using the traditional text manuscript model with all standard sections present (Abstract, Introduction, Protocol, Representative Results, Discussion, and References), JoVE then converts the text into a script. Next, JoVE films the entire contents of the manuscript in the researcher’s own lab or clinic. Dynamic video presentation provides a visual level of detail that mirrors that of an in-person training experience. Through observing how recent findings are obtained and analyzed, the time and resources spent reproducing and learning new published methods will drastically decrease while also increasing the reproducibility of results.

JoVE publishes 70 video articles a month in both the physical and life sciences with sections in Chemistry and Applied Physics, among others. In addition to citations, the video articles receive several thousand views with total monthly web traffic typically reaching 300,000 visitors. The high number of web visitors shows that the science community is interested in video publication.

Recently published video article in JoVE Chemistry
http://www.jove.com/video/50874?access=vpjnhr5y


Does Video Publication Deliver?

Given the high usage statistics, we were interested in determining how usage translates to increased utilization of published methods. To this end, we interviewed several researchers at various institutions to inquire about their interaction with JoVE. A postdoctoral researcher at Baylor College of Medicine, Nikolaos Giagtzoglou, shared his JoVE experience with us.6 While developing a new application of his research, Dr. Giagtzoglou looked to the literature to learn three techniques for working with Drosophila (fruit flies). After spending time reading several traditional text publications, Dr. Giagzoglou was unable to learn from these reports and utilize the techniques in his experiment.

The obvious next step was to contact the author of the paper and request a training session. Unfortunately, when Dr. Giagtzoglou reached out to the authors, he found that “it can be hard to coordinate busy schedules to travel and learn the method.6” During his search for more literature sources, Dr. Giagtzoglou stumbled upon a JoVE article presenting the technique in video format and immediately recognized its value. Using the JoVE video article, Dr. Giagtzoglou shares with us that “I really had no starting point to learn these techniques, and JoVE was invaluable...Watching a JoVE video-article is so much more helpful than reading just materials or methods, which can have grammatical mistakes, bad syntax, or may be hard to interpret.6

Similarly, Dr. Casey, an Assistant Professor at Purdue University, found a JoVE article when searching for methods describing how to dissect the suprachiasmatic nucleus in mice, a complicated neuroscience procedure.6 About her JoVE experience, she said, “I had a collaborator in Buffalo who knew the SCN surgery, and I’ve seen it done before. By using the JoVE video, we saved money in travel costs to go to Buffalo repeatedly to learn the technique.6” A cost analysis showed that Dr. Casey saved over six thousand dollars that would have been spent on travel expenses and reagents because the JoVE video article provided the necessary visual training needed to learn and perform the procedure independently.

In addition to saving valuable time, money and resources, video articles have the potential to help build the foundation for new ideas and research by providing the necessary instructions for non-experts to learn and implement new techniques in their own laboratory. In fact, Dr. Casey’s testimony reflects just that, “I’ve been doing research for 20 years, and having JoVE makes things so much easier. You can educate yourself on research other scientists are doing around you and get familiarized on a technique before you try it. I like to watch techniques and refresh myself on experiments I haven’t conducted in 18 years but need now.6” In this way, video articles facilitate the transfer of knowledge from one research lab to another from anywhere around the globe without the need to travel.

Researchers have also used video articles to validate results. After publishing novel results in a high impact journal, Dr. Jonathan T. Butcher at Cornell University explains that he received numerous inquiries from researchers in the field questioning the validity of the results within since “these other labs were not able to reproduce our results using the written instructions in the methods section of our novel research paper.6” Frustrated with having to defend his research in response to these claims, Dr. Butcher published this same method in JoVE and no longer receives correspondence disputing the validity or reproducibility of the lab’s results. Dr. Butcher believes this is because “the video format conveys complicated methods significantly better than text alone and helped to validate our novel results.6” Dr. Butcher’s testimony confirms that traditional text manuscripts are no longer sufficient to describe the complexity of contemporary research methods.

 

Video Publication for the Advancement of Science

Despite a decrease in reproducibility, science journals are publishing an increasing number of articles. This opposite trend suggests that peer reviewed journals are not assisting the research community with the tools necessary to accurately present complex research. While science continues to advance at a rapid rate, publishing has not kept pace with these advances.

The first major change to science publishing in 350 years, JoVE utilizes video technology to capture complex research in both physical and life sciences. These video accounts provide the viewers with the necessary level of detail for understanding, learning, and reproducing, the research presented. Increased transparency through visualization also serves to validate methods and their results. Building confidence in research through increased validation will encourage funding agencies to grant larger budgets to research labs and institutions. The culmination of these myriad benefits will provide a springboard for the advancement of science.

 

References:

  1. Prinz, F., Schlange, T., and Asadullah, K. Nature Reviews, 2012, 10, 712.
  2. Begly, C. G. and Ellis, L. M. Nature, 2012, 483, 531.
  3. Moses, H., Dorsey, E. R., Matheson, D. H., and Their, S. O. JAMA, 2005, 294, 294.
  4. Dorsey, E. R., de Roulet, J., Thompson, J. P., Reminick, J. I., Thai, A., White-Stellato, Z., Beck, C. A., George, B. P., and Moses, H. JAMA, 2010, 303, 137.
  5. Nature, 2013, 496, 398.
  6. Interview conducted in internal case studies

Comments

Video and the Process of Education (posted for Harry Pence)

For generations the process of education has conformed to the process of society.  In both cases, communication mainly occurred either verbally or by means of writing on paper or chalkboard.  This meant that the educational process appeared to be a comfortable and familiar environment to the students who encountered it.  This equivalency is no longer true.  In the modern world, communication frequently occurs electronically, often by a text message or an open Skype connection.  Videos, which have become part of the human experience comparatively recently, are now common.  YouTube, by no means the only video server on the web, has hundreds of millions of videos and hundreds of thousands of new ones are uploaded every day.   All levels of learners find videos to be advantageous because the viewer can repeat a confusing section as often as he or she likes, without fear of complaint from the presenter.  This is just as true for a novice as it is for an experienced researcher who is trying to learn a new technique.  And most importantly, you can actually see what is happening, not just try to recreate it in your imagination from a text.  And more and more people are not just viewing videos but creating them. 

When a student walks into a classroom or a scientist tries to learn a new technique, he or she enters a world where these processes are no longer routinely available.  JoVE is a small step towards making the process of formal learning more similar to the process of everyday experience.  The question is not whether JoVE is a good thing, it obviously is, the real question is how much longer the process of education will be limited to traditional media rather than expanding to include the varied new media environments that people encounter in their lives.    

Harry E. Pence
SUNY Distinguished Teaching Professor Emeritus
SUNY Oneonta

Response

Dear Harry, 

Thank you for your comments and thoughts on the progress of information sharing and learning in the digital age. We couldn’t agree with you more that video learning has become more common as video tools are more readily available. As you point out, the ability to pause, rewind, and repeat has become a natural part of absorbing new information. This is crucial for a diverse group of researchers who will have extremely varied levels of experiences and will thus require different learning needs.

To touch upon the real issue, “how much longer the process of education will be limited to traditional media rather than expanding to include the varied new media environments that people encounter in their lives,” is a great topic, and we are glad that you have brought it to the attention of this newsletter. A few liberal, private colleges and universities have already started to approach this issue and are moving towards digital classrooms. Just this year, my alma mater, Moravian College, has converted to a “mac” college and built the cost of a macbook pro laptop computer into the cost of tuition. This branding at the college allows professors to structure their lectures around this technology without being concerned with the accessibility question. Certainly, this raises financial concerns as macbook pro laptops are not the most affordable options. However, their initiative is an important one and will likely convert into a higher rate of learning both inside and outside of the classroom. How many other institutions will implement similar practices? How long before all institutions and educators seriously consider restructuring the classroom to include the tools of technology? Educators will most likely drive the support and push for progress.

Cheers,
Val and Kira

Interesting indeed!

I just wish something similar could be done for research in social sciences. Or can JOVE be useful there too?

(The first thing that came to my mind on reading the article was the series "Is it a good idea to microwave <anything>" of videos (http://www.youtube.com/show/isitagoodideatomicrowavethis). Apologies for that, Val and Kira. No comparison of any kind suggested here, of course.)

  1. But why not publish through an existing journal and make a video (with all care taken) available separately? Why would it not achieve the same thing?
  2. Does JOVE address the basic reasons behind irreproducibility?
  3. Wouldn't a video reduce inclination among others to attempt reproducing the result in their own facilities? (Any statistics here?) What happens if they do and the result is still irreproducible? Does JOVE take any more responsibility here than other journals?
  4. Do you provide an edited video or do you provide unedited footage (also)? Do you start from calibration of equipment?
  5. Can your technique be combined with open lab notebooks? If possible, you could invite reviewers (or some notables) to watch the procedure through live streaming as it is being filmed and allow them to interact with the researchers and maybe also draw attention of the video team to complexities they feel are relevant. This of course would impact the original script.
  6. If other researchers create videos of their attempts at reproducing the results, do you offer a repositary so that other viewers of the original get access to these other videos too?

(I have so far not watched the videos because of connectivity/hardware issues. I suppose the above questions would not have arisen had I done so.)

Regards

~Milind

Response

Dear Milind,

Thank you for your comments. Let me first give some background about our journal and our publication process, which may clarify a few points as well as answer a few of your questions. Since JoVE publishes video articles, the publication process must stray from the traditional model with an emphasis on a rigorous internal review. Internal review of the text manuscript is completed by at least two PhD science editors at JoVE to ensure that the text complies with the format of our journal. Particularly, the protocol section must be written with enough detail to describe how to complete the action being described. After completing internal review, each article is sent out for peer review, where it is reviewed by at least two peers in the field. Upon completion of peer review, a PhD-level scriptwriter translates the manuscript text into a storyboard (which includes a script) and sends it to the author for approval to ensure that the method is presented accurately. Once approved, the video is filmed. The raw film footage is sent to JoVE headquarters, where voice-over talent is added and the footage is edited to present a cohesive story. A different PhD JoVE science editor and the authors review the final product before both the text manuscript and video are published. In total, at least four in-house PhD scientists and two external PhD peer reviewers scrutinize each article to ensure that each method is presented to a high standard of scientific rigor. Research scientists have the ability to make and distribute their own videos through video sharing websites like YouTube. However, these videos and the content in them are not reviewed for quality standards or scientific accuracy.

This multi-stage process is necessary to ensure that both the video and text component of each video article meets our scientific quality standards. As you might imagine, this process would not be conducive to converting traditional text articles into video articles. However, many of our authors publish a methods video article in JoVE as a compliment to their novel results publication.

Addressing JoVE’s dedication to increasing reproducibility in science, we do not conduct any internal replication studies. What we strive to achieve with our video articles is increased transparency. Through disseminating intricate details of complex methods in dynamic video format, all the necessary information to reproduce the methods shown is provided to the reader/viewer.

Thanks for bringing these questions/comments to the conversation. 

Cheers,
Val and Kira

 

 

 

 

Thanks for the additional insight.

Dear Val and Kira,

I appreciate your response and your focus on transparency.

However, some of my questions remain.... I hope they get answered in bits and pieces as the discussion goes ahead.

Thanks.

More on irreproducibility

 Hi Milind,

I’d like to chirp in here and comment that some of your questions are very good, but I doubt can be answered in this Newsletter.  Like “does JoVE address the basic reasons behind irreproducibility?”  That topic, in and of itself, would require an entire symposium to address, and there would still be unanswered questions.

But for the “bits-and-pieces”, I can see some that JoVE may address and others it may not. One of the problems with the peer-review system is that few reviewers can justify the time to review work at the level of repeating an experiment. JoVE does require the experiment to be performed and documented on film. But what if the experiment is staged?  What if the equipment was faulty? Who buys/generates the reagents? Calibrates the equipment? And what if the expert being reviewed is a person of authority whom you would want to believe and give the benefit of doubt?

Here is an interesting blog post of a video where Tony Williams made “blue water balls” (as an April Fool’s joke), with the blog commenting on how we may place too much trust in authority.

http://www.chemconnector.com/2013/04/02/do-we-place-too-much-trust-in-experts/

If I was invited to review a Noble Laureates’ work, or take a video in his/her lab, I’d be inclined to trust that person and his/her equipment, and question my own knowledge, if what they report does not quite seem right. So although I suspect the largest problem with the peer review system is that reviewers’ seldom can justify the time needed for a thorough review, there is also an element of trust in whom is being reviewed. And I am not saying this need be an issue of intentional deceit, we also have trust in our equipment, and the person performing the experiment may naively be using faulty equipment and believe in the false results being reported. In this situation, going to that person’s lab and filming that person using that equipment would not rectify the problem. But I do not think this is the reason so many publications are irreproducible, and JoVE is a step in the right direction. I like Kira’s comment that many of their authors report results in traditional publications, and methods in JoVE. That is, using JoVE to support the traditional publication.
Cheers,
Bob

 

Thanks, Prof. Belford.

I just wondered whether it can be taken a step or two forward...

One by marrying this with Open Lab Notebook! We could call it an Open Lab Shoot, if you wish. Whether the Open Lab Notebook movement is getting anywhere is another question...

 

Another by also hosting/linking videos of successful/unsuccessful repetition, or repetition with differing equipment along with the original video.

But of course, JoVE can do only this much on their own.

Thanks, again.

~Milind

Response

Dear Bob and Milind,

First, as Bob points out, peer reviewers are not likely to spend the time replicating the experiments within the paper they are reviewing. Flaws in the peer review process have recently come to light, such as the “sting” operation published in Science Magazine wherein a spoof manuscript was accepted by 157 journals (http://www.sciencemag.org/content/342/6154/60.full).

To specifically comment on the question,“does JoVE address the basic reasons behind irreproducibility?” we are currently in the process of trying to collect information that answers this question. The case study we are designing is targeted at users of published video articles. Ideally, we will be able to quantify the reproducibility of the methods published. However, this will be a lengthy process and will depend largely on the willingness of the user to share his or her experiences with us.

Addressing your questions about responsibility, no, JoVE does not take more responsibility than other journals if published methods are irreproducible. Unfortunately, we do not have the resources or time to reproduce the methods in-house and rely on authors and peer reviews for field-specific experimental details. The reproducibility initiative, spearheaded by the Science Exchange (who just received a $1.3 M grant), tackles the reproducibility question on a case-by-case basis and devotes both time and resources to replicating experiments from published work.

What we can say about JoVE, however, is that through the process of visualizing the experiment, authors are forced to include details that are normally not reported in traditional text manuscripts. We also require that all authors focus on commenting on critical steps, limitations, trouble-shooting, modifications, and impact to the field in the discussion section. In this way, if a researcher is having trouble reproducing the method, he or she will have some guidance in making necessary adjustments.

Admittedly, we are unfamiliar with open lab notebooks and cannot add too much to your questions about combining video articles with this resource. Currently, we are saturated with our section content and our new science education collection and wouldn’t have the staff to engage in a new project. We are still a growing journal and are trying to spread awareness and support, however, we will certainly look into this suggestion.

We would like to close in saying that we have thoroughly enjoyed this discussion and would happily continue any discussions through private communications, such as email.

Cheers,
Val and Kira

One last question from me: Why "Visualized"

Hi Val, Kira,

The word "visualized" is often used in the sense of "creating" or "conjuring up" to come up with models/imagery/mental picture of something that is not easily understandable, typically because of higher number of dimensions or rapidfire/shortlived transformations, or of somethings that are nothing but theoretical postulations. It implies an attempt at simplistic rendering, at dispensing with rigour for the sake of mass understanding. It reminds one of sci-fi, virtual reality etc. At least this is how I visualize the word!

JoVE is intent on providing a faithful record of something that is happening and not leaving anything to imagination. Why did you choose the word to stand for the V in JoVE?

Of course, what is in a name?

Finally... Thanks for JoVE and for this discussion.

Chemical Safety Feature in JoVE?

Dear Val and Kira,

I went and looked at some of the general laboratory techniques and saw this video on an Introduction to Working in the Hood, and it led me to several questions/thoughts. In chemistry there are several organizations like ACS CHAS (Division of Chemical Health and Safety), that are devoted to ensuring a culture of safe and prudent practices in the field. I saw your video

http://www.jove.com/science-education/5036/an-introduction-to-working-in-the-hood and felt it had inconsistent usage of PPEs, as if the video was not following the transcript.  Sometime goggles were on, sometimes not, one lady had hair in her face, while the very next one used a hair net. The other thing I was surprised is that there were no references directing people to further resources.

Now I realize the above videos are of a different nature than a person’s publication of a new and novel technique, and that you appear to have made these open access – which is good as everyone deserves information on material like safety. My thought is, have you thought of teaming up with a society or group of safety professionals to develop videos on laboratory safety? I know there are several people collecting video resources on the web, like those in the Google Doc this C&E News blog links to,

http://cenblog.org/the-safety-zone/2013/06/lab-safety-video-library/

I think an open access section on chemical safety in JoVE would be of value to the chemical education community.

Cheers,

Bob

 

Chemical Safety Feature in JoVE?

The challenge with a chemical safety library is defining the audience and the subsequent level of detail that can be addressed in a video. For example, I found the "hood" video that Bob refers to very confusing, as it mixed three types of hoods together (laminar flow hoods, biosafety cabinets and fume hoods) and the statements made do not apply to all three. For example, few fume hoods rely on laminar flow to contain contaminants in them. And laminar flow hoods do not have sashes, as the closing statement suggests.

This approach to discussing lab hoods may make sense in a biological lab context, but would be very confusing to chemists used to working in laboratory chemical hoods. And the challenge with regard to lab safety information is that as laboratories are becoming increasingly interdisciplinary, defining the audience is becoming more challenging.

Chemical safety

Dear Bob and Ralph,

You raise some very relevant points that are already on our radar as we work to improve our content. There is very large editorial oversight when we produce each of our videos in order to make sure the visual content is scientifically rigorous. Each video article (text and video component) is reviewed by at least 4 scientists at JoVE in addition to several peer reviewers. However, the videographers themselves are not scientists. The adherence to safety standards within a video article is often dependent on the authors themselves. One thing I have learned since working in publishing is that while there are some uniform safety standards, many guidelines are field specific. For example, in my field of cell biology where we worked with mammalian cell lines, mouth-pipetting would never be acceptable. However, we have recently accepted an article where the authors and peer reviewers agree that mouth-pipetting is the standard for some non-mammalian cell culture techniques using C. elegans and Drosophila.

That said, we do strive for our content to maintain the utmost scientific practices and stringent adherence to safety regulations. I have recently been in contact with a few safety organizations to work towards collaboration. I was approached by a representative from CSHEMA (Campus Safety Health and Environmental Management Association) who may help us make video articles focused explicitly on laboratory safety. We will also reach out to your recommendations, like ACS CHAS, as we would like to include more chemical safety content.

As an aside, we would like to thank you both for your candid feedback. Our goal with the Science Education collection is to create content that is both relevant and accurate for a broad range of educators. We will bring your feedback to our production staff and reevaluate the hood video.

Best Regards,

Kira and Val

Response

Dear Bob,

The issue of reproducibility has been surfacing in both the primary literature through journals, such as nature, and secondary literature more accessible to the public, such as the article you recently stumbled upon in the economist. True, on a fundamental level, research can be defined as the process of searching for answers to the posed question repeatedly. However, published works should be reserved for findings that have been shown to be reproducible through this exhaustive process. Unfortunately, the push to publish novel results is stronger than the push to reproduce these results. Consequently, static journals are structured to support the sheer volume of publications and not to support the intricate details of the methods used to find these results. More and more, the crucial experimental procedures are buried in dense paragraphs available to the reader in supplementary documents. These supplementary files are often largely ignored by peer reviewers, meaning these important details are often published without critical review. This point is best demonstrated with a recently published Organometallics paper (http://pubs.acs.org/doi/abs/10.1021/om4000067), wherein the supplementary material describing the synthesis of an organometallic compound is published with the following statements: “please insert NMR data here! where are they? and for this compound, just make up an elemental analysis…” These sentences passed through both the peer review process and an internal review by the editors handling this particular paper without correction.

Video articles place emphasis on the experimental details. A dynamic presentation of methods shows a level of detail that cannot be achieved with static text. Transparency of these methods is crucial to replicating and validating the results obtained. To specifically answer your questions regarding the storyboards, after a manuscript passes peer review, JoVE science editors draft a storyboard, which includes a script for voice-over, for each video produced by JoVE. The storyboard is sent to the authors for approval prior to filming. If an author chooses to make his or her own video, JoVE sends both the video and the text manuscript to peer review at the same time, so the storyboard depends on author direction and peer review approval.

Cheers,
Val and Kira

 

Article in Economist

Dear Val and Kira,

While reading your article I was thinking you were hitting a bit hard on the reproducibility of published work, but an article in last week’s economist was just brought to my attention that clearly supports your premise.

http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble

This article points out that “replication is hard and thankless”, …  it “can be seen as an implicit challenge to authority. Often, only people with an axe to grind pursue replications with vigour – a state of affairs which makes people wary of having their work replicated”.

In many ways JoVE’s videos bring replication into the publishing process. I note that there is the ability of authors to create their own videos.  When you make videos, do you put together a storyboard, and if so, do you review the storyboard of authors who generate their own videos?

Thanks again for bringing many of these issues to our attention.
Cheers,
Bob

Visions for JoVE in Chemistry Education

Dear Val and Kira,

Thank you for this very interesting and thought-provocative article. I truly found it most worthwhile reading, and fear I have more questions than I dare ask. But let me start the discussion with a basic question. I note the oldest archive dates to October 2006, and that JoVE Chemistry started in February of this year. As the majority of the readers on this list are chemical educators, I would like to know a bit on the vision for the role JoVE could play in chemical education.

I went and watched the short video instructions for submitting an article,
http://www.jove.com/publish?utm_source=Button1#2.1

I sort of have two questions here.  Who would publish in JoVE, and once published, who would they reach? That is, can you describe the selection process with respect to approving articles for publication? Especially as to what types of articles might be appropriate with respect to chemistry education. But also, getting to the nuts-and-bolts, isn’t this a very expensive publishing platform? One of the things about chemical educators is we work in schools across the fiscal spectrum, from high schools to community colleges to Universities with minimal resources to those with large endowments. The one thing all these schools have in common is students, and the need to provide the best education possible.  If one was to publish an education based experiment in JoVE, would it be available to this broad community? And what type of educators would be most appropriate to publish in JoVE?

Thanks again for this very enjoyable and thought provoking article.
With Respect,
Bob Belford

 

Response

Dear Bob,

Thank you for the comment and questions. We have several answers for you and would like to start by giving a bit of background. JoVE was co-founded in 2006 by a biologist, Dr. Moshe Pritsker. Naturally, Dr. Pritsker founded a journal focused in his own specialty, but as JoVE began to gain popularity, we have branched out to encompass a variety of fields. In 2011, JoVE began publishing its first content in the physical sciences with "Bioengineering", which bridges the biological and engineering worlds. This was later followed up with "Applied Physics" and more recently "Chemistry" and "Environment." We hope to continue to expand and include more content from the physical sciences.

Our mission as a video methods journal is to increase the transparency of research protocols so that researchers can learn and share science more efficiently. With that in mind, here are answers to your specific questions:

1. Who would publish in JoVE, and once published, who would they reach? That is, can you describe the selection process with respect to approving articles for publication? Especially as to what types of articles might be appropriate with respect to chemistry education. 

Both researchers and educators publish in and use JoVE video articles. Our criteria for publication are that the technique presented be scientifically rigorous and, frankly, useful for other scientists. This means the technique does not necessarily have to be novel; it can be a "gold standard" technique mastered in a lab or a new application of an older technique. Standard results-based articles provide very little space for materials and methods explanations, so many researchers choose to publish their methods in JoVE as a complement to novel results papers. Several of our authors have come to us because they received significant criticism on a recently published article when other researchers were not able to replicate the results. After publishing their methods in JoVE, our authors are able to validate their results publications.

Educators will often publish in JoVE in order to capture a visual laboratory concept or to maintain a complicated technique. One of our repeat authors, Dr. Robin Cooper, is a physiology professor from the University of Kentucky, and he has published both methods articles and teaching modules in JoVE. Here is an example of one of Dr. Cooper's laboratory exercises that he published in JoVE. Dr. Cooper uses his JoVE videos to help prepare his own students for his lab courses, and he shares the video with colleagues to help them teach similar courses around the country. By capturing the laboratory exercise in a JoVE video, no details are lost due to distance or a different professor's teaching style.

Both educators and researchers use JoVE's content to learn new techniques in or outside of their field, train new students and expose students to a variety of research applications. In addition to our peer reviewed content, we have recently released a section of basic laboratory techniques called "Science Education Collection." This section combines schematics and live video to teach new lab students basic research practices, from how to use a pipette, to how to run a gel and introduction to animal models. This content is currently focused on biological techniques, but we will soon be expanding to include more basic chemistry components as well.

2. But also, getting to the nuts-and-bolts, isn’t this a very expensive publishing platform? One of the things about chemical educators is we work in schools across the fiscal spectrum, from high schools to community colleges to Universities with minimal resources to those with large endowments. The one thing all these schools have in common is students, and the need to provide the best education possible.  If one was to publish an education based experiment in JoVE, would it be available to this broad community? And what type of educators would be most appropriate to publish in JoVE?

Publishing in JoVE is comprable to publishing in a more standard text-only journal. It costs JoVE approximately 10,000 USD to produce each JoVE video (costs of videographers, editing and publishing), but we ask our authors to contribute only a fraction of this. We offer 2 publication options: standard access (2400 USD), which means the viewer must belong to an institute with a subscription, and open access (4200 USD), which means the content is available to everyone. We are able to support the remainder of our author's publication fees by maintaining a subscription model. That said, there are several options for funding a JoVE article. Many of our research authors will apply for sponsorship from a biotech company in exchange for a license to the video. For example, if a researcher is using a Zeiss microscope in their research, Zeiss may choose to sponsor the article because the final product (a scientifically rigorous, peer reviewed video article) is great advertising for Zeiss' product. In the case of sponsorship, the company does not gain any editorial oversight, and the article remains academic and unbiased. Another option that many of our authors also pursue is educational grants for video publication. Many funding agencies recognize the benefits of video publication for increasing scientific knowledge transfer and encourage video publication through grant funding. We have several authors who have chosen the educational grant route and are able to publish their laboratory protocols in open access so that other educators may also benefit from the video publication.

Cheers,

Kira and Val