+ - 0:00:00
Notes for current slide
Notes for next slide

Research in the era of (ir)reproducibility and Open Science

1 / 18
  • Openness / Transparency
  • Quality: Reproducibility and replicability
  • Collaboration / Team work / Remote work
  • Communication / Public Engagement / Social Media
2 / 18

We are in the middle of several exponential growth curves:

  • Data production
  • Data storage and transfer
  • Computing power
  • Published research
  • Complexity of methods (e.g. AI)
3 / 18
  • Industrialisation of the research work flow
  • Specialisation in research tasks

4 / 18

5 / 18
  • Changes in the way we work:

    • Remote work
    • Online communities
    • Ad hoc teams
  • Research on research:

    • Meta analysis (your output is somebody else's input)
    • Metaresearch: evidence-based evaluation and development of research methods
6 / 18

How do we move towards reproducibility?

Peng RD. Science 2011. DOI: 10.1126/science.1213847.

7 / 18

First computationally reproducible paper (eLife - 2018)

Lewis LM. eLife 2018;7:e30274 DOI: 10.7554/eLife.30274.

8 / 18

Some journals are moving in this direction

PLOS Computational Biology is running a pilot:

"We will soon be able to offer expert technical peer review specifically checking that submitted systems biology or physiology-based models run according to the results presented in the manuscript submitted to the journal. The peer review will be delivered in addition to our usual scientific assessment.."

9 / 18

Developments in making manuscripts interactive

10 / 18

There are still very strong barriers

11 / 18

There are still very strong barriers

  • Lack of awareness
  • Tradition, culture and common practices need to change
  • 'Business as usual' seems the shortest route to success
11 / 18

There are still very strong barriers

  • Lack of awareness
  • Tradition, culture and common practices need to change
  • 'Business as usual' seems the shortest route to success

  • Tools and training in specific skills needed

  • Takes time and dedicated funding
11 / 18

There are still very strong barriers

  • Lack of awareness
  • Tradition, culture and common practices need to change
  • 'Business as usual' seems the shortest route to success

  • Tools and training in specific skills needed

  • Takes time and dedicated funding

  • Researchers need to see the value in adopting an open reproducible workflow

  • Funding and reward systems need to be adapted:
    • Peer review and Publication
    • Academic recognition / careers
    • Research funding mechanisms
11 / 18

There are still very strong barriers

  • Lack of awareness
  • Tradition, culture and common practices need to change
  • 'Business as usual' seems the shortest route to success

  • Tools and training in specific skills needed

  • Takes time and dedicated funding

  • Researchers need to see the value in adopting an open reproducible workflow

  • Funding and reward systems need to be adapted:

    • Peer review and Publication
    • Academic recognition / careers
    • Research funding mechanisms
  • Law: privacy concerns about sharing data, IP protection, patents, etc

11 / 18

Personal barriers

Fear of:

  • Scooping or ideas being stolen
  • Not being credited for ideas
  • Errors and public humiliation
  • Risk to reputation
  • Reduced scientific quality
  • Information overload

Tennant (2017)

12 / 18

Current scientific culture not prepared for analytic and computation era

13 / 18

Open science debates and initiatives don't recognize role of software

E.g. EU H2020 Open Science Mandate only mentions data and publications.

14 / 18

Little to no training in software or programming

Source from xkcd.

15 / 18

Knowing the space and finding an efficient workflow

Figure from Innovations in Scholarly Communication.

16 / 18

What does it mean for you?

17 / 18

What does it mean for you?

  • Find and collaborate with those familiar with these concepts (online and/or in real life)
17 / 18

What does it mean for you?

  • Find and collaborate with those familiar with these concepts (online and/or in real life)

  • Cite research that is or tries to be more reproducible

17 / 18

What does it mean for you?

  • Find and collaborate with those familiar with these concepts (online and/or in real life)

  • Cite research that is or tries to be more reproducible

  • Keep the principles of reproducibility in mind, then find the tools

17 / 18

What does it mean for you?

  • Find and collaborate with those familiar with these concepts (online and/or in real life)

  • Cite research that is or tries to be more reproducible

  • Keep the principles of reproducibility in mind, then find the tools

  • Practice reproducible and open science

    • More on this later in session
17 / 18

Recognise importance of code and data: Cite them!

18 / 18

Recognise importance of code and data: Cite them!

# Example:
citation("dplyr")
##
## To cite package 'dplyr' in
## publications use:
##
## Hadley Wickham, Romain
## François, Lionel Henry and
## Kirill Müller (2020). dplyr:
## A Grammar of Data
## Manipulation. R package
## version 1.0.0.
## https://CRAN.R-project.org/package=dplyr
##
## A BibTeX entry for LaTeX users is
##
## @Manual{,
## title = {dplyr: A Grammar of Data Manipulation},
## author = {Hadley Wickham and Romain François and Lionel Henry and Kirill Müller},
## year = {2020},
## note = {R package version 1.0.0},
## url = {https://CRAN.R-project.org/package=dplyr},
## }
18 / 18
  • Openness / Transparency
  • Quality: Reproducibility and replicability
  • Collaboration / Team work / Remote work
  • Communication / Public Engagement / Social Media
2 / 18
Paused

Help

Keyboard shortcuts

, , Pg Up, k Go to previous slide
, , Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow