Get your files and folders in order (step 1)

E-mail course: 6 steps towards reproducible research

Oct 28, 2022

Hi there!

Great to see you back here 😃

Today is all about getting our files and folders organised ✨

Step 1: Get your files and folders in order

My first research project was a mess 🙈. I had hundreds of files with dubious file names and sometimes several files with similar code written for computing on different infrastructures (my computer, the institute server, the cluster of the computing facility).

I felt like the worst researcher of all times. But I wasn’t. Many struggle with organising their files and folders in increasingly complex research projects.

Frustrated person in front of laptop — Photo by Andrea Piacquadio from Pexels: https://www.pexels.com/photo/woman-in-white-shirt-showing-frustration-3807738/

How to organise files and folders well?

It basically comes down to structuring folders and files systematically from the beginning.

Think about what a good folder structure could be for your research project. A standard project of mine looks something like this:

├── Makefile ├── README.md ├── analysis │ ├── analysis_modelling.R │ ├── anaysis_summary-stats.R │ ├── basis │ │ ├── functions_analyses.R │ │ └── functions_simulation.R │ ├── data_clean │ │ ├── clean_data.R │ │ ├── data_study1.csv │ │ ├── data_study2.csv │ │ ├── metadata_study1.xml │ │ └── metadata_study2.xml │ ├── data_raw │ │ ├── data_study1_raw.csv │ │ └── data_study2_raw.csv │ ├── playground.R │ └── simulation.R ├── journal-correspondence │ ├── journal-1 │ │ ├── p2p_1.txt │ │ └── response_1.txt │ └── journal-2 │ └── p2p_2.txt ├── paper-manuscript.Rnw └── paper-references.bib — File tree: blue = folders, white = files

Not every project is the same and likely your project will be more complex than this. But if you think about good organisation from the beginning, it will be easier in the long run.

What do you think about file oder folder organisation? Is your foldor structure similar to mine? Let’s discuss here or on Twitter!

Your task

Check one of your current research projects and get it organised 💪:

Does your folder structure help you and others find files fast?
What would a better structure look like?
If the project is a collaborative effort: suggest your new structure to your collaborators. Do they like it or do they have better ideas?
Implement the new structure 🚀

Next up

In the next newsletter we will discuss step 2: Use good names for files, folders, functions, …

Reproducible Research: 6 helpful steps. (1) Get your files + folders in order; (2) Use good names for files, folders, functions, …; (3) Document with care: README, Metadata, code comments, …; (4) Version control code, text, …; (5) Stabilize computing environment and software; (6) Publish your research outputs: Code, data, documents, …

All the best,

Heidi

P.S.: Did you know I am organising an Open Science Retreat? Come and join us!

Heidi’s Newsletter

Discussion about this post