Table 2 Operationalisation of the analysed tools

ARRIVE [19]

 A1a

Y/N

With Y, it is clear which groups were subject to specific experimental conditions

 A1b

Y/N

With Y, it is clear what is used as a unit of analysis. If repeated measures were available, it needed to be clear how they were analysed, e.g. as separate repeated measures or as a mean value

 A2a

Y/N

N per group and total are clear. If ranges were reported, we scored N

 A2b

Y/N

Report of any a priori sample size was sufficient for a Y

 A3a

Y/N

If potential criteria were described as demographic information only, we scored N

 A3b

Y/N

If any exclusions were explicitly stated, we scored Y, even though there might have been more

 ^a3c

-

Merged with A2a and A1b

 ^a4a

-

Merged with S1

 ^a4b

-

Merged with S4 and S6

 a5

-

Merged with S3, S5, and S7

 A6a

Y/N

All assessed outcome measures were explicitly mentioned for a Y

 A6b

Y/N/I

The primary outcome measure, or that used for sample size calculation, was explicitly mentioned

 A7a

Y/N/I

Methods AND software had to be mentioned for a Y

 A7b

Y/N/I

Explicit or implicit (results only) mention of assumption testing resulted in a Y

 ^aA8a

-

Merged with A9, C15, and S2

 ^aA8b

-

Merged with A9, C15, and S2

 A9a

Y/N

The extractors both had to feel sufficiently informed to initiate reproduction of the experiment. This comprised knowledge of the species, strain, age, sex/gender, health status, etc

 A9b

Y/N

For a Y, information needed to be provided to know when the study was performed, up to the day of the week. E.g. “between June and September” [20] was scored N

 A9c

Y/N

The laboratory needed to be clear for a Y, and for larger universities and hospitals with multiple laboratories, we scored N

 A9d

Y/N

The rationale of at least two parts of the experimental design needed to be explained explicitly

 ^a10a

-

Merged with A6b and A7a

 ^a10b

-

Merged with A6b and A7a

 ^a11

Y/N

N if any of the requested elements (mostly strain and sex) was missing from the abstract. For human studies, we ignored strain

 ^a12a

Y/N

The extractors needed to understand the research question and its relevance for a Y

 ^a12b

Y/N/I

The model validity had to be explicitly described for a Y. Not scored for human baseline studies

 ^a13

Y/N

The objectives needed to be clear to both extractors for a Y

 ^a14

Y/N

The registration number of the ethics proposal and the name of the committee needed to be provided for a Y

 ^a15

Y/N/I

The extractors needed to have a reasonable idea of the inside of the cages for a Y. Mention of “standard” housing types could be sufficient. Not scored for human studies

 ^a16a

Y/N/I

Any mention of refinement other than anaesthesia was sufficient for a Y. Not scored for human studies

 ^a16b

Y/N

Any mention was sufficient for a Y

 ^a16c

Y/N/I

Any mention was sufficient for a Y. Not scored for human studies

 ^a17a

Y/N

Interpretation had to relate explicitly to theory/hypotheses/background literature/experimental set-up for a Y, and an overview of the results without interpretation resulted in an N

 ^a17b

Y/N

Any explicit mention or at least two implicit mentions of limitations for a Y

 ^a18

Y/N/I

Interpretation described the extent of external validity, either implicitly or explicitly, for a Y

 ^a19

-

Merged with S9

 ^a20

Y/N

We scored Y if the data were available where stated

 ^a21a

Y/N

Y with explicit mention of conflicts or the absence thereof

 ^a21b

Y/N

N if any of the requested elements (mostly involvement of the funder) was missing

Cochrane’s RoB tool [12]

 ^a1

-

Merged with C8b, C15, S1, and S2

 ^a2

-

Merged with S2–S6

 ^a3

-

Merged with A2A and S8

 ^a4

-

Merged with S6 and S7

 ^a5

-

Merged with S9

CONSORT statement [15]

 C1a

Y/N

Y if the title mentions the type of study. “Animal study” was not considered enough and “phenotypic model characterization” was

^a1b

-

Merged with A11

 ^a2a

-

Merged with A12a

 ^a2b

-

Merged with A12b

 C3a

Y/N

Y if the experimental study design was mentioned. Allocation ratio was merged with A2a

 C3b

Y/N

Y if (absence of) protocol deviations were explicitly mentioned

 ^a4a

Y/N

Merged with A3a

 C4b

Y/N

More lenient than A9c; Y if we were certain of the type of settings (e.g. “laboratory” or “hospital”)

 ^a5

-

Merged with A9a

 ^a6a

-

Merged with A6b

 ^a6b

-

Merged with A6a and C1a

^a7a

-

Merged with A2b

 C7b

Y/N/I

Any mention of interim analyses and/or stopping rules was sufficient for a Y. I for studies with a single measurement and explicitly short duration

 ^a8a

-

Merged with S1

 C8b

Y/N/I

The type of randomisation had to be mentioned. Irrelevant for studies without intervention

 ^a9

-

Merged with S3

 C10

Y/N

Y: It was clear who did what in group allocation. Irrelevant for studies without intervention

 ^a11a

-

Merged with S5

^a11b

-

Merged with S5

 ^a12a

-

Merged with A7a

 C12b

Y/N/I

Y with minimal description of the methods for additional analyses. I for studies prespecifying a single analysis

 ^a13a

-

Merged with A2a and S8

 ^a13b

-

Merged with A2a and S8

 ^a14a

-

Merged with A9b

 ^a14b

-

Merged with C7b

 C15

Y/N

For a Y, baseline data needed to be provided at group level for at least age/weight and genetics

 ^a16

-

Merged with A2a

 ^a17a

-

Merged with A10a and A10b

 ^a17b

-

Merged with A10a and A10b

 ^a18

-

Merged with C12b

 ^a19

-

Merged with A16b

 ^a20

-

Merged with A17b

 ^a21

-

Merged with A18

 ^a22

-

Merged with A17a

 ^a23

-

Merged with S9

 ^a24

-

Merged with S9

 ^a25

-

Merged with A21b

SYRCLE’s RoB tool [13]

 S1

L/U/H/I

“Randomly picked from the box” would score U. I for studies without interventions. Allocation sequence generation was not scored for noninterventional studies

 S2

L/U/H

For an L, baseline data needed to be comparable for at least age/weight, sex/gender, and type of mutation for intervention studies or genetic background for CF-control studies

 S3

L/U/H/I

The investigator allocating animals/participants was adequately blinded for an L. Latin-square-like designs would always have scored an H. I for human studies without interventions. For animal model studies, we could have scored potential bias for the model generation here (but in the rare cases where it was relevant, it was U)

 S4

L/U/H/I

Was there RoB related to the animal housing? In theory, Latin-square-like cage placement would have scored an L here. I for human studies

 S5

L/U/H

The investigators performing and/or caring for subjects during and between the experiments were all adequately blinded for an L

 S6

L/U/H

Was there RoB related to the outcome assessment (order/method)? In theory, counterbalanced orders would have scored an L here

 S7

L/U/H

The outcome assessors were all adequately blinded for an L

 S8

L/U/H

For an L-score, the data had to be either explicitly complete, or incomplete outcomes had to be equally distributed over the groups

 S9

L/U/H

We were more strict than the tool here and always scored H if no protocol was posted

 S10

L/U/H

We scored H if the methods were unclear or caused reasons for concern at points not addressed in any of the other elements