How to Prepare Replication Packages for Reviewers?

As computational methods become central to research, a manuscript increasingly represents only part of the evidence behind a scientific claim. Data, source code, software environments, model configurations, documentation, and analysis instructions may all be necessary for another researcher to understand how reported results were produced. A well-prepared replication package brings these elements together in a structured and reviewable form.

For authors working in computer science, artificial intelligence, machine learning, IoT, cybersecurity, data science, software engineering, and related fields, preparing such a package before submission can strengthen methodological transparency and make the peer-review process more efficient. This approach is consistent with the broader reproducibility practices promoted by major scholarly publishers and with the emphasis on complete, reproducible methods in the author guidance of the Ubiquitous Technology Journal (UTJ), published by Crosslink Studies.

What Is a Replication Package?

A replication package is an organized collection of the materials required to inspect, reproduce, or extend the computational results reported in a research article.

Depending on the study, it may contain:

• Source code and scripts
• Raw or processed data, where legally and ethically shareable
• Configuration files
• Software and package requirements
• Model parameters and experimental settings
• Statistical analysis scripts
• Tables and figure-generation scripts
• Documentation and README files
• Sample or synthetic data
• Version information
• Instructions for reproducing the reported results

The objective is not simply to provide “all project files.” A useful package should allow a technically competent researcher to understand what each file does, what inputs are required, how the workflow should be executed, and which outputs correspond to the manuscript.

Start with the Results, Not the Files

One common mistake is to collect files at the end of a project without establishing what the package needs to reproduce.

Instead, begin with the manuscript’s major results.

For each important table, figure, statistical result, simulation, machine-learning experiment, or performance comparison identify the exact data, code, parameters, and processing steps required to generate it.

A useful internal mapping might look like:

Figure 2 → preprocessing script → processed dataset → model script → configuration file → output

This approach prevents an apparently complete package from containing files that are unrelated to the published analysis while omitting a small but essential script.

1. Organize the Package Clearly

A reviewer should not have to search through dozens of files to discover where the analysis begins.

Avoid including unnecessary temporary files, personal folders, old drafts, cached outputs, passwords, API keys, or unrelated experiments.

The principle is simple: organized research materials are easier to inspect, reproduce, and preserve.

2. Make the README the Starting Point

The README is often the most important document in a replication package. It should provide a concise roadmap covering:

Study information

• Article title
• Authors
• Version of the package
• Repository or persistent identifier

Requirements

• Operating system, where relevant
• Programming language and version
• Required software
• Libraries and package versions
• Hardware or GPU requirements, if necessary

Data

• Data sources
• File descriptions
• Preprocessing procedures
• Restrictions on access or redistribution

Execution

• Installation instructions
• Order in which scripts should be executed
• Input and output locations
• Commands required to run the analysis

Reproduction

• Which script produces each table or figure
• Expected outputs
• Approximate execution requirements, where useful

A README should answer the reviewer’s most important practical question: “What do I need to do to reproduce the results?”

3. Document the Computational Environment

Code can fail even when the underlying methodology is correct because software environments change. A package may depend on a particular version of Python, R, MATLAB, Stata, Java, TensorFlow, PyTorch, or another framework. A library update can also change results or cause an older script to stop working.

Researchers should therefore record programming language and version, major software versions, package/library versions, operating system, when relevant, hardware requirements, random seeds and model parameters

4. Make Data Reusable or Explain Why It Cannot Be Shared

A replication package should contain the data necessary to reproduce the analysis whenever sharing is legally and ethically permitted. However, data availability does not mean unrestricted data redistribution.

Researchers must consider privacy requirements, informed-consent conditions, institutional restrictions, commercial licenses, third-party ownership, and confidentiality agreements.

When original data cannot be distributed, authors should not simply omit the issue. Instead, explain why the data cannot be shared and provide an appropriate alternative where possible for example, a synthetic dataset, derived data, preprocessing instructions, or information about how qualified researchers can request access.

5. Connect Every Result to Its Reproduction Step

A strong replication package should make the relationship between the manuscript and the files obvious.

6. Test the Package Before Submission

Never assume that code will work simply because it worked on the author’s computer. Before submission, conduct a clean reproduction test.

Ideally, ask a colleague who was not directly involved in developing the code to:

1. Download the package.
2. Follow only the README.
3. Install the documented dependencies.
4. Run the workflow.
5. Generate selected tables or figures.
6. Compare the outputs with the manuscript.
7. Record any errors or missing instructions.

This independent test can reveal problems that the original researcher no longer notices.

7. Preserve a Stable Version

A replication package should correspond to the version of the research that was submitted or published. Do not continue changing the repository without recording versions.

A good workflow is:

Develop → Test → Freeze → Version → Archive → Cite

A persistent identifier such as a DOI can help researchers locate the exact version associated with the publication.

8. Address Licensing and Third-Party Materials

Replication packages can contain software, datasets, libraries, images, or other materials owned by third parties. Before release, check whether you have permission to redistribute each component.

Researchers should:

• identify third-party materials;
• record their licenses;
• retain required copyright notices;
• avoid redistributing restricted datasets;
• document dependencies; and
• provide links or acquisition instructions where redistribution is not permitted.

This is especially important for AI and software research, where projects may combine open-source libraries, pretrained models, external datasets, APIs, and proprietary tools.

9. Link the Package to the Manuscript

The replication package should not exist independently of the paper. Where appropriate, the manuscript should identify:

• where the code is available;
• where the data are available;
• the package version;
• the persistent identifier;
• access restrictions;
• relevant licenses; and
• any special instructions.  

What Reviewers Should Be Able to Determine

A well-prepared replication package does not require reviewers to become software engineers or inspect every line of code.

Instead, it should make it possible to determine:

Can the reported analysis be followed?
Are the necessary inputs available or appropriately documented?
Are the computational steps sufficiently clear?
Can the major results be regenerated?
Are important assumptions and dependencies documented?

Preparing a Replication Package for UTJ Submission

For authors preparing manuscripts for the Ubiquitous Technology Journal, reproducibility should be considered during research preparation rather than added immediately before submission. UTJ’s author guidance states that experimental methods should be presented clearly and completely so that other scientists can reproduce the work. The journal also asks authors to follow its structured manuscript requirements and provides specific guidance on figures, tables, references, and research presentation.

This is particularly relevant to UTJ’s broad technology scope, which includes areas such as artificial intelligence, IoT, robotics, telecommunications, cybersecurity, pervasive computing, edge and cloud computing, human-computer interaction, distributed systems, and related fields.

A carefully prepared replication package can therefore complement the manuscript by making the computational foundation of the research easier to inspect and reuse.

Final Pre-Submission Checklist

Before submitting, ask:

• Code: Is all essential research code included?
• Data: Are required datasets available or appropriately explained?
• README: Can another researcher understand the workflow?
• Dependencies: Are software and package versions documented?
• Parameters: Are important settings and random seeds recorded?
• Outputs: Can major tables and figures be regenerated?
• Organization: Are files clearly named and structured?
• Testing: Has someone else tested the package?
• Licensing: Are third-party materials legally shareable?
• Versioning: Is the submitted version clearly identified?
• Repository: Is the package stored in an appropriate, stable location?
• Manuscript: Are code and data availability statements accurate?