Uploading raw reads

Contents:

• Raw reads specific metadata
• Compressing your files
• Dehosting (host read removal)
• Preparing your metadata: examples

Pathoplexus allows users to include raw reads when submitting sequences (see here for our general submission documentation). To include your raw reads files, you will need to upload them and link them to their respective sequences via the files.rawReads field in your metadata file. The files.rawReads field should be a space-separated list containing the names of all files associated with the sequence.

When on the submission page, you can either click on ‘Upload folder’ to upload an entire folder containing raw reads files, or ‘Upload files’ to select individual files to upload. Once you’ve selected files to upload, the webpage will let you know if there are files listed in the files.rawReads metadata column that are missing or if you’ve uploaded files that are not listed in files.rawReads. Once all the files included in files.rawReads have been uploaded and linked and no unexpected files are present, you can proceed with submission.

If the raw reads for a sequence are already available in the INSDC, do not upload them again. Instead, supply their run, BioSample, and BioProject accessions in the INSDC-linking metadata fields insdcRawReadsAccession, biosampleAccession, and bioprojectAccession, and we will link to them rather than submitting a second copy.

Raw reads specific metadata

When including raw reads files in your submission you will also need to provide additional metadata fields in your metadata file. Please refer to the table below to see which fields are required and desired when submitting raw reads and include them in your metadata file. For further details on the metadata required in your submission see our metadata docs.

Compressing your files

Raw reads files must be compressed with gzip before you upload them (extension .gz). If your files are not yet compressed, you can use the instructions below.

macOS and Linux

Open a terminal, navigate to the folder containing your files, and run:

gzip raw_reads_1.fastq

This replaces raw_reads_1.fastq with raw_reads_1.fastq.gz. To compress all fastq files in the current folder at once:

gzip *.fastq

Windows

Windows does not include gzip by default, but you can use 7-Zip to compress your files (free and open source):

1. Install 7-Zip.
2. In File Explorer, right-click your fastq file.
3. Choose 7-Zip → Add to archive…
4. Set Archive format to gzip and click OK.

This creates a .gz file next to your original file. When you select several files, 7-Zip creates one .gz file per input file, which is what we need. Please do not put multiple raw reads files into a single archive.

Alternatively, if you use the Windows Subsystem for Linux (WSL), you can follow the macOS and Linux instructions above.

Dehosting (host read removal)

Overview

Users must only submit files free of any human genetic data. This is particularly relevant to raw-read sequencing data, and any files uploaded should be screened for human reads and have those reads removed. This is commonly called dehosting or host read removal. It is the responsibility of data providers to ensure that submitted data does not contain human reads that could compromise participant privacy or ethical standards. Pathoplexus performs additional screening and will reject files where it detects an excessive number of reads likely to be human. This screening should not be relied upon as a substitute for dehosting prior to submission. Even for samples without a human host, it is advised to remove reads from the host species from samples, to ensure downstream analyses are performed only on relevant viral sequences.

Available Tools

Several established tools are available for host read removal. Different projects may choose different implementations depending on their workflows, performance requirements, and supported reference databases. Examples include:

• Kraken2 – a taxonomic classification tool that can be used for host filtering using an appropriate reference database. https://github.com/DerrickWood/kraken2
• Hostile – a dedicated host depletion pipeline designed for removing host-derived reads from sequencing datasets. https://github.com/bede/hostile
• Deacon – a high-performance host depletion tool supporting custom depletion indices. https://github.com/bede/deacon

These examples are provided for reference only; any approach that effectively removes host reads while preserving non-host sequences is acceptable.

Validation

Regardless of the tool used, users should validate that:

• Host-derived reads, particularly human reads, have been effectively removed.
• Non-host reads are retained at an acceptable rate.
• The filtering strategy is appropriate for the host species and sequencing protocol used.
• The resulting dataset complies with applicable privacy, ethical, and institutional requirements before submission.

Preparing your metadata: examples

This section shows some examples of how to include raw reads in your submission to Pathoplexus. Each example shows the submission of three sequences — seq1, seq2, and seq3 — with associated raw reads. The raw reads for seq1 were generated using paired-end sequencing and are uploaded as a forward and reverse fastq file. Raw reads for seq2 and seq3 are single-ended.

Example 1: uploading individual files

To upload the raw reads files, you can click ‘Upload files’ and select the raw reads files from your local filesystem: raw_reads1_fw.fq.gz, raw_reads1_rv.fq.gz, raw_reads_2.fq.gz, and raw_reads_3.fq.gz

Metadata file

You must then configure your files.rawReads metadata column to link each of these files to its respective sequence:

End result

After sequence processing and approval, the files will be shown as follows when viewing the sequences on Pathoplexus:

• seq1: raw_reads1_fw.fq.gz raw_reads1_rv.fq.gz
• seq2: raw_reads_2.fq.gz
• seq3: raw_reads_3.fq.gz

Example 2: uploading a single folder

Instead of uploading files individually, you can click ‘Upload folder’ and upload a folder that contains all four raw reads files at once.

Folder structure on your machine

uploaded_folder
├── raw_reads_1_fw.fq.gz
├── raw_reads_1_rv.fq.gz
├── raw_reads_2.fq.gz
└── raw_reads_3.fq.gz

Metadata file

You must then configure your files.rawReads metadata column to link each of these files to its respective sequence:

End result

After sequence processing and approval, the files will be shown as follows when viewing the sequences on Pathoplexus:

• seq1: raw_reads_1_fw.fq.gz raw_reads_1_rv.fq.gz
• seq2: raw_reads_2.fq.gz
• seq3: raw_reads_3.fq.gz

uploaded_folder
├── raw_reads_1_fw.fq.gz
├── raw_reads_1_rv.fq.gz
├── raw_reads_2.fq.gz
└── raw_reads_3.fq.gz

uploaded_folder
├── subfolder_1
│   └── raw_reads_fw.fq.gz
│   └── raw_reads_rv.fq.gz
│
├── subfolder_2
│   └── raw_reads.fq.gz
│
└── subfolder_3
    └── raw_reads.fq.gz