dREG Gateway

Documents

1)  Login:
The user needs to log in by clicking 'login' link at the top-right corner of the page. Having an account provides a number of benefits, and is free and easy.

dREG login

2)  Create a new project (optional)
Optionally, users can choose to make a new 'project' in the dREG gateway to archive a collection of dREG data from related experiments. This will allow a collection of experiments to be stored in close proximity to each other.

dREG project

3)  Start new dREG
Select the menu 'Start dREG' below the dREG logo to create an data analysis for your data, as the following screenshot.

dREG experiment

4)  Select bigWig files
Select bigWig files representing PRO-seq, GRO-seq, or ChRO-seq signal on the plus and minus strand. Please notice that two GPU resources are available now, currently it is easier to get the computation resources on Comet.sdsc.xsede.org than Bridges.psc.edu.

dREG experiment create

5)  Submit the job
Click the 'save and launch' button. BigWig file are transferred to the XSEDE server and a GPU queue is scheduled to run dREG. After submitting, the user can check the status in the next web page, as shown below. Depend on the queue status, the job maybe wait for a long time to start prediction. Once started, it will only take 1-4 hours to complete.

6)  Check the status
The user can check the status of their 'experiment' by clicking the menu 'Saved dREG runs' below the dREG logo.

dREG experiment browse

7)  Check the results
Once a job is completed, the user can select 'Full results' in the drop-down list and then LEFT-click 'Download' link in the experiment summary page to download a compressed file described in the 'output' sheet in this page, or the user can download any single file from the drop-down list. The downloaded file with the 'tar.gz' extension can be decompressed by the 'tar' command, the file with the 'gz' extension can be decompressed by the 'gunzip' command in Linux. Please don't use RIGHT-click to open a tab for downloading.

In Safari, it could be problematic because Safari tries to unzip the compressed results automatically using a non-compatible compress method. Please check this link to disable this feature.

dREG experiment summary

8)  Switch to Genome Browser
The convenient tool ptovided by the gateway is the user can check the results in the Genome Browser by clicking 'Switch to genome browser' link. The genome identifier must be specified by two ways, 1) select from the drop-down list or 2) fill the inentifier in the textbox. Please use LEFT-click to open a genome browser window.

dREG experiment summary

9)  Check the storage
the user can LEFT-click 'Open Folder' link in the experiment summary page to check the storage for the current job or click the menu 'Storage' under the dREG logo to check the folders and files for all jobs(experiments). The following figure shows the data files in the job's folder, including two bigWig files, one result in bedgraph format, two outputs of job scheduler on GPU nodes.

dREG experiment summary

10)  When you meet failure
Currently when you run the dREG jobs, there are two types of errors you may have. One error may come from the system, called a system error, such as no computing time on specific GPU nodes or an internal errors in Apache Airavata. The other type of error is caused by the users' bigwig, called bigwig error, which can occur when read counts are normalized, each read is mapped to a region, or read counts in minus strand are positive values. The following figures show how to identify the error and how to handle it.

a)  System error
When users submit the experiment, the failure will be shown in the experiment summary page soon as figure 10-S1 or 10-S2. The experiment status is "Failed" and many java errors are shown in the "Errors" item. Users can't solve this problem and should report this error the web master.

System error(1)
Figure 10-S1

System error(2)
Figure 10-S2

b)  Bigwig error
After the experiment is complete, no results can be downloaded and job status shows a failure (see Figure 10-S3). Users can find the dREG log file or task log file to identify the problem. Enter into "storage directory" by clicking the "open" link. The users can find "ARCHIVE" folder where Apache Airavata copy back all files from the computing node. Check the dREG log file (out.dREG.log) to see the bigwig problem or check the task log file ("slurm-tasknoxxx.out") and find the reason why the task is aborted. Figure 10-S4 and 10-S5 give a two examples for this kind of error. If the bigwig has problems, please refer to this link to solve the problems.

Bigwig error
Figure 10-S3

This figure shows the bigWig problems in the dREG log file.

Bigwig error(1)
Figure 10-S4

This figure shows the task log file in which explains the task was killed due to time limit.

Bigwig error(2)
Figure 10-S5

The dREG gateway is web service built on the Apache Airavata software framework and the XSEDE platform using the following software packages:

[1] dREG package: https://github.com/Danko-Lab/dREG.

The dREG package is developed to detect the divergently oriented RNA polymerase in GRO-seq, PRO-seq, or ChRO-seq data using support vector machines (e1070 or Rgtsvm package).

[2] dREG.HD package: https://github.com/Danko-Lab/dREG.HD.

The dREG.HD package refines the location of TREs obtained using dREG by imputing DNAse-I hypersensitivity.

[3] Rgtsvm package: https://github.com/Danko-Lab/Rgtsvm.

Rgtsvm implements support vector classification and support vector regression on a GPU to accelerate the computational speed of training and predicting large-scale models.

[4] Airavata PHP Gateway: https://github.com/apache/airavata-php-gateway.git.

Airavata PHP Gateway provides an API to build web sites which interact with high performance computers that are part of XSEDE.

dREG run generates a compressed file including the dREG and dREG.HD results as follows:

 

File name Description
$PREFIX.dREG.infp.bed.gz Informative positions with the scores predicted by the dREG model.
Decompress it with 'gunzip' in Linux.
$PREFIX.dREG.peak.full.bed.gz dREG peaks calling results with score, probability and center position information.
Decompress it with 'gunzip' in Linux.
$PREFIX.dREG.peak.score.bed.gz dREG peaks calling results only with score.
Decompress it with 'gunzip' in Linux.
$PREFIX.tar.gz Including above 3 files, can be decompressed by 'tar -xvzf' in Linux.