Data structure on the UMCG cluster

The BIOS group at the UMCG cluster can be found at
/groups/umcg-bios

This is the general folder for all bios projects, which can only be accessed by members with permission to enter this group.

The file structure in the BIOS group works as follows. For a full overview of the directories with their subdirectories (current on 02-12-2015), please view the tree file further down below.

/prm02
PeRManent dirs: The group's directories for raw data and final results: this dir has a backup and only contains compressed files that are the result of processing or analysis and need to be kept safe.

	/rawdata
	Storage of raw data

	/projects
	Storage of essential results from projects

	/users
	Storage of essential files from users


/tmp04
TeMPorary dirs: The group's large, fastest directories for (shared) temporary data. Data from this folder will be automatically removed after three months if it has not been used. 
Ideally the essential data from this directory should be backed up to /prm02 every night.

	/rawdata
	Raw data that can be used for analyses

	/projects
	Directories for each project

		/<project_name>

			/<version>

				/jobs
				Directories with the pipeline output files and scripts for this project (in case the project uses a pipeline)
					
				/results
				Directories with results from this project

	/users
	Directories where each user can run their analyses that are not in the form of a pipeline yet

		/umcg-<user-ID>
		

When using this file system, please keep in mind that you will need to back up the essential files to prm02 on a regular basis. The tmp04 directory will clear all files that have not been used for three months as a way to keep it tidy.

Full Tree Structure of File System

This version is current on 02-12-2015

/groups/umcg-bios/
|-- prm02
|   |-- projects
|   |   |-- annotations
|   |   |-- eQTLmapping
|   |   |   |-- BIOS
|   |   |   |   |-- exon_level
|   |   |   |   `-- gene_level
|   |   |   |-- Geuvadis
|   |   |   |   |-- gene_level
|   |   |   |   |   |-- eQTLs_meta_20PCs
|   |   |   |   |   |   `-- old_wrong_gte
|   |   |   |   |   |       |-- plots
|   |   |   |   |   |       `-- plots_opposite_effects_replication_bbmri
|   |   |   |   |   |           `-- plots [error opening dir]
|   |   |   |   |   |-- eQTLs_meta_20PCs_replication_bbmri_genelevel
|   |   |   |   |   |   `-- old_wrong_gte
|   |   |   |   |   |-- eQTLs_meta_20PCs_secondary_effects
|   |   |   |   |   |   |-- distance_to_TSS
|   |   |   |   |   |   |-- expressionWithPrimaryEffectsRegressedOut
|   |   |   |   |   |   |-- geuvadis_eQTLs_meta_20PCs_1-0_RegressedOut_15012015
|   |   |   |   |   |   |-- geuvadis_eQTLs_meta_20PCs_1-1_RegressedOut_15012015
|   |   |   |   |   |   |-- geuvadis_eQTLs_meta_20PCs_1-2_RegressedOut_15012015
|   |   |   |   |   |   |-- geuvadis_eQTLs_meta_20PCs_1-3_RegressedOut_15012015
|   |   |   |   |   |   `-- geuvadis_eQTLs_meta_20PCs_1-4_RegressedOut_15012015
|   |   |   |   |   `-- replicationOfBiosAllLevels
|   |   |   |   `-- meta-exon_level
|   |   |   |       |-- eQTLs_meta_20PCs
|   |   |   |       |   `-- old_gte
|   |   |   |       `-- eQTLs_meta_20PCs_bbmri_replication
|   |   |   |           `-- old_gtes
|   |   |   |-- geuvadis_gene_replication_in_bbmri_oldGte
|   |   |   `-- GoShifter_annotation
|   |   |       |-- compute-pipeline
|   |   |       |   |-- protocols
|   |   |       |   `-- templates
|   |   |       `-- LL+RS+CODAM+LLS_eqtls_genes_23062014
|   |   |           |-- differenceGeuvadis
|   |   |           |   |-- BIOS-Geuvadis
|   |   |           |   |   |-- notReplicated
|   |   |           |   |   |   |-- goshifter_output [error opening dir]
|   |   |           |   |   |   `-- scripts
|   |   |           |   |   |-- replicatedOpposite
|   |   |           |   |   |   |-- goshifter_output [error opening dir]
|   |   |           |   |   |   `-- scripts
|   |   |           |   |   `-- replicatedSame
|   |   |           |   |       |-- goshifter_output [error opening dir]
|   |   |           |   |       `-- scripts
|   |   |           |   |-- notReplicated
|   |   |           |   |   |-- goshifter_output [error opening dir]
|   |   |           |   |   `-- scripts
|   |   |           |   |-- replicatedOpposite
|   |   |           |   |   |-- goshifter_output [error opening dir]
|   |   |           |   |   `-- scripts
|   |   |           |   `-- replicatedSame
|   |   |           |       |-- goshifter_output [error opening dir]
|   |   |           |       `-- scripts
|   |   |           |-- goshifter_output [error opening dir]
|   |   |           `-- LD
|   |   |-- expression
|   |   |   |-- BBMRI+Geuvadis_joint_normalization
|   |   |   |-- gene_level_additional
|   |   |   |-- Geuvadis
|   |   |   |   |-- EUR+YRI
|   |   |   |   |   `-- EUR+YRI_eQTLs_meta_25PCs
|   |   |   |   |-- gene_level
|   |   |   |   |-- interactionAnalysis
|   |   |   |   |-- meta-exon_level
|   |   |   |   `-- TriTyper
|   |   |   |       |-- CEU
|   |   |   |       |   `-- CEU.TMM.ProbesWithZeroVarianceRemoved.Log2Transformed.ProbesCentered.SamplesZTransformed
|   |   |   |       |-- FIN
|   |   |   |       |   `-- FIN.TMM.ProbesWithZeroVarianceRemoved.Log2Transformed.ProbesCentered.SamplesZTransformed
|   |   |   |       |-- GBR
|   |   |   |       |   `-- GBR.TMM.ProbesWithZeroVarianceRemoved.Log2Transformed.ProbesCentered.SamplesZTransformed
|   |   |   |       `-- TSI
|   |   |   |           `-- TSI.TMM.ProbesWithZeroVarianceRemoved.Log2Transformed.ProbesCentered.SamplesZTransformed
|   |   |   |-- meta-exon_level_additional
|   |   |   |-- per_cohort
|   |   |   |   `-- LL
|   |   |   `-- transcript_level_additional
|   |   |-- genotypes
|   |   |   |-- CODAM-imputed-20140306-TriTyper
|   |   |   |-- LL-imputed-20140306-TriTyper
|   |   |   |-- LLS-imputed-20140402-TriTyper
|   |   |   |-- NTR_AC-imputed-20140606-TriTyper
|   |   |   |-- NTR_Aff6-imputed-20140606-TriTyper
|   |   |   |-- PAN-imputed-20140306-TriTyper
|   |   |   |-- RNAseq_genotypes
|   |   |   |   |-- CODAM_SNVMix_TriTyper
|   |   |   |   |-- LL_SNVMix_TriTyper
|   |   |   |   `-- RS_SNVMix_TriTyper
|   |   |   `-- RS-imputed-20140306-TriTyper
|   |   |-- imputed
|   |   |   |-- CODAM
|   |   |   |   |-- genotypes
|   |   |   |   |   |-- b36
|   |   |   |   |   |-- b37
|   |   |   |   |   |   |-- old
|   |   |   |   |   |   `-- qc
|   |   |   |   |   `-- rawdata
|   |   |   |   `-- jobs
|   |   |   |-- NTR
|   |   |   |   |-- genotypes
|   |   |   |   |   |-- NTR_B37_AC_Epigen_CAUT
|   |   |   |   |   |   |-- b37
|   |   |   |   |   |   |   `-- backup
|   |   |   |   |   |   `-- rawdata
|   |   |   |   |   |-- NTR_B37_AC_Epigen_CX
|   |   |   |   |   |   `-- rawdata
|   |   |   |   |   `-- NTR_B37_Aff6_Epigen
|   |   |   |   |       |-- b37
|   |   |   |   |       |   `-- backup
|   |   |   |   |       `-- rawdata
|   |   |   |   `-- jobs
|   |   |   |-- PAN
|   |   |   |   |-- genotypes
|   |   |   |   |   |-- b37
|   |   |   |   |   |   |-- original
|   |   |   |   |   |   `-- qc
|   |   |   |   |   `-- rawdata
|   |   |   |   `-- jobs
|   |   |   `-- RS
|   |   |       |-- genotypes
|   |   |       |   |-- b36
|   |   |       |   |-- b37
|   |   |       |   `-- rawdata
|   |   |       `-- jobs
|   |   |-- interactionAnalysis
|   |   |   |-- BIOS
|   |   |   |   |-- biosInteractions8
|   |   |   |   |-- eQTLs_all_levels
|   |   |   |   `-- GWAS_eQTLs
|   |   |   |       |-- Body_mass_index
|   |   |   |       |-- Bone_mineral_density
|   |   |   |       |-- Celiac_disease
|   |   |   |       |-- Crohns_disease
|   |   |   |       |-- HDL_cholesterol
|   |   |   |       |-- Height
|   |   |   |       |-- Inflammatory_bowel_disease
|   |   |   |       |-- LDL_cholesterol
|   |   |   |       |-- Mean_platelet_volume
|   |   |   |       |-- Metabolic_traits
|   |   |   |       |-- Metabolite_levels
|   |   |   |       |-- Multiple_sclerosis
|   |   |   |       |-- Platelet_counts
|   |   |   |       |-- Prostate_cancer
|   |   |   |       |-- Psoriasis
|   |   |   |       |-- Red_blood_cell_traits
|   |   |   |       |-- Rheumatoid_arthritis
|   |   |   |       |-- Schizophrenia
|   |   |   |       |-- Total_cholesterol
|   |   |   |       |-- Triglycerides
|   |   |   |       |-- Type_1_diabetes
|   |   |   |       `-- Ulcerative_colitis
|   |   |   `-- Geuvadis
|   |   `-- oldImputations
|   |       |-- CODAM
|   |       |   `-- imputedGenotypeData
|   |       |-- LLS
|   |       |   `-- chunks
|   |       `-- Rotterdam
|   |           |-- Aligend
|   |           |-- imputation
|   |           |-- imputationTriTyper
|   |           |-- imputed
|   |           |-- mergedRs
|   |           |-- mergedRs1Rs2
|   |           |-- mergedRs1Rs2AligedToRs3
|   |           |-- phasing
|   |           |   `-- generatedScripts
|   |           |-- rs1
|   |           |-- rs2
|   |           `-- rs3
|   `-- rawdata
|       `-- rnaseq
|-- scr01 -> /local/groups/umcg-bios/scr01
|-- scr02 -> /local2/groups/umcg-bios/scr02
`-- tmp04
    |-- projects
    |   |-- bbmriSampleInfo
    |   |   `-- v2.1.1
    |   |-- biosDatabase
    |   |-- BIOS_RNA
    |   |   |-- batch0
    |   |   |   |-- jobs
    |   |   |   |   |-- genotypeCalling
    |   |   |   |   |-- QC
    |   |   |   |   `-- quantification
    |   |   |   `-- results
    |   |   |       |-- addOrReplaceReadGroups
    |   |   |       |-- baseQualityScoreRecalibration
    |   |   |       |-- collectMultipleMetrics_genotypeCalling
    |   |   |       |-- collectMultipleMetrics_QC
    |   |   |       |-- collectRnaSeqMetrics_genotypeCalling
    |   |   |       |-- collectRnaSeqMetrics_QC
    |   |   |       |-- covariateAnalysis
    |   |   |       |-- fastqc
    |   |   |       |-- filteredBam
    |   |   |       |-- flagStat
    |   |   |       |-- genotypeHarmonizer
    |   |   |       |-- haplotypeCaller
    |   |   |       |-- hisat
    |   |   |       |-- indelRealignment
    |   |   |       |-- kallisto
    |   |   |       |-- markDuplicates
    |   |   |       |-- mergeBams
    |   |   |       |-- sortedBam
    |   |   |       |-- splitAndTrim
    |   |   |       |-- unfilteredBam
    |   |   |       |-- unifiedGenotype
    |   |   |       |-- variantEval
    |   |   |       `-- verifyBamID
    |   |   `-- batch1
    |   |       |-- jobs
    |   |       |   |-- QC
    |   |       |   `-- quantification
    |   |       `-- results
    |   |           |-- collectMultipleMetrics_QC
    |   |           |-- collectRnaSeqMetrics_QC
    |   |           |-- fastqc
    |   |           |-- filteredBam
    |   |           |-- genotypeHarmonizer
    |   |           |-- hisat
    |   |           |-- kallisto
    |   |           |-- sortedBam
    |   |           |-- unfilteredBam
    |   |           |-- unifiedGenotype
    |   |           |-- variantEval
    |   |           `-- verifyBamID
    |   `-- genotypes
    |       |-- CODAM-imputed-20140306-TriTyper
    |       |-- LL-imputed-20140306-TriTyper
    |       |-- LLS-imputed-20140402-TriTyper
    |       |-- NTR_AC-imputed-20140606-TriTyper
    |       |-- NTR_Aff6-imputed-20140606-TriTyper
    |       |-- PAN-imputed-20140306-TriTyper
    |       |-- RNAseq_genotypes
    |       |   |-- CODAM_SNVMix_TriTyper
    |       |   |-- LL_SNVMix_TriTyper
    |       |   `-- RS_SNVMix_TriTyper
    |       `-- RS-imputed-20140306-TriTyper
    |-- rawdata
    |   `-- rnaseq
    `-- users
        |-- umcg-aclaringbould
        |   |-- abun1
        |   |-- abun2
        |   |-- abun3
        |   |-- bbmri_duplicates
        |   |-- eQTLpipeline
        |   |   |-- backup
        |   |   |-- cis
        |   |   |-- first_norm
        |   |   `-- mixupmapping
        |   |       |-- Cis-eQTLs
        |   |       |-- CODAM
        |   |       |   |-- Cis-eQTLs
        |   |       |   `-- MixupMapper
        |   |       |-- LL
        |   |       |   |-- Cis-eQTLs
        |   |       |   `-- MixupMapper
        |   |       |-- LLS
        |   |       |   |-- Cis-eQTLs
        |   |       |   `-- MixupMapper
        |   |       |-- NTR_AC
        |   |       |   |-- Cis-eQTLs
        |   |       |   `-- MixupMapper
        |   |       |-- NTR_Aff6
        |   |       |-- PAN
        |   |       |   |-- Cis-eQTLs
        |   |       |   `-- MixupMapper
        |   |       `-- RS
        |   |           |-- Cis-eQTLs
        |   |           `-- MixupMapper
        |   `-- scripts
        `-- umcg-ndeklein
            `-- samplesheets

277 directories