#SBATCH --nodelist=node[01-09]
#SBATCH --nodelist=node01
Category / Unix
Unix generate a random string
head /dev/urandom | tr -dc A-Za-z0-9 | head -c10
# Assigning it to a variable
ranNum=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c10)
Source:
https://unix.stackexchange.com/questions/230673/how-to-generate-a-random-string
Unix rearrange columns
awk 'BEGIN {FS="\t"; OFS="\t"} {print $2, $3, $4, $1}' input.txt > output.txt
FS and OFS specify the input/output separator
Liftover bam files
The most straightforward way is using CrossMap
.
Taking from hg19 to hg38 as example:
pip install CrossMap
CrossMap.py bam -a hg19ToHg38.over.chain input.bam output
#.bam extension will be added automatically
genome liftover chain files can be downloaded here: http://hgdownload.cse.ucsc.edu/goldenpath/hg19/liftOver/ (change according to your needs)
It is suggested to always use ‘-a’ option according to the CrossMap website.
Source:
http://crossmap.sourceforge.net/#convert-bam-cram-sam-format-files
Unix check if a file exists
test -f yourFile
FILE=/folder/yourFile
if [ -f "$FILE" ]; then
echo "$FILE exists."
else
echo "$FILE does not exist."
fi
If need to match wildcard in the file names
if test -n "$(find /dir/to/search -maxdepth 1 -name 'files*' -print -quit)"
then
echo found
else
echo not found
fi
Source:
https://linuxize.com/post/bash-check-if-file-exists/
https://stackoverflow.com/questions/6363441/check-if-a-file-exists-with-wildcard-in-shell-script
Unix convert a string to lower case
echo "ABCD-EFGH" | tr '[:upper:]' '[:lower:]'
Source:
https://stackoverflow.com/questions/2264428/how-to-convert-a-string-to-lower-case-in-bash
Ignore row with only NaN in plotHeatmap – deepTools
If there are NaN in the output from computeMatrix, the generated heatmap is not sorted and a warning message stating Mean of empty slice
will show up.
To overcome this, those null values need to be replaced using 0 in the computeMatrix
step by --missingDataAsZero
tag.
computeMatrix scale-regions -S xxx.bw -R xxx.bed --missingDataAsZero -m xxx -b xxx -a xxx --numberOfProcessors xx -o xxx.gz
plotHeatmap -m xxx.gz -out xxx.png
Source:
https://groups.google.com/g/deeptools/c/B55e0kT_0Ec?pli=1
https://github.com/deeptools/deepTools/issues/490
Slurm display full job name
sacct -u username --format=JobID,JobName%30
sacct -u username --format=JobID,TIME,JobName%30,Start%12,Elapsed,NCPU,CPUTime
Source:
https://stackoverflow.com/questions/42217102/expand-columns-to-see-full-jobname-in-slurm
Python check if a file or directory exists
import os.path
# Check if a file exist, return True/False
os.path.exists("test.txt")
# Check if a directory exist, return True/False, for >= python3.4
import pathlib
pathlib.Path("test").exists()
Source:
https://www.guru99.com/python-check-if-file-exists.html
Python return os.system and subprocess output as a string
When we want to use Unix command in python we can directly use os.system()
to realize it. However, if we only want to return the output as a string, for example, return ls
file names into a string, we need to use subprocess.check_out
instead.
To note that, the output from subprocess.check_out()
is a bytes object instead of a string, thus we need to further decode
to transform into a string.
fileList = subprocess.check_output('ls someFolder/*', shell=True).decode('utf-8').strip().split('\n')
Source:
https://stackoverflow.com/questions/606191/convert-bytes-to-a-string