Below commands all work.
df -h /yourDisk
df -h
df -kSource:
https://docs.oracle.com/cd/E23823_01/html/817-0403/spmonitor-6.html
Category / Unix
Jupyter Notebook check and stop all running servers/kernels
Check running servers:
jupyter notebook listStop running servers (both works):
jupyter notebook stop
kill $(pgrep jupyter)Source:
https://github.com/jupyter/notebook/issues/1950
https://stackoverflow.com/questions/10162707/how-to-close-ipython-notebook-properly/32745046#32745046
R set memory limit on Mac for Rstudio
Step 1: Open terminal
Step 2:
cd ~
touch .Renviron
open .RenvironStep 3: Save the following as the first line of .Renviron:
R_MAX_VSIZE=100Gb As noted in the source,
Sys.setenv('R_MAX_VSIZE'=32000000000)only works in command line, doesn’t work for Rstudio
Source:
https://stackoverflow.com/questions/51295402/r-on-macos-error-vector-memory-exhausted-limit-reached
Slurm sbatch exclude nodes or node list
Both works.
#SBATCH --exclude=node[01-09]
#SBATCH --exclude=node01,node02,node03,node04,node05,node07,node08,node09,node10Unix tar compress and decompress tar.gz files or folder
Compress multiple files:
tar -czvf name-of-archive.tar.gz file1 file2 folder1 folder2 Compress folder:
tar -czvf name-of-archive.tar.gz /path/to/directory-or-fileCompress a folder while excluding some files:
tar -czvf archive.tar.gz /home/ubuntu --exclude=*.mp4Decompress tar.gz file:
tar -xzvf archive.tar.gz
tar -xzvf archive.tar.gz -C targetDir
tar -xzvf archive.tar.gz -directory targetDirSouce:
https://www.howtogeek.com/248780/how-to-compress-and-extract-files-using-the-tar-command-on-linux/
Unix zip compress and decompress files or folder
Compress files:
zip output.zip file1 file2 file3Compress folder:
zip -r output.zip folder1 folder2Decompress zip file:
unzip output.zip # unzip into the directory where the zip file exists
unzip output.zip -d folder1 # unzip into folder from where the zip file existsSource:
https://www.cyberciti.biz/faq/how-to-create-a-zip-file-in-unix/
Unix calculate the size of a folder/directory
du -sh directory_nameSource:
https://unix.stackexchange.com/questions/3019/how-can-i-calculate-the-size-of-a-directory/3021
Python Jupyter Notebook: install packages under the current working environment
In Jupyter Notebook, use the below command within the notebook to install packages in the current working environment. This is specifically useful when you are working in the notebook, and need to install new packages to the matching python version you are using.
!{sys.executable} -m pip install XXX
Interrupt python process without exit python in cmd
In Mac: control+c
Liftover bigwig files
There are two ways to lift-over bigwig files from one genome build to another. One is using CrossMap or step by step as below. (CrossMap is almost the same to the break down steps)
CrossMap method: (taking from hg19 to hg38 as example)
pip install CrossMap
CrossMap.py bigwig hg19ToHg38.over.chain input.bw output.bwgenome liftover chain files can be downloaded here: http://hgdownload.cse.ucsc.edu/goldenpath/hg19/liftOver/ (change according to your needs)
Step by step method: (bw –> bedGraph –> liftover –> bw)
bigWigToBedGraph input.bw input.bedGraph
liftOver input.bedGraph hg19ToHg38.over.chain input_hg38.bedgraph unMapped
fetchChromSizes hg38 > hg38.chrom.sizes
LC_COLLATE=C sort -k1,1 -k2,2n input_hg38.bedgraph > input_hg38.sorted.bedgraph
bedGraphToBigWig input_hg38.sorted.bedgraph hg38.chrom.sizes output.bw
bigWigToBedGraph,liftOver,fetchChromSizes,bedGraphToBigWigare all UCSC utilities which can be installed from here: http://hgdownload.soe.ucsc.edu/downloads.html#utilities_downloads
Source:
http://hgdownload.soe.ucsc.edu/downloads.html#utilities_downloads
http://crossmap.sourceforge.net/#