Snakemake has great support for most HPC systems, via custom submission command you can specify in profile and it works amazingly most of the time. However, occasionally it would fail to correctly recognize job status in some edge cases.
Read more-
posts
-
Handle Edge Cases on HPC with Snakemake
-
How to Manage Workflow with Resource Constraint on HPC
Note: This is an update to my previous post: How to Run Snakemake pipeline on HPC.
In my previous post, I disucessed some tips on how to effectively manage workflow using Snakemake on an HPC system. However, I have recently noticed that Snakemake support for
Read more--cluster-config
is offcially deprecated in favor of--profile
. I spent most of today digging into this feature and now I’m happy to share with you my latest setup. -
/dev/stderr vs >&2: a Deep Dive Into Linux STDIO
I was assisting a bioinformatics workshop for FAANG group at Plant and Animal Genome (PAG) and I ran into an interesting question with an example provided by one of the instructors on the topic of Linux output redirecting. Take a look at this simplified example script
Read moreprint_stderr.sh
: -
Running Jupyter Lab on HPC
For years I have been sticking with R and RStudio, primarily because RStudio’s useful and user-friendly GUI design (and my own comfort with Tidyverse). Indeed, my biggest complain against Jupyter Notebook whenever somebody introduced it to me was its bare-bone functionality. Until I discovered Jupyter Lab!
Read more -
How to run snakemake pipeline on HPC
Snakemake is a handy workflow manager written in Python. It handles workflow based on predefined job dependencies. One of the great features of Snakemake is that it can manage a workflow on both a standalone computer, or a clustered HPC system. HPC, or “cluster” as it’s often referred to, requires additional considerations.
Read more -
Write scientific paper in markdown
If you have ever written a scientific paper in MS Word (or any other word processing software) then you’re probably experienced frustrations like me: Word constantly crashing when loaded with too many high-resolution figures, citations getting messed up when working with others that use different citation managers, or hard to keep track of hundreds of versions of the draft, etc.
Read more -
How To Set NASA Astronomy Picture Of the Day As Wallpaper Using Python
NASA uploads a selected astronomy picture daily as their Astronomy Picture of the Day (APOD). For a space sci-fi junkie myself, I find these pictures extremely pretty and I want to use them as my wallpapers! But obviously downloading them from NASA everyday is a huge hassle. Luckily, NASA provides open APIs that we can make use of to automate this process.
Read more