Four Handy Bash One-Liners

July 6, 2022

In my experience, most computational chemists only know a handful of basic Bash commands, which is a shame because Bash is incredibly powerful. Although I'm far from an expert, here are a few commands I frequently find myself using:

1. sed For Find-and-Replace.

$ sed -i “s/b3lyp/m062x/” *.gjf

If you want to resubmit a bunch of transition states at a different level of theory, don't use a complex package like cctk! You can easily find and replace text using sed, which runs almost instantly even for hundreds of files. (Note that the syntax for modifying in-place is slightly different on macOS.)

2. Renaming Lots of Files

$ for f in *.gjf; do mv $f ${f/.gjf/_resubmit.gjf}; done

Unfortunately, you can't rename lots of files with a single command in Bash, but using a for; do; done loop is almost as easy. Here, we simply use parameter expansion to replace the end of the filename, but the possibilities are endless.

3. Counting Occurrences in a File

$ for f in */*.out; do echo $f; grep "SCF Done" $f | wc -l; done

Here we again use a for loop, but in this case we use grep to search for the string "SCF Done". We then pipe the output of this search to the wc -l command, which counts the number of lines. Since grep returns each result on a new line, this prints the number of optimization steps completed.

4. Cancelling Jobs By Matching Name

$ squeue -u cwagen | grep "g16_ts_scan" | awk '{print $1}' | xargs -n 1 scancel

Although the slurm workload manager allows one to cancel jobs by partition or by user, to my knowledge there isn't a way to cancel jobs that match a certain name. This is a problem if, for instance, you're working on two projects at once and want to resubmit only one set of jobs. Here, we use squeue to get a list of job names, search for the names that match, extract the job number using awk, and finally cancel each job by building the scancel commands with xargs. (This should be easily modifiable for other workload managers.)



If you want email updates when I write new posts, you can subscribe on Substack.