Bio-Py Tutor
Free reference — no account needed

Unix commands for biologists

Before Python, most bioinformatics happens in a terminal. Every command below is written the way you will actually use it: what it does, the option flags worth remembering, and an example with sequencing or lab data. The general shape is always command -flags arguments.

Six habits that save beginners hours

  • Press Tab to auto-complete file and command names — it prevents most typos.
  • Press the Up arrow to bring back your previous command instead of retyping it.
  • Ctrl+C stops a command that is running; Ctrl+D ends input or closes the shell.
  • Unix is case-sensitive: Data.csv and data.csv are two different files.
  • Avoid spaces in filenames — use underscores, e.g. wt_replicate_1.fastq.
  • When in doubt, run man <command> or <command> --help before guessing.

Creating, copying and deleting

These commands do not ask for confirmation by default. Deletion is permanent — there is no trash bin.

mkdir

mkdir [options] name

Make a new directory.

-p
Create parent directories as needed, no error if it exists.
-v
Print a line for each directory created.
$ mkdir -p project/data/raw

Creates the whole nested path at once.

touch

touch file

Create an empty file, or update a file's timestamp.

-c
Do not create the file if it is missing.
$ touch notes.md results.csv

Creates two empty files.

cp

cp [options] source destination

Copy files or directories.

-r
Recursive — required to copy a directory.
-i
Interactive — ask before overwriting.
-v
Verbose — show what is being copied.
-p
Preserve timestamps and permissions.
$ cp reads.fastq backup/

Copy one file into a folder.

$ cp -rv raw_data/ /mnt/archive/

Copy a whole folder, printing progress.

mv

mv [options] source destination

Move or rename files and directories.

-i
Ask before overwriting an existing file.
-n
Never overwrite.
$ mv draft.txt final.txt

Rename a file.

$ mv *.png figures/

Move all PNGs into figures/.

rm

rm [options] file

Remove files. Deletion is immediate and permanent.

-i
Confirm each deletion — recommended while learning.
-r
Recursive — needed to delete a directory.
-f
Force, no prompts. Dangerous when combined with -r.
$ rm -i temp.txt

Safe delete with a confirmation prompt.

$ rm -r old_run/

Delete a folder and everything inside it.

rmdir

rmdir directory

Remove a directory only if it is empty — a safer alternative to rm -r.

-p
Also remove empty parent directories.
$ rmdir empty_folder

Fails harmlessly if the folder has contents.

ln

ln [-s] target linkname

Create a link (shortcut) to a file.

-s
Symbolic link — the usual choice; works across filesystems.
$ ln -s /data/genome.fa genome.fa

Point to a big shared file without copying it.

Reading file contents

Sequencing and expression files are often huge. Peek at them before opening them in an editor.

cat

cat [options] file...

Print a whole file, or join several files together.

-n
Number every output line.
-A
Show invisible characters like tabs and line endings.
$ cat sample.fasta

Print a small file.

$ cat part1.csv part2.csv > all.csv

Concatenate two files into one.

less

less [options] file

Scroll through a large file page by page. Press q to quit, / to search.

-N
Show line numbers.
-S
Do not wrap long lines — good for wide tables.
$ less -S counts.tsv

Browse a wide count matrix without wrapping.

head

head [-n N] file

Show the first lines of a file (10 by default).

-n N
Show the first N lines.
-c N
Show the first N bytes.
$ head -n 4 reads.fastq

Show one FASTQ record.

tail

tail [-n N] file

Show the last lines of a file.

-n N
Show the last N lines.
-f
Follow — keep printing new lines as they arrive.
$ tail -f pipeline.log

Watch a running analysis log live.

wc

wc [options] file

Count lines, words and characters.

-l
Count lines only.
-w
Count words.
-c
Count bytes.
$ wc -l samples.csv

How many rows does my table have?

diff

diff [options] fileA fileB

Show the differences between two files.

-u
Unified diff — the compact format used by Git.
-i
Ignore case differences.
-r
Compare directories recursively.
$ diff -u run1.csv run2.csv

Check whether two runs produced identical output.

Pipes and redirection

Every command reads input and writes output. Connecting them is what turns small tools into an analysis pipeline.

|

command1 | command2

Pipe — send the output of one command into the next.

$ grep '^>' seqs.fa | wc -l

Count FASTA headers by piping grep into wc.

> and >>

command > file

Redirect output to a file. > overwrites, >> appends.

2>
Redirect error messages instead of normal output.
&>
Redirect both output and errors.
$ python analysis.py > results.txt 2> errors.txt

Keep results and errors in separate files.

xargs

command | xargs command2

Turn a list of items into arguments for another command.

-n1
One argument per invocation.
-I{}
Placeholder for each item.
$ find . -name "*.sam" | xargs -I{} gzip {}

Compress every SAM file found.

tee

command | tee file

Write to a file and to the screen at the same time.

-a
Append instead of overwriting.
$ ./run.sh | tee run.log

Watch the run while saving the log.

Permissions and ownership

Each file has read (r), write (w) and execute (x) permissions for the owner, the group and everyone else.

chmod

chmod mode file

Change file permissions.

+x
Make a script executable.
755
Owner full access; others read and execute.
644
Owner can edit; others read only.
-R
Apply recursively to a directory.
$ chmod +x align.sh

Allow a shell script to be run with ./align.sh.

chown

chown user:group file

Change the owner or group of a file (usually needs sudo).

-R
Apply recursively.
$ sudo chown -R student:lab shared_data/

Hand a folder over to the lab group.

sudo

sudo command

Run a single command with administrator privileges.

-u user
Run as a specific user rather than root.
$ sudo apt install samtools

Install software system-wide.

Processes, disks and system info

Useful when a job hangs or the cluster complains that you are out of space.

ps

ps [options]

List running processes.

aux
Show all processes from all users with details.
-ef
Full-format listing, another common combination.
$ ps aux | grep python

Find your running Python jobs and their PIDs.

top / htop

top

Live view of CPU and memory usage. Press q to quit.

-u user
Show only one user's processes.
$ top -u $USER

Check how much memory your alignment is using.

kill

kill [signal] PID

Stop a process by its process ID.

-9
Force kill when a normal kill does not work.
killall name
Kill every process with that name.
$ kill -9 48213

Terminate a runaway job.

df

df [options]

Show free disk space per filesystem.

-h
Human-readable sizes.
$ df -h

Check whether the data partition is full.

du

du [options] path

Show how much space files and folders use.

-h
Human-readable sizes.
-s
Summary total only.
-d1
One level deep.
$ du -sh */

Size of each subfolder — find the space hog.

man

man command

Read the manual page for any command. The first place to look.

/text
Search inside the page; press n for the next hit, q to quit.
command --help
Quick option summary when man is unavailable.
$ man grep

Full documentation for grep, including every flag.

history

history

List the commands you have run recently.

!123
Re-run command number 123.
Ctrl+R
Search backwards through history interactively.
$ history | grep blast

Recover that BLAST command you ran last week.

Archives, downloads and remote machines

Most sequencing data arrives compressed and lives on a remote server.

tar

tar [options] archive.tar.gz files

Bundle many files into one archive, and unpack them again.

-c
Create an archive.
-x
Extract an archive.
-z
Compress or decompress with gzip.
-v
Verbose file listing.
-f
Specify the archive filename (always needed).
$ tar -czvf run1.tar.gz run1/

Compress a results folder.

$ tar -xzvf run1.tar.gz

Unpack it again.

gzip / gunzip

gzip file

Compress or decompress a single file.

-k
Keep the original file.
-d
Decompress (same as gunzip).
$ gzip -k reads.fastq

Compress while keeping the original.

wget / curl

wget URL | curl -O URL

Download files from the internet on the command line.

-O file
Save under a chosen filename.
-c
Resume an interrupted download (wget).
-L
Follow redirects (curl).
$ wget -c https://ftp.ensembl.org/genome.fa.gz

Fetch a reference genome, resumable.

ssh

ssh user@host

Log in to a remote machine such as your institute's cluster.

-p N
Connect on a non-standard port.
-i key
Use a specific private key file.
$ ssh student@cluster.uni.edu

Open a shell on the compute cluster.

scp / rsync

scp source user@host:dest

Copy files between your computer and a remote server.

-r
Copy directories recursively.
rsync -avz
Archive mode, verbose, compressed — resumable and skips unchanged files.
--progress
Show transfer progress (rsync).
$ scp results.csv student@cluster:~/analysis/

Push one file to the cluster.

$ rsync -avz --progress cluster:~/run1/ ./run1/

Pull a large folder down safely.

echo

echo text

Print text or the value of a variable.

-n
No trailing newline.
-e
Interpret escapes like \n and \t.
$ echo $HOME

Show the path to your home directory.

Ready to combine this with Python?

The terminal moves and filters your data; Python analyses it. Try the two-minute demo — no signup required.

Try the demo