Linux Bash/Terminal – Advanced – Pattern Searching

Bash Pattern Searching Reference (PAL Series)

Update: These were written for my students while my role was a PAL (Peer Assisted Learning) Leader for my course, I intended to keep all the basic/intermediate commands necessary for terminal use in one place to ease the students into using Linux efficiently. I’ve kept them up just in case they’re of use to someone.

Wildcards

* – Match anything
? – Match a single character

$ ls
render_10.exr render_13.exr scene_10.tif scene_9.exr
render_11.exr render_14.exr scene_11.exr scene_9.tif
render_12.exr scene_10.exr scene_11.tif
$ ls scene*       # Match any file beginning with "scene"
scene_10.exr scene_11.exr scene_9.exr
scene_10.tif scene_11.tif scene_9.tif
$ ls *.exr        # Match any file ending with ".exr"
render_10.exr render_12.exr render_14.exr scene_11.exr
render_11.exr render_13.exr scene_10.exr scene_9.exr
$ ls scene_?.tif  # Match scene_1.tif, scene_2.tif, scene_3.tif etc but not scene_10.tif
scene_9.tif
$ ls scene_??.tif # Match scene_10.tif etc but not scene_1.tif etc
scene_10.tif scene_11.tif

Grep

Allows for much more powerful pattern matching than wildcards alone, but you need to learn regular expressions to really utilise it’s power.

A simple usage however is to search the output of a command

$ history | grep "touch *"
 2006 touch output.exr 
 2011 touch {a,b}{x,y,z}
 2013 touch {a,b}/{x,y,z}
 2021 touch {a,b}/{x,y,z}
 2022 touch {a,b}{x,y,z}

You can also use it directly on a file to search just that file

$ grep TARGET tracer.pro      # Search for string "TARGET" in tracer.pro 
TARGET=tracer
 QMAKE_EXTRA_TARGETS += first copydata
 PRE_TARGETDEPS+=C:/NGL/lib/NGL.lib

Linux Bash/Terminal – Advanced – Expansion (Time saving tricks)

Time Saving Tricks Reference (PAL Series)

Update: These were written for my students while my role was a PAL (Peer Assisted Learning) Leader for my course, I intended to keep all the basic/intermediate commands necessary for terminal use in one place to ease the students into using Linux efficiently. I’ve kept them up just in case they’re of use to someone.

Brace Expansion

Expanding two sets of braces

$ touch {a,b}{x,y,z}
$ ls
ax ay az bx by bz

Nesting expanding braces with existing directories

$ mkdir include src
$ touch include/{vector,matrix,complex}.h src/{vector,matrix,complex,main}.c
$ ls
include src
$ ls include/
complex.h matrix.h vector.h
$ ls src/
complex.c matrix.c main.c vector.c

As you can see, brace expansions work like expanding the brackets in algebra.
They’re very handy for quickly generating directory structures with little typing. Manual page.

Sequence Expansion

Create 16 files

$ touch file_{0..16}.txt # Sequence from 0 to 16
$ ls
file_0.txt file_12.txt file_15.txt file_2.txt file_5.txt file_8.txt
file_10.txt file_13.txt file_16.txt file_3.txt file_6.txt file_9.txt
file_11.txt file_14.txt file_1.txt file_4.txt file_7.txt

Create directories with odd numbers only in their names

$ mkdir dir_{1..9..2}     # Sequence from 1 to 9, with a step of 2 (skip every other number)
$ ls
dir_1 dir_3 dir_5 dir_7 dir_9

Linux Bash/Terminal – Advanced – Redirect command output

Redirection Reference (PAL Series)

Update: These were written for my students while my role was a PAL (Peer Assisted Learning) Leader for my course, I intended to keep all the basic/intermediate commands necessary for terminal use in one place to ease the students into using Linux efficiently. I’ve kept them up just in case they’re of use to someone.

Linux makes use of 3 standard streams

  • Standard input – stdin
  • Standard output – stdout
  • Standard error – stderr

By default these stream operate with the terminal; writing to stdout or stderr will display text in the terminal for example, and stdin will read from the terminal by default too.

However, it is often very useful to redirect these streams. For example, what if we wanted to run a program and store all it’s output in a text file?

$ ls > listOutput.txt     # Redirect stdout to listOutput.txt
$ cat listOutput.txt      # Print the contents
include
Makefile
README.md
shaders
src
tracer.pro

Using > to redirect will completely overwrite whatever data was in listOutput.txt.

If we wanted to append to the file instead, we can use the >> operator.

$ echo "I'm at the end" >> listOutput.txt     # Append the result of the echo command
$ cat listOutput.txt                          # Print the contents
include
Makefile
README.md
shaders
src
tracer.pro
I'm at the end

What if we wanted to ignore errors from a program? (assuming errors are being printed to stderr, as they should. Some programs don’t always adhere though)

$ goMaya 2> /dev/null   # Run Maya and redirect stderr to the special device /dev/null, which silently ignores all input

By redirecting to /dev/null, we’re no longer printing stderr to the terminal and it is basically a way of silencing a stream.

This also applies to the other streams, a common technique to completely silence a program and run it in the background is

$ dropbox >/dev/null 2>/dev/null &;
# or
$ dropbox >/dev/null 2>&1 &    # This achieves the same effect and basically says, redirect stdout to /dev/null, and redirect stderr to stdout

Or perhaps you want to run a program silently, but append all of it’s errors to a log file

$ renderManager >/dev/null 2>>logfile.txt

Piping

An alternative to redirection is piping, which works better with joining multiple commands together. A good explanation is here

$ cat Makefile | wc -l    # Pipe the output of the cat command into wc, using it to count the number of lines in the file
508

Here’s a really good example.

Summary

Stream Redirect symbol File Descriptor
stdin < &0
stdout > &1
stderr 2> &2

In the shell what is 2>&1?

Linux Bash/Terminal – Basics – Files/Folders

File/Folder Quick Reference (PAL Series)

Update: These were written for my students while my role was a PAL (Peer Assisted Learning) Leader for my course, I intended to keep all the basic/intermediate commands necessary for terminal use in one place to ease the students into using Linux efficiently. I’ve kept them up just in case they’re of use to someone.

Making directories

Create a directory :

$ mkdir myDir

Create more than one directory at once :

$ mkdir dir1 dir2

Create subdirectory :

$ mkdir myDir/mySubdir

Create a subdirectory even if it’s parent does not exist :

$ mkdir -p i/don\'t/exist/yet/subdir

Creating files

Create an empty file

$ touch empty.txt

Create multiple empty files

$ touch hello.c world.c Makefile

Updating the timestamp for an existing file

$ ls -l output.exr 
-rw-r--r--. 1 tom tom 3291 Oct 30 21:23 output.exr
$ touch output.exr 
$ ls -l output.exr 
-rw-r--r--. 1 tom tom 3291 Nov 25 22:28 output.exr

Reading files

cat – Echo to terminal

$ cat hello.c # Echo the contents to the terminal
hello
world
hello
world
hello
world
hello
world
$

cat’s main purpose is actually to combine files into new files, but we can use it to print the contents too.
More info (the example uses redirection, covered here)

more – Scroll through large file (old utility)

$ more hello.c
hello
world
hello
world
--More-- 50%   # We can only move forward by pressing Enter, no scrolling back

less – Scroll through large file (modern utility, behaves like more but can scroll backwards too, use this because less is more )

$ less hello.c
hello
world
hello
world
hello.c       # We can move forward and back

What’s the difference?

Linux Bash/Terminal – Basics – Filesystem

Fatal error: Array and string offset access syntax with curly braces is no longer supported in /customers/5/6/8/goto-tom.co.uk/httpd.www/blog/wp-content/plugins/easy-table/inc/Encoding.php on line 156 WordPress › Error

There has been a critical error on this website.

Learn more about troubleshooting WordPress.