Linux Shell Scripting Basics: Automate Tasks with Bash
Learn to write shell scripts that automate repetitive tasks. Understand variables, conditionals, loops, functions, and error handling to build robust automation.
Why Shell Scripting Matters
Shell scripting transforms manual tasks into automated workflows. Instead of typing the same commands every day, you write a script once and run it whenever needed. What takes 20 minutes manually takes 2 seconds with a script.
Scripts aren't just about saving time - they're about consistency and reliability. Scripts execute the same steps every time, in the same order, without forgetting anything. They document your procedures in executable form.
Every system administrator, DevOps engineer, and power user relies on shell scripts. Deployment scripts, backup automation, log processing, system monitoring - these all use shell scripting as the glue that combines Linux tools into powerful workflows.
This guide covers seven fundamental shell scripting concepts that work across all Linux environments - from Ubuntu servers to Arch desktops to curated setups like Omakub. Learn these basics and you'll write scripts that automate tasks, process data, and make your work dramatically more efficient.
Common Shell Scripting Mistakes
Not Quoting Variables
The Problem: Unquoted variables break when they contain spaces or are empty. This is one of the most common shell scripting bugs, causing scripts to fail in mysterious ways.
The Solution: Always quote variables: "$var" not $var. Quote command substitution: "$(command)". Quote arrays: "$@" not $@. The only time to skip quotes is in [[ ]] tests and arithmetic (( )).
Forgetting to Make Scripts Executable
The Problem: Running ./script.sh fails with 'Permission denied' because the execute bit isn't set. This confuses beginners who don't understand file permissions.
The Solution: After creating a script, run chmod +x script.sh to make it executable. Or use bash script.sh to run without execute permission. Check with ls -l to see permissions.
Using = Instead of -eq for Numbers
The Problem: In [ ] tests, = compares strings, not numbers. This causes wrong comparisons: [ 10 = 9 ] is true (string comparison), but [ 10 -eq 9 ] is false (numeric).
The Solution: Use -eq, -ne, -lt, -le, -gt, -ge for numeric comparisons. Use =, != for string comparisons. In [[ ]], you can use = for strings but still need -eq for numbers.
Not Checking Command Success
The Problem: Scripts continue after commands fail, causing cascading errors. A failed mkdir means subsequent operations use the wrong directory. A failed download means processing corrupt data.
The Solution: Use set -e to exit on errors automatically. Or check each critical command: command || error_exit "Command failed". Check $? for specific exit codes. Test that files exist before processing them.
7 Essential Scripting Fundamentals
These fundamentals cover everything you need to write practical, robust shell scripts for automation and task orchestration.
Script Structure and the Shebang Line
Every shell script starts with a shebang line that tells the system which interpreter to use. Learn proper script structure, how to make scripts executable, and best practices for organizing code.
#!/bin/bash
# Script: backup.sh
# Purpose: Backup important files
# Author: Your Name
# Date: 2024-01-15
# Exit on error
set -e
# Exit on undefined variable
set -u
# Variables
BACKUP_DIR="/backup"
SOURCE_DIR="/home/user/documents"
DATE=$(date +%Y%m%d)
# Main logic
echo "Starting backup..."
tar -czf "$BACKUP_DIR/backup-$DATE.tar.gz" "$SOURCE_DIR"
echo "Backup complete: backup-$DATE.tar.gz"
# Exit successfully
exit 0
# Make it executable:
# chmod +x backup.sh
# Run it:
# ./backup.shPro Tips: Always use #!/bin/bash as the first line. Use set -e to make scripts exit on errors (safer for automation). Add comments to explain why, not what (code shows what). Test scripts with bash -n script.sh to check syntax without running.
Variables and Command Substitution
Variables store data for reuse throughout your script. Command substitution captures command output into variables, enabling dynamic script behavior.
#!/bin/bash
# Simple variables
NAME="John"
AGE=30
echo "User: $NAME, Age: $AGE"
# Command substitution
CURRENT_DATE=$(date +%Y-%m-%d)
USER_COUNT=$(who | wc -l)
DISK_USAGE=$(df -h / | awk 'NR==2 {print $5}')
echo "Date: $CURRENT_DATE"
echo "Users logged in: $USER_COUNT"
echo "Disk usage: $DISK_USAGE"
# Using braces for clarity
FILE="report"
echo "Creating ${FILE}_2024.txt"
# Quotes matter
LITERAL='$HOME' # Stores literal "$HOME"
EXPANDED="$HOME" # Expands to /home/user
# Read-only variable
readonly CONFIG_FILE="/etc/myapp.conf"
# Arrays
FILES=("file1.txt" "file2.txt" "file3.txt")
echo "First file: ${FILES[0]}"
echo "All files: ${FILES[@]}"
# Arithmetic
COUNT=5
COUNT=$((COUNT + 1))
echo "Count: $COUNT"Pro Tips: Always quote variables: "$var" not $var (prevents word splitting). Use ${var} braces for clarity. Command substitution with $() is clearer than backticks. Export variables if child processes need them: export VAR=value.
Conditionals: if, elif, else
Conditionals let scripts make decisions based on conditions. Test files, strings, numbers, and command exit codes to control script flow.
#!/bin/bash
# File tests
FILE="data.txt"
if [ -f "$FILE" ]; then
echo "File exists"
else
echo "File not found"
exit 1
fi
# String comparison
USERNAME="admin"
if [ "$USERNAME" = "admin" ]; then
echo "Admin user detected"
fi
# Check if variable is empty
if [ -z "$VAR" ]; then
echo "Variable is empty"
fi
# Numeric comparison
AGE=25
if [ "$AGE" -ge 18 ]; then
echo "Adult"
else
echo "Minor"
fi
# Multiple conditions with [[ ]]
if [[ -f "$FILE" && -r "$FILE" ]]; then
echo "File exists and is readable"
fi
# Pattern matching with [[ ]]
FILENAME="report.pdf"
if [[ "$FILENAME" == *.pdf ]]; then
echo "PDF file detected"
fi
# Testing command success
if ping -c 1 google.com &> /dev/null; then
echo "Internet connection available"
else
echo "No internet connection"
fi
# elif for multiple conditions
SCORE=85
if [ "$SCORE" -ge 90 ]; then
echo "Grade: A"
elif [ "$SCORE" -ge 80 ]; then
echo "Grade: B"
elif [ "$SCORE" -ge 70 ]; then
echo "Grade: C"
else
echo "Grade: F"
fiPro Tips: Always quote variables in tests: [ "$var" = "value" ]. Use [[ ]] for bash scripts (more features, safer). Use [ ] for portable sh scripts. Test file existence before operating on files. Remember: 0 exit code = success/true.
Loops: for, while, and until
Loops process lists, iterate while conditions are true, or repeat until conditions are met. Essential for batch processing and automation.
#!/bin/bash
# for loop with list
for fruit in apple banana orange; do
echo "Fruit: $fruit"
done
# for loop with files
for file in *.txt; do
echo "Processing: $file"
wc -l "$file"
done
# for loop with command output
for user in $(cut -d: -f1 /etc/passwd); do
echo "User: $user"
done
# for loop with range
for i in {1..5}; do
echo "Number: $i"
done
# C-style for loop
for ((i=1; i<=10; i++)); do
echo "Iteration $i"
done
# while loop
COUNT=1
while [ $COUNT -le 5 ]; do
echo "Count: $COUNT"
COUNT=$((COUNT + 1))
done
# Read file line by line
while IFS= read -r line; do
echo "Line: $line"
done < input.txt
# until loop (opposite of while)
COUNT=1
until [ $COUNT -gt 5 ]; do
echo "Count: $COUNT"
COUNT=$((COUNT + 1))
done
# Loop with break
for i in {1..10}; do
if [ $i -eq 5 ]; then
break # Exit loop when i=5
fi
echo $i
done
# Loop with continue
for i in {1..5}; do
if [ $i -eq 3 ]; then
continue # Skip iteration when i=3
fi
echo $i
donePro Tips: Use for loops for known lists, while loops for conditions. Quote variables in loops: for file in "$@". Read files with while read, not for (handles spaces better). Use break to exit early, continue to skip iterations.
Functions: Organizing Reusable Code
Functions group related commands for reuse, make scripts more organized, and enable code abstraction. Essential for larger scripts and maintainability.
#!/bin/bash
# Simple function
greet() {
echo "Hello, $1!"
}
greet "World" # Output: Hello, World!
# Function with local variables
calculate_sum() {
local num1=$1
local num2=$2
local sum=$((num1 + num2))
echo $sum
}
result=$(calculate_sum 5 10)
echo "Sum: $result" # Output: Sum: 15
# Function with return status
check_file() {
local file=$1
if [ -f "$file" ]; then
return 0 # Success
else
return 1 # Failure
fi
}
if check_file "data.txt"; then
echo "File exists"
else
echo "File not found"
fi
# Function with multiple outputs
get_system_info() {
echo "Hostname: $(hostname)"
echo "Uptime: $(uptime -p)"
echo "Users: $(who | wc -l)"
}
get_system_info
# Function that processes all arguments
print_args() {
echo "Number of arguments: $#"
for arg in "$@"; do
echo " Argument: $arg"
done
}
print_args apple banana orange
# Practical example: backup function
backup_directory() {
local source=$1
local destination=$2
local date=$(date +%Y%m%d)
if [ ! -d "$source" ]; then
echo "Error: Source directory not found"
return 1
fi
tar -czf "$destination/backup-$date.tar.gz" "$source"
return 0
}
if backup_directory "/home/user/docs" "/backup"; then
echo "Backup successful"
else
echo "Backup failed"
fiPro Tips: Use functions for any code you repeat. Keep functions focused on one task. Use local for all function variables to avoid conflicts. Return 0 for success, non-zero for errors. Test functions independently before using in main script.
Command-Line Arguments and User Input
Scripts can accept arguments when run and prompt for user input. This makes scripts flexible and interactive.
#!/bin/bash
# Basic argument usage
echo "Script name: $0"
echo "First argument: $1"
echo "Second argument: $2"
echo "Number of arguments: $#"
echo "All arguments: $@"
# Check argument count
if [ $# -lt 2 ]; then
echo "Usage: $0 <source> <destination>"
exit 1
fi
SOURCE=$1
DEST=$2
echo "Copying from $SOURCE to $DEST"
# Default values for optional arguments
NAME=${1:-"Guest"} # Use $1 or default to "Guest"
echo "Hello, $NAME"
# Reading user input
read -p "Enter your name: " USERNAME
echo "Hello, $USERNAME"
# Silent input (for passwords)
read -sp "Enter password: " PASSWORD
echo # New line after silent input
# Input with timeout
if read -t 5 -p "Enter choice (5 sec timeout): " CHOICE; then
echo "You chose: $CHOICE"
else
echo "Timeout! Using default."
fi
# Yes/No confirmation
read -p "Continue? (y/n): " ANSWER
if [[ "$ANSWER" =~ ^[Yy]$ ]]; then
echo "Continuing..."
else
echo "Aborted."
exit 1
fi
# Processing all arguments
echo "Processing files:"
for file in "$@"; do
if [ -f "$file" ]; then
echo " Found: $file"
else
echo " Not found: $file"
fi
done
# Using shift to process arguments
while [ $# -gt 0 ]; do
echo "Processing: $1"
shift # Move to next argument
donePro Tips: Always check argument count before using $1, $2, etc. Quote "$@" to preserve spaces in arguments. Use read -p for prompts, -s for passwords. Provide clear usage messages when arguments are wrong. Use ${var:-default} for default values.
Exit Codes and Error Handling
Exit codes indicate success or failure. Proper error handling makes scripts robust, prevents cascading failures, and aids debugging.
#!/bin/bash
# Exit on error, undefined variables
set -e
set -u
set -o pipefail
# Function to handle errors
error_exit() {
echo "Error: $1" >&2
exit 1
}
# Check if file exists
FILE="data.txt"
[ -f "$FILE" ] || error_exit "File $FILE not found"
# Test command success
if ping -c 1 google.com &> /dev/null; then
echo "Internet available"
else
echo "No internet connection" >&2
exit 1
fi
# Using $? to check exit code
grep "pattern" file.txt
if [ $? -eq 0 ]; then
echo "Pattern found"
else
echo "Pattern not found"
fi
# Short-circuit operators
mkdir /tmp/mydir || error_exit "Failed to create directory"
cd /tmp/mydir || error_exit "Failed to change directory"
# Command succeeds, do next action
rm temp.txt && echo "File removed successfully"
# Cleanup function with trap
cleanup() {
echo "Cleaning up..."
rm -f /tmp/temp_*
}
trap cleanup EXIT
# Error handler
error_handler() {
local line=$1
echo "Error on line $line" >&2
cleanup
exit 1
}
trap 'error_handler $LINENO' ERR
# Critical section
critical_operation() {
# Temporarily disable exit-on-error
set +e
risky_command
local status=$?
set -e
if [ $status -ne 0 ]; then
echo "Warning: risky_command failed"
# Continue anyway
fi
}
# Exit with appropriate code
if [ "$SUCCESS" = true ]; then
exit 0
else
exit 1
fiPro Tips: Use set -e for safer scripts. Return specific exit codes from functions (0=success). Check $? immediately after commands. Use trap for cleanup even on errors. Echo errors to stderr with >&2. Test error handling by intentionally breaking things.
Your First Steps with Shell Scripting
Build scripting skills through hands-on practice with real automation tasks.
Write Your First Simple Script
Create a script that takes a name as argument and greets the user. Include shebang, argument check, and a function. Make it executable with chmod +x. This covers the basic structure and teaches the fundamental workflow.
Practice Conditionals and Loops
Write a script that checks if files exist using if, then processes them with a for loop. Add error messages for missing files. This builds understanding of control flow and error handling in practical context.
Build a Practical Automation Script
Create a backup script that accepts source and destination arguments, checks they exist, creates a dated backup with tar, and reports success or failure. This combines all fundamentals into something useful you'll actually use.
Go Deeper with Shell Scripting
This guide covers essential scripting fundamentals. The Practical Linux Handbook includes advanced scripting techniques, complex automation examples, debugging strategies, and complete real-world scripts.
What You'll Learn:
- Complete shell scripting (bash, variables, functions)
- Advanced scripting patterns and best practices
- Error handling and debugging techniques
- Real-world automation examples
- Script security and safety practices
Beyond Scripting:
- 70+ Linux commands with detailed examples
- File operations and permissions
- Process management and monitoring
- Network administration and troubleshooting
- 3 bonus editor cheat sheets (Vim, Nano, Emacs)
Frequently Asked Questions
What's the difference between sh and bash?
sh is the POSIX standard shell (simpler, more portable). bash is the Bourne Again Shell with more features (arrays, [[ ]], etc.). Use #!/bin/bash for bash-specific features, #!/bin/sh for maximum portability. Most Linux systems have sh as a link to bash or dash.
How do I debug shell scripts?
Use bash -x script.sh to see each command as it executes. Add set -x at the top of your script. Use echo statements to show variable values. Check syntax with bash -n script.sh. Use shellcheck (linter) to find common issues.
When should I use a shell script vs a Python script?
Use shell scripts for: command orchestration, file operations, system admin tasks, quick automation. Use Python for: complex logic, data processing, APIs, cross-platform code. Shell is great for gluing commands together; Python is better for algorithms and data structures.
How do I pass the output of one function to another?
Use command substitution: result=$(function1); function2 "$result". Or pipe: function1 | function2. Or use global variables (less clean). For complex data, consider using files or process substitution.
What does set -euo pipefail do?
set -e exits on errors, -u exits on undefined variables, -o pipefail fails if any command in a pipe fails. Together they make scripts safer by catching common errors early. Add at the top of scripts for production use.
How can I make my script accept both files and stdin?
Use: ${1:--} as the filename. This uses $1 if provided, otherwise - (stdin). Then: while read line; do echo "$line"; done < "${1:--}". Now script works as: ./script.sh file.txt or cat file.txt | ./script.sh.
Continue Your Linux Journey
Ready to Automate with Shell Scripts?
Join thousands who've learned to write powerful automation scripts with The Practical Linux Handbook.