SSH Remote Access: Step-by-Step Guide for Linux Beginners

SSH (Secure Shell) lets you control a remote Linux system as if you were sitting at its keyboard. Whether you're managing a server, connecting to a cloud instance, or accessing your home machine from work — SSH is the tool.

This guide walks you through SSH from your very first connection to key-based auth and common use cases.

What Is SSH?

SSH is a protocol for creating an encrypted connection between two computers over a network. The connection works entirely through a terminal — you type commands on your machine, they execute on the remote machine, and you see the output.

SSH replaced older protocols like Telnet and rsh because those sent everything (including passwords) in plain text. SSH encrypts everything.

Basic Connection

ssh username@hostname

Examples:

ssh alice@192.168.1.50        # connect by IP address
ssh alice@server.example.com  # connect by hostname
ssh alice@server.example.com -p 2222  # non-standard port

The first time you connect to a new server, SSH shows a fingerprint prompt:

The authenticity of host '192.168.1.50 (192.168.1.50)' can't be established.
ED25519 key fingerprint is SHA256:abc123...
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Type yes to accept and add the host to your ~/.ssh/known_hosts file. On subsequent connections, SSH verifies the fingerprint matches — if it doesn't, you'll get a warning (which can indicate a man-in-the-middle attack or that the server was reinstalled).

Installing SSH Server

To connect to a machine, it needs an SSH server (sshd) running.

# Debian/Ubuntu
sudo apt install openssh-server
sudo systemctl enable --now ssh

# Fedora/RHEL
sudo dnf install openssh-server
sudo systemctl enable --now sshd

# Check status
sudo systemctl status sshd

The SSH server listens on port 22 by default. Make sure your firewall allows it:

sudo ufw allow ssh          # Ubuntu with ufw
sudo ufw allow 22/tcp       # alternative

SSH Key Authentication (Recommended)

Password authentication works, but SSH keys are more secure and more convenient — no password prompts once configured.

How it works

You generate a key pair:

Private key — stays on your local machine, never shared
Public key — copied to the server

When you connect, the server challenges you with something only your private key can answer. No password travels over the network.

Generating a key pair

ssh-keygen -t ed25519 -C "your-comment-here"

-t ed25519 — use the modern Ed25519 algorithm (preferred over RSA)
-C — a label so you can identify the key later

Prompts:

File location — accept the default (~/.ssh/id_ed25519) or choose a different path
Passphrase — optional but recommended for private keys on shared machines; leave empty for automated scripts

This creates two files:

~/.ssh/id_ed25519      # private key — never share this
~/.ssh/id_ed25519.pub  # public key — this goes on servers

Copying the public key to a server

ssh-copy-id alice@192.168.1.50

If ssh-copy-id isn't available:

cat ~/.ssh/id_ed25519.pub | ssh alice@192.168.1.50"mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

Both commands append your public key to ~/.ssh/authorized_keys on the server.

Testing key authentication

ssh alice@192.168.1.50
# should connect without a password prompt

SSH Configuration File

You don't have to type long ssh commands every time. The ~/.ssh/config file stores connection settings:

Host homeserver
    HostName 192.168.1.50
    User alice
    Port 22
    IdentityFile ~/.ssh/id_ed25519

Host workserver
    HostName work.example.com
    User alicework
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_work

Now you can connect with just:

ssh homeserver    # instead of: ssh alice@192.168.1.50
ssh workserver    # instead of: ssh alicework@work.example.com -p 2222

Common config options:

Option	Effect
`HostName`	IP or hostname
`User`	Username on the remote system
`Port`	SSH port (default 22)
`IdentityFile`	Which private key to use
`ServerAliveInterval 60`	Send keepalive every 60 seconds
`ForwardAgent yes`	Forward your SSH agent to the remote host

Copying Files with scp and rsync

scp — secure copy

scp localfile.txt alice@server:/home/alice/       # upload
scp alice@server:/home/alice/file.txt ./          # download
scp -r ./localdir alice@server:/home/alice/       # upload a directory
scp alice@server:/path/to/file alice@other:/dest  # server to server

scp uses the same auth as ssh — it'll use your key if configured.

rsync — efficient sync

rsync only transfers what's changed, making it much faster for repeated syncs:

rsync -avz ./localdir/ alice@server:/remote/dir/       # upload (note trailing slash)
rsync -avz alice@server:/remote/dir/ ./localdir/       # download
rsync -avz --delete ./src/ alice@server:/dest/         # sync (delete removed files)

Flags:

-a — archive mode (preserves permissions, timestamps, symlinks)
-v — verbose
-z — compress during transfer
--delete — remove files on destination that no longer exist locally

SSH Tunneling and Port Forwarding

SSH can forward ports through its encrypted tunnel — useful for accessing services that aren't exposed publicly.

Local port forwarding

Access a remote service locally:

ssh -L 8080:localhost:80 alice@server

Now http://localhost:8080 on your machine reaches port 80 on the server. Useful when the server's port 80 isn't publicly accessible.

Remote port forwarding

Expose a local service on the remote machine:

ssh -R 9000:localhost:3000 alice@server

Now port 9000 on the remote server forwards to port 3000 on your local machine. Useful for demos when you don't have a public IP.

Dynamic port forwarding (SOCKS proxy)

ssh -D 8888 alice@server

Configure your browser to use localhost:8888 as a SOCKS5 proxy. All browser traffic routes through the server.

Keeping Connections Alive

Long SSH sessions can drop due to network timeouts. Prevent this:

In ~/.ssh/config:

Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3

Or per-connection:

ssh -o ServerAliveInterval=60 alice@server

Running Commands Without an Interactive Session

ssh alice@server "ls -la /var/www"          # run a single command
ssh alice@server "sudo systemctl status nginx"
ssh alice@server < local-script.sh          # pipe a local script to remote bash

Common SSH Security Hardening

After initial setup, consider these /etc/ssh/sshd_config changes (requires sudo):

# Disable password authentication (after setting up keys)
PasswordAuthentication no

# Disable root login
PermitRootLogin no

# Change default port (reduces automated scan noise)
Port 2222

# Limit who can SSH in
AllowUsers alice bob

# Restart sshd after changes
sudo systemctl restart sshd

Always verify key auth works before disabling passwords. Locking yourself out of a remote server is a bad day.

Troubleshooting SSH

ssh -v alice@server        # verbose — shows handshake details
ssh -vvv alice@server      # very verbose

# Common issues:
# "Connection refused" — sshd not running, or port blocked by firewall
# "Permission denied (publickey)" — key not in authorized_keys, or wrong key
# "Host key verification failed" — server fingerprint changed (check ~/.ssh/known_hosts)
# "Too many authentication failures" — ssh tried wrong keys; use -i to specify one

# Check server-side logs:
sudo journalctl -u sshd -f
sudo tail -f /var/log/auth.log    # Debian/Ubuntu