← all cheat sheets
OPERATOR REFERENCE · LINUX SERVER ADMIN
Linux Server Commands
Files · Users · Processes · Services · Network · Storage
the day-to-day sysadmin toolkit — one page, systemd-era, distro-neutral where it can be
Scope note: commands target modern systemd-based distros (Ubuntu/Debian, RHEL/Rocky/Alma, Fedora, SUSE). Package manager, firewall, and log paths differ by family — those are called out. Older SysV/init or minimal containers may lack systemctl, ip, or journalctl — VERIFY against your box. Anything destructive is flagged; read before you paste as root.
INSPECT · what's the state?
change
ACT · config / service / fix
confirm
VERIFY · did it take?
01 First 60 Seconds on a New Box
whoami ; id
hostnamectl
uptime
Identity, privileges, hostname/OS/kernel, and how long it's been up (a recent reboot is often the "what changed").
cat /etc/os-release
uname -a
arch
/etc/os-release tells you which package manager and paths apply before you run anything family-specific.
nproc ; free -h
df -h ; lsblk
top -bn1 | head -20
CPU count, memory, disk usage, block devices — the shape of the machine in four commands.
SVC
What's Running / Broken
systemctl list-units --failed
systemctl list-units --type=service --state=running
ss -tulpn
--failed first — the fastest read of what's already broken. ss -tulpn shows listening ports + owning process.
02 Files, Directories & Navigation
| Task | Command | Notes |
|---|
| List detailed / hidden | ls -lah | Long, human sizes, includes dotfiles |
| Tree of dir sizes | du -sh * | sort -h | Find what's eating space in the current dir |
| Disk free | df -h | Check per-mount; a full / or /var breaks logging & installs silently |
| Copy / move / remove | cp -a src dst · mv a b · rm -i f | -a preserves perms/links; -i prompts before delete |
| Make dirs / empty file | mkdir -p a/b/c · touch f | -p creates parents as needed |
| Find by name | find / -name 'nginx.conf' 2>/dev/null | Redirect stderr to hide permission-denied noise |
| Find by size / age | find /var/log -type f -size +100M -mtime +30 | Big, old files — classic disk-cleanup hunt |
| Symlink | ln -s /real/path /link | -s = symbolic; without it you get a hard link |
| Where is a binary | which cmd · type cmd · command -v cmd | Resolve what actually runs from your PATH |
| Archive / compress | tar czf a.tgz dir/ · tar xzf a.tgz | create/extract, gzip, file; use J for xz, j for bzip2 |
03 Viewing & Searching Text
less +F /var/log/syslog # tail-follow
head -n 50 file ; tail -n 50 file
tail -f /var/log/nginx/access.log
less beats cat for big files; tail -f follows a live log. In less, Shift+F follows, Ctrl+C stops.
grep -rin 'error' /var/log/
grep -v '^#' config | grep -v '^$'
journalctl -u nginx | grep -i fail
-recursive -ignore-case -n line numbers -v invert. The two--v trick strips comments & blank lines from a config.
awk '{print $1}' access.log | sort | uniq -c
sed -n '10,20p' file
cut -d: -f1 /etc/passwd
Column extraction, ranged printing, delimiter splitting. sort | uniq -c | sort -rn = a top-N frequency count of anything.
04 Users, Groups & Permissions
USR
User & Group Management
useradd -m -s /bin/bash alice
passwd alice
usermod -aG sudo alice # add to group
userdel -r alice # -r removes home
groupadd devs ; groups alice
-aG is critical — append to groups; -G alone replaces all secondary groups and can lock a user out of sudo.
On RHEL the sudo group is wheel, on Debian/Ubuntu it's sudo. VERIFY before assuming.
PRM
Permissions & Ownership
chmod 640 file # rw-r-----
chmod -R u+rwX,go-w dir
chown -R alice:devs /srv/app
umask # default mask
Octal: 4=r 2=w 1=x, per owner/group/other. Capital X sets execute only on dirs and already-executable files — safe for recursive use.
sudo -l # what can I run?
sudo -i # root login shell
visudo # edit safely
sudo -u www-data cmd # run as another user
Always edit sudoers with visudo — it syntax-checks before saving, so a typo can't lock everyone out of root.
find / -perm -4000 2>/dev/null # SUID
chmod +t /shared # sticky bit
getfacl file ; setfacl -m u:bob:rw file
SUID/SGID run as the file owner — audit them. Sticky bit on a shared dir stops users deleting each other's files. ACLs give finer grants than owner/group/other.
05 Processes & Resource Monitoring
| Task | Command | Notes |
|---|
| Live process view | top · htop | htop (if installed) is interactive — scroll, filter, kill by F-keys |
| Snapshot all processes | ps aux --sort=-%cpu | head | Sort by CPU or -%mem to find the hog |
| Process tree | pstree -p | Shows parent/child lineage with PIDs |
| Find a process | pgrep -a nginx · pidof sshd | pgrep -a shows the full command line |
| Signal / kill | kill -TERM PID · kill -9 PID | Try TERM (graceful) first; -9 (KILL) is last resort — no cleanup |
| Kill by name | pkill -f pattern · killall proc | -f matches the whole command line, not just the name |
| What's using a file/port | lsof -i :443 · fuser -v /mnt/data | Find the process holding a port or a busy mountpoint |
| Memory pressure | free -h · vmstat 1 | Watch si/so (swap in/out) in vmstat — sustained swap = real pressure |
| Disk I/O | iostat -x 1 · iotop | High %util/await = disk bound, not necessarily throughput |
| Load over time | uptime · sar -q | Load avg vs core count; sar gives history if sysstat is enabled |
Background & detach: cmd & backgrounds it, jobs/fg/bg manage them, nohup cmd & or tmux/screen survive a disconnect. For long jobs on SSH, always use tmux/screen.
06 Services & systemd
SVC
systemctl — Service Control
systemctl status nginx
systemctl start|stop|restart nginx
systemctl reload nginx # re-read config, no drop
systemctl enable --now nginx # boot + start now
systemctl disable nginx
reload re-reads config without dropping connections where the service supports it; restart fully bounces it. enable = start at boot, separate from running now.
journalctl -u nginx -e # unit, jump to end
journalctl -u nginx -f # follow live
journalctl -b # this boot
journalctl -p err -b # errors only
journalctl --since '1 hour ago'
-u filters to one unit, -b -1 is the previous boot (useful after a crash), -p filters by priority. --disk-usage / --vacuum-time=7d to manage log size.
systemctl list-units --failed
systemctl cat nginx # show unit file
systemctl edit nginx # override drop-in
systemctl daemon-reload # after editing units
Edit with systemctl edit (creates a drop-in override) rather than hand-editing vendor unit files that a package update will overwrite. Always daemon-reload after changing unit files.
systemctl list-timers
systemd-analyze blame # slow boot units
systemctl get-default # target/runlevel
systemctl reboot | poweroff
systemd timers are the modern cron replacement. systemd-analyze blame ranks what's slowing boot — faster than guessing which service to disable.
07 Networking
| Task | Command | Notes |
|---|
| Show interfaces / IPs | ip addr · ip -br a | -br = brief one-line-per-interface; ifconfig is deprecated |
| Routing table | ip route · ip r get 8.8.8.8 | get shows which route/interface a destination actually uses |
| Listening ports + PID | ss -tulpn | Modern netstat; -l listening -p process -n numeric |
| Test a port | nc -zv host 443 · curl -v host:443 | TCP reachability without the app protocol succeeding |
| DNS lookup | dig host +short · resolvectl query host | dig @8.8.8.8 host bypasses the local resolver to isolate the fault |
| Reachability / path | ping host · mtr host · traceroute host | mtr = continuous ping-per-hop, best for intermittent loss |
| HTTP debug | curl -Iv https://host · wget -qO- url | -I headers only, -v shows TLS + redirects |
| ARP / neighbors | ip neigh | L2 neighbor table on the local segment |
| NetworkManager | nmcli dev status · nmcli con up NAME | On NM-managed hosts, prefer nmcli over hand-editing configs |
| Capture packets | tcpdump -ni eth0 port 443 -w cap.pcap | -n no DNS, -w writes a pcap for Wireshark |
08 Firewall — Know Which One Is Active
ufw status verbose
ufw allow 22/tcp
ufw allow from 10.0.0.0/8 to any port 5432
ufw enable ; ufw delete allow 80
Simplest frontend. Allow SSH before you enable or you lock yourself out of a remote box.
Enabling a firewall over SSH without allowing 22 first = instant lockout. Test on console access when you can.
FWD
firewalld (RHEL/Rocky)
firewall-cmd --state
firewall-cmd --list-all
firewall-cmd --add-service=https --permanent
firewall-cmd --reload
Zone-based. --permanent writes config but doesn't apply until --reload — a very common gotcha.
nft list ruleset
iptables -L -n -v --line-numbers
iptables -S
The low-level layer ufw/firewalld drive. nftables is the modern replacement for iptables — one may be a compat shim over the other. VERIFY which manages rules.
09 Storage, Disks & Mounts
| Task | Command | Notes |
|---|
| Block devices & mounts | lsblk -f · blkid | -f shows filesystem + UUID + mountpoint |
| Disk usage / free | df -hT · du -sh /var/* | -T also shows filesystem type |
| Find inode exhaustion | df -i | Disk "full" with free space? Out of inodes — often millions of tiny files |
| Mount / unmount | mount /dev/sdb1 /mnt · umount /mnt | umount -l lazy-unmounts a busy mount |
| Persistent mounts | vi /etc/fstab ; mount -a | mount -a tests fstab without a reboot — do this before rebooting |
| Format filesystem | mkfs.ext4 /dev/sdb1 | destroys data — triple-check the device name |
| LVM overview | pvs · vgs · lvs | Physical → volume group → logical volume; lvextend -r grows LV + FS |
| Disk health | smartctl -a /dev/sda | SMART attributes — reallocated sectors / pending = failing drive |
| Swap | swapon --show · free -h | Confirm swap exists & size before diagnosing memory pressure |
Wrong-device mistakes here are unrecoverable. Confirm with lsblk immediately before any mkfs, dd, or partition edit — device letters (sda/sdb) can shift between boots.
10 Package Management — By Distro Family
| Task | Debian/Ubuntu (apt) | RHEL/Rocky/Fedora (dnf) | SUSE (zypper) |
|---|
| Update index | apt update | dnf check-update | zypper refresh |
| Upgrade all | apt upgrade | dnf upgrade | zypper update |
| Install | apt install nginx | dnf install nginx | zypper install nginx |
| Remove | apt remove nginx | dnf remove nginx | zypper remove nginx |
| Search | apt search term | dnf search term | zypper search term |
| What provides a file | dpkg -S /path · apt-file search | dnf provides /path | zypper what-provides |
| List installed | dpkg -l · apt list --installed | rpm -qa · dnf list installed | zypper se -i |
| Package of a command | dpkg -S $(which cmd) | rpm -qf $(which cmd) | rpm -qf $(which cmd) |
Also common: snap and flatpak for sandboxed apps, pip/npm for language packages — keep OS packages and language packages mentally separate. apt full-upgrade / dnf handle dependency changes that plain upgrade won't.
11 SSH, Transfers & Remote Ops
ssh-keygen -t ed25519 -C "you@host"
ssh-copy-id user@host
ssh -i ~/.ssh/id_ed25519 user@host
ssh -J jump user@internal # jump host
Ed25519 keys over passwords. -J proxies through a bastion. Config lives in ~/.ssh/config for reusable host aliases.
scp file user@host:/path/
rsync -avz --progress src/ user@host:/dst/
rsync -avzn src/ dst/ # -n dry run
rsync beats scp for large/repeat transfers — only sends deltas, resumes, and -n previews first. Mind the trailing slash on source dirs.
# /etc/ssh/sshd_config
PermitRootLogin no
PasswordAuthentication no
sshd -t # test config
systemctl reload sshd
Key-only auth, no direct root. Always sshd -t to validate config and keep your current session open until a new one connects.
Don't close your only session after editing sshd_config — open a second connection to confirm you can still log in.
12 Scheduling, Env & Shell
crontab -e ; crontab -l
# min hr dom mon dow cmd
0 2 * * * /usr/local/bin/backup.sh
systemctl list-timers # systemd alt
Fields: minute, hour, day-of-month, month, day-of-week. Use full paths and redirect output (>>log 2>&1) — cron has a minimal environment and won't find your PATH.
env · printenv PATH
export VAR=value
history · !! · !$
alias ll='ls -lah'
Persist exports/aliases in ~/.bashrc (interactive) or /etc/environment (system-wide). !! reruns last command, !$ is its last argument.
cmd > out.txt 2>&1 # stdout+stderr
cmd | tee log.txt # see + save
cmd1 && cmd2 || echo fail
diff <(cmd1) <(cmd2) # process sub
2>&1 merges stderr into stdout, tee writes and displays, &&/|| chain on success/failure.
tmux new -s work
tmux attach -t work # reconnect
# Ctrl-b d = detach
screen -S work ; screen -r
Run long jobs (upgrades, migrations, restores) inside tmux/screen so a dropped SSH connection doesn't kill them mid-run.
13 Server Troubleshooting — Symptom → First Check
| Symptom | First Checks |
|---|
| Service won't start | systemctl status svc ; journalctl -u svc -e |
| "No space left on device" | df -h ; df -i ; du -sh /var/* ; journalctl --disk-usage |
| Load average very high | top ; ps aux --sort=-%cpu ; uptime ; iostat -x 1 |
| Out of memory / OOM kills | free -h ; dmesg -T | grep -i oom ; journalctl -k |
| Port not reachable | ss -tulpn ; firewall status ; curl -v localhost:PORT |
| Can't resolve DNS | resolvectl status ; dig @8.8.8.8 host ; cat /etc/resolv.conf |
| Slow disk / I/O wait | iostat -x 1 ; iotop ; dmesg -T | tail (I/O errors) |
| Hardware / kernel faults | dmesg -T | tail -50 ; journalctl -k -b |
| Recent change / who did what | last ; journalctl --since '2 hours ago' ; ls -lt /etc |
| Config edited but no effect | systemctl reload/restart svc ; check for a config test flag first |
Method: reproduce → check the service's own status + logs → check the resource it depends on (disk, memory, network, a downstream service) → change one thing → verify under the same conditions. dmesg and journalctl answer most "what just happened" questions faster than guessing.