Yash Verma / Cloud runbook

Debugging commands

11 commands from the cloud runbook. Fill in a placeholder once and every command uses it.

Trace system calls of a process

Prints out all system calls a PID makes (crucial for deep debugging).

strace -p <pid>

Slows the traced process noticeably; be careful on busy production hosts.

strace, syscalls

List open files by process

Shows all files, sockets, and connections opened by a PID.

lsof -p <pid>

lsof, files, sockets

Live CPU profiling (perf)

Shows a live view of the hottest CPU functions in the system.

sudo perf top

perf, cpu, profiling

Virtual memory stats

Shows system performance statistics (CPU, memory, I/O) every 2 seconds.

vmstat 2 10

vmstat, memory, io

Disk I/O statistics

Shows CPU and disk I/O stats, refreshing every second.

iostat -xz 1

iostat, disk, io

Count connections by state

Groups and counts TCP connections by their state (ESTABLISHED, TIME_WAIT, etc.).

ss -tan | awk 'NR>1 {print $1}' | sort | uniq -c | sort -rn

connections, tcp, debug

Capture a core dump of a running process

Writes a core file and lets the process carry on; it pauses only while the dump is written. gcore ships with gdb.

sudo gcore -o core <pid>

core, gdb, post-mortem

Curl with timing breakdown

Shows detailed timing of DNS, connect, TLS, and transfer phases.

curl -o /dev/null -s -w "DNS: %{time_namelookup}s\nConnect: %{time_connect}s\nTLS: %{time_appconnect}s\nTotal: %{time_total}s\n" https://example.com

curl, timing, latency

Debug a pod with an ephemeral container

Attaches a temporary debug container that shares the target container's processes; useful when the image has no shell.

kubectl debug -it <pod-name> -n <namespace> --image=busybox:1.36 --target=<container-name>

kubernetes, ephemeral, shell

Find deleted files still held open

Lists files that were deleted but are still open. Their space is not freed until the process closes them, so a disk can be full while du looks fine.

sudo lsof +L1

disk, lsof, space

Find out-of-memory kills

Searches the kernel log for processes the OOM killer ended.

sudo journalctl -k | grep -iE 'out of memory|killed process'

oom, memory, kernel