SSH hardening without locking yourself out

What you are aiming for

• Login by public key only. No passwords accepted at all.
• No direct root login; you arrive as a normal user and use sudo.
• A short allow-list of users who may log in.
• Every change in one drop-in file you can read, diff and copy to the next server.

Make a good key on your machine

ssh-keygen -t ed25519 -a 64 -C "you@laptop-2026"

Set a passphrase and let your agent (ssh-agent, macOS Keychain, 1Password, or a hardware key) hold it. -a 64 raises the key derivation rounds, which slows offline guessing if the private key file is ever stolen. For a hardware token, use -t ed25519-sk instead; the private key then never leaves the device.

Copy the public half to the server. If the account still accepts passwords, ssh-copy-id does it in one step:

ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected]

Otherwise, paste the contents of id_ed25519.pub into the SSH keys section of the portal, or into ~/.ssh/authorized_keys from the browser console.

Write a drop-in, not an edit

Debian 12, Ubuntu 22.04 and later, and AlmaLinux/Rocky 9 all ship an Include /etc/ssh/sshd_config.d/*.conf line at the top of the main config. The first value sshd reads for each keyword wins, so a file in that directory overrides the defaults below it.

sudo tee /etc/ssh/sshd_config.d/10-hardening.conf >/dev/null <<'EOF'
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AuthenticationMethods publickey
AllowUsers deploy
MaxAuthTries 3
LoginGraceTime 20
X11Forwarding no
ClientAliveInterval 300
ClientAliveCountMax 2
EOF

Watch for files that load earlier than yours. Cloud images sometimes include 50-cloud-init.conf with PasswordAuthentication yes. Because 10- sorts first, your file wins, but check with the command below rather than trusting the filename.

Validate before you restart

sudo sshd -t && echo "config ok"
sudo sshd -T | grep -Ei '^(permitrootlogin|passwordauthentication|allowusers|authenticationmethods)'

Expected output from the second command:

permitrootlogin no
passwordauthentication no
authenticationmethods publickey
allowusers deploy

sshd -T prints the effective configuration after every include has been merged. If it disagrees with your file, another file is setting the value first.

Restart and prove it from a second terminal

sudo systemctl restart ssh      # Debian/Ubuntu (service is "ssh")
sudo systemctl restart sshd     # AlmaLinux/Rocky

On Ubuntu 24.04, sshd is socket-activated by default; restarting ssh.service still picks up the new config. Keep your current session open, then from a new terminal:

ssh [email protected]                             # should work
ssh -o PubkeyAuthentication=no [email protected]  # should fail
ssh [email protected]                               # should fail

The second attempt should end with Permission denied (publickey). That message is the confirmation that passwords are no longer offered at all.

Optional: fail2ban

sudo apt install -y fail2ban
sudo tee /etc/fail2ban/jail.d/sshd.local >/dev/null <<'EOF'
[sshd]
enabled = true
backend = systemd
maxretry = 4
findtime = 10m
bantime = 1h
EOF
sudo systemctl enable --now fail2ban
sudo fail2ban-client status sshd

On AlmaLinux and Rocky, install epel-release first, then fail2ban. The backend = systemd line matters on modern images, which no longer write /var/log/auth.log by default.

Narrow who can reach port 22

If your team connects from a few fixed addresses, allow port 22 only from those in the portal firewall and in nftables. If people work from anywhere, put SSH behind WireGuard and expose only the UDP port for the tunnel. Either way, the public internet stops seeing an SSH banner at all.

Keys for automation

CI pipelines and deploy scripts should get their own keys, never a copy of yours. Restrict them in authorized_keys:

restrict,command="/usr/local/bin/deploy.sh",from="198.51.100.0/24" ssh-ed25519 AAAA... ci-deploy

restrict turns off port forwarding, agent forwarding and PTY allocation in one word. command= pins the key to a single script, so a leaked CI secret cannot open a shell.

Pitfalls

• Wrong permissions: sshd ignores authorized_keys if the home directory or .ssh is group- or world-writable. chmod 700 ~/.ssh; chmod 600 ~/.ssh/authorized_keys.
• AllowUsers with a typo locks everybody out, including you. That is why the second-terminal test exists.
• Changing the port on AlmaLinux without sudo semanage port -a -t ssh_port_t -p tcp 2222: sshd fails to bind and SELinux logs the denial.