SSH Hardening for Sysadmins: Five‑Stage Rollout and Copyable sshd_config

We harden SSH by requiring key based authentication (Ed25519 or hardware backed credentials for privileged accounts), disabling password and root logins, enforcing modern cipher, key exchange, and MAC selections, applying network level rate limiting, and centralizing authentication logging. We follow the consensus positions of OpenSSH release notes, NIST IR 7966, and CISA identity guidance as the authoritative basis for these decisions, and we validate every configuration change with sshd -t and a backup session before restarting the service.

TL;DR:

• Before editing, confirm the installed OpenSSH version, secure console or other out of band access, back up configuration and keys, and write rollback steps.
• Validate every edit immediately, keep an existing session open, and test access on one canary host through a full business cycle before wider rollout.
• For privileged or production accounts, require hardware backed keys and enforce multi factor authentication through AuthenticationMethods with a PAM module.
• Inventory each key by owner, purpose, creation date, and expiration, then revoke and reissue it immediately after device loss, role change, or suspected compromise.
• Log fingerprints for successful SSH authentications and send records to a central system, so investigators can distinguish keys used under the same account.

Table of Contents

• A Copyable SSH Hardening Checklist
• What to Check Before Touching Your SSH Configuration
• Hardening sshd_config: Authentication, Access, and Algorithms
• Managing SSH Keys: Generation, Inventory, and Hardware Backing
• Firewall Rules and Rate Limiting at the Network Layer
• Designing Bastions and ProxyJump Architecture
• Logging and Centralizing SSH Telemetry
• Keeping OpenSSH Patched Without Breaking Access
• Hardening the Client Side: Workstations and Automation
• Our Phased Rollout and Audit Checklist
• Where Most Teams Get SSH Hardening Wrong
• Get an SSH Audit Instead of Guessing at Your Exposure
• FAQ
• Sources

A Copyable SSH Hardening Checklist

We build our rollout from a prioritized sequence rather than a scattershot edit of sshd_config. Each item below reflects an operational priority, from authentication posture down to patching cadence.

• Require public key authentication for every account; disable password authentication once key distribution is confirmed.
• Generate new keys as Ed25519; reserve RSA at a strong bit length sufficient for legacy systems that cannot accept elliptic curve keys.
• Set PermitRootLogin no and route administrative work through named accounts with sudo.
• Restrict logins with AllowUsers or AllowGroups rather than leaving the account list implicit.
• Lower MaxAuthTries to around 3 and reduce LoginGraceTime to limit the window for credential guessing.
• Disable X11Forwarding and leave AllowAgentForwarding off unless a reviewed workflow requires it.
• Remove weak key exchange, cipher, and MAC algorithms from sshd_config; keep only modern entries.
• Rotate host keys on a defined schedule and confirm known_hosts entries match before trusting a connection.
• Apply firewall rules that limit SSH exposure to known source ranges wherever the environment allows it.
• Centralize authentication logs and alert on repeated failures or unexpected key fingerprints.
• Patch OpenSSH on a staged schedule rather than waiting for a single annual update window.

Before restarting the daemon, we run sshd -t to catch syntax errors, then launch a parallel session with sshd -D -p 2222 to confirm the new configuration accepts connections. We keep an existing root or console session open until the new policy is confirmed, since a single malformed Match block can end remote access entirely. Older clients that cannot negotiate modern algorithms may need a scoped Match exception rather than a blanket rollback.

What to Check Before Touching Your SSH Configuration

A hardening change that locks out the only administrator with access is a more common failure mode than any external attack. We treat the prechecks below as mandatory, not optional.

1. Confirm the installed OpenSSH version and which directives and algorithms it supports, since older builds reject newer syntax outright.
2. Identify an out of band path, console access, hypervisor session, or serial connection, that does not depend on the SSH service staying reachable.
3. Copy sshd_config, the host key files, and every account’s authorized_keys to a location outside the host being modified.
4. Write down the exact rollback command, typically restoring the backed up config file and reloading the service, before making any edit.
5. Validate syntax locally with sshd -t after every edit, never after a batch of edits.
6. Apply the change to one non critical host first and observe login behavior for a full business cycle before expanding the rollout.

Skipping the out of band access check is the single most frequent cause of SSH lockouts we encounter in remediation work, since a firewall rule or a mistyped AllowUsers line can sever the only remaining path to the box.

Hardening sshd_config: Authentication, Access, and Algorithms

The sshd_config file carries most of the actual risk reduction, and the directives fall into four functional groups: authentication, access restriction, timeout and brute force controls, and algorithm selection.

Authentication. PasswordAuthentication no paired with PubkeyAuthentication yes removes the most commonly exploited entry point, since credential stuffing and brute force scanning both depend on password prompts. For environments that require a second factor, AuthenticationMethods publickey,keyboard-interactive combined with a PAM module such as Google Authenticator or Duo enforces layered verification without abandoning key based login. We treat multi factor enforcement as mandatory for any account with elevated privileges, even when the broader fleet still relies on keys alone during a phased rollout.

Access restriction. PermitRootLogin no removes the single most targeted account name from remote reach. AllowUsers deploy admin-jdoe or AllowGroups ssh-users narrows the pool of accounts that can authenticate at all, which matters because a compromised but unused local account is still an open door if it is never explicitly excluded. Match blocks let us apply different policies by network or user group within a single file:

Match Address 10.0.0.0/8
    PasswordAuthentication no
    PermitRootLogin no

Match User backup-agent
    ForceCommand /usr/local/bin/backup-only.sh
    AllowAgentForwarding no

The ForceCommand directive is particularly useful for automation accounts, since it restricts a key’s use to a single predefined action regardless of what the connecting client requests.

Brute force and timeout controls. LoginGraceTime 20 and MaxAuthTries 3 shrink the window an attacker has to attempt credential combinations per connection. Newer OpenSSH releases add PerSourcePenalties, which penalizes repeated failures from a single source address at the daemon level rather than relying solely on an external tool, a meaningful addition given how OpenSSH release notes track this kind of built in mitigation across versions.

Forwarding controls. X11Forwarding no removes a feature almost no production server needs. AllowAgentForwarding should default to no and only be enabled after a specific workflow justifies it, since forwarding exposes an agent socket to every intermediate host in the chain. Unneeded AllowTcpForwarding should likewise be disabled unless a defined tunneling use case exists.

Algorithm selection. Modern OpenSSH defaults already favor [email protected] and the AES-GCM family for ciphers, curve25519 based exchanges for key exchange, and have removed older MD5 and SHA-1 based MACs from default negotiation. We confirm the active list with sshd -T | grep -E "Ciphers|KexAlgorithms|MACs" and prune any legacy entry that a previous administrator may have left enabled for compatibility. Where legacy clients genuinely require an older cipher, we isolate that exception inside a Match Address block scoped to the specific subnet rather than relaxing the global policy.

Managing SSH Keys: Generation, Inventory, and Hardware Backing

Key management determines how much of the hardening above actually holds up over time, since a well configured daemon still fails if keys are generated carelessly or never retired.

• Generate new keys with ssh-keygen -t ed25519 -C "jdoe@admin-host-2026", since Ed25519 is the recommended default for new SSH keys.
• Reserve RSA only for systems that cannot yet accept elliptic curve keys, and require a minimum of 4096 bits when RSA is unavoidable.
• Require hardware backed keys, such as ed25519-sk, for any account with privileged or production access, since a hardware backed security key prevents the private key material from ever leaving the device.
• Maintain an inventory that records key owner, purpose, creation date, and expected cryptoperiod rather than relying on memory or scattered authorized_keys files.
• Revoke and reissue immediately on role change, device loss, or suspected compromise rather than waiting for a scheduled review.

NIST IR 7966 recommends defining explicit cryptoperiods for SSH keys and protecting the write access of authorized_keys files, since a world writable authorized_keys entry undermines every other control in the chain. We move automation and service account keys into root owned, non user writable locations and apply command= and from= restrictions directly in the authorized_keys line, limiting a given key to one command and one source address regardless of what the connecting client sends. Human operated keys should always carry a passphrase, a point echoed in guidance on SSH client and key management hardening, which also recommends a dedicated key per purpose rather than a single key reused across every system an administrator touches.

For fleets beyond a handful of hosts, SSH certificates issued by an internal certificate authority replace the need to distribute and revoke individual authorized_keys entries across every server. A certificate carries a validity window and a principal list, so access expires automatically and revocation becomes a matter of refusing to sign a renewal rather than chasing down every host that holds a copy of a compromised key.

Pro Tip: Tag every key in your inventory with an expiration date at creation time, not retroactively, so rotation becomes a scheduled task rather than an incident response scramble.

Firewall Rules and Rate Limiting at the Network Layer

Algorithm and authentication hardening reduces what a connecting client can do, but network controls reduce how many clients ever reach the negotiation stage at all.

• Restrict inbound SSH with nftables or iptables to known management ranges where the environment permits a fixed source list.
• Apply connection rate limiting at the firewall, for example iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m limit --limit 3/min --limit-burst 5 -j ACCEPT, to slow automated scanning.
• Deploy fail2ban or an equivalent tool to parse authentication logs and temporarily ban source addresses after repeated failures.
• Use cloud security groups or edge filtering to keep management ports off the public internet entirely where the architecture allows it.
• For devices that cannot be hardened directly, such as embedded appliances, place them behind a VPN tunnel or an isolated management network rather than exposing SSH publicly.

NIST IR 7966 and CISA’s identity and access management guidance both treat compensating network controls, VPN tunnels, network segmentation, and edge filtering, as necessary fallbacks when a device’s SSH implementation cannot be directly hardened. We apply that same logic to any legacy appliance still running an outdated SSH daemon: isolate it on a segmented network rather than exposing it alongside fully hardened hosts.

Designing Bastions and ProxyJump Architecture

A bastion host centralizes the point of entry into an environment, which makes it both a high value target and the easiest place to concentrate monitoring and access control.

We keep bastion hosts minimal: no unrelated services, strict multi factor enforcement, and a small, named list of administrators permitted to authenticate at all. Every session through the bastion should be logged centrally, since a compromised bastion with no logging defeats the purpose of centralizing access in the first place.

ProxyJump is the preferred pattern for reaching internal hosts through a bastion, in place of agent forwarding. Agent forwarding exposes the agent socket to every intermediate host in the chain, and a compromised intermediate host can potentially use that forwarded agent to authenticate elsewhere without the key owner’s knowledge. A ProxyJump connection, by contrast, tunnels the session through the bastion without placing the agent socket on it:

ssh -J bastion.internal admin@target-host

Where the environment supports it, short lived certificates issued just in time for a specific session reduce the standing risk of a bastion login credential that remains valid indefinitely.

Logging and Centralizing SSH Telemetry

Configuration hardening reduces the number of successful attack paths, but detection depends entirely on what gets logged and where that log data ends up.

• Log the key fingerprint for every successful authentication, not just the username and source address.
• Record failed authentication attempts with source IP, username attempted, and timestamp for correlation later.
• Ingest SSH authentication logs into a SIEM or an ELK based pipeline rather than leaving them on local disk alone.
• Alert on spikes in authentication failures, unexpected host key changes, or a key fingerprint logging in for the first time.
• Correlate SSH telemetry with intrusion detection or endpoint detection tooling rather than treating SSH logs as an isolated data source.

NIST IR 7966 recommends logging key fingerprints as part of standard operational practice for SSH environments, since a username and password pair alone cannot distinguish between two different keys presented by the same account. Retention policy should align with whatever incident response timeline the organization already commits to, and logs intended for forensic use should be stored in a form that cannot be altered after the fact.

Keeping OpenSSH Patched Without Breaking Access

Hardening is not a one time configuration pass. It is a maintenance discipline that has to survive version upgrades and configuration drift over time. This point is raised directly in discussions of continuous OpenSSH patch management.

We track OpenSSH release notes for both new default algorithms and disclosed vulnerabilities, and we apply security patches on a staged schedule rather than an annual batch. Every change, patch or configuration edit alike, goes through sshd -t for syntax validation and a canary host before wider rollout. We keep the previous sshd_config and host key backups retrievable and document the exact rollback command in advance, so recovery from a bad change does not depend on reconstructing steps from memory during an outage.

Hardening the Client Side: Workstations and Automation

Server side hardening means little if the keys used to connect originate from a poorly secured workstation or an automation pipeline with loose trust settings.

Per host IdentityFile entries in ~/.ssh/config, combined with IdentitiesOnly yes, prevent a client from offering the wrong key to the wrong server, which matters when an administrator holds multiple keys for different environments. Permissions on ~/.ssh and its contents should remain restricted to the owning user, agent lifetimes should be capped rather than left indefinite, and human held keys should always require a passphrase. For CI and automation pipelines, preseeding known_hosts entries avoids a trust on first use decision happening unattended inside a production deployment job.

Our Phased Rollout and Audit Checklist

SSH hardening efforts benefit from a structured five stage sequence: inventory, policy definition, pilot, phased rollout, and ongoing audit. The inventory stage catalogs every host, key, and authorized_keys location before any configuration changes begin, since remediation without a baseline tends to miss forgotten service accounts.

During audit work, it is important to examine key inventory completeness, the location and permissions of every authorized_keys file, bastion configuration and MFA enforcement, and whether logging coverage reaches a central system rather than sitting unread on each host. A runbook built from this sequence, starting with inventory, then piloting on a single host and expanding only after a full observation cycle, can catch access gaps that a one time configuration review may miss.

Where Most Teams Get SSH Hardening Wrong

We find that the riskiest gap in most environments is not an outdated cipher list. It is an authorized_keys file that has accumulated years of unreviewed entries, some tied to employees who left long ago.

Agent forwarding left enabled by default is the second most common mistake we encounter, usually inherited from a workflow convenience decision made years before anyone reviewed it. Prioritization matters more than completeness on day one: harden the accounts with the widest blast radius, root equivalents and automation keys with broad access, before chasing every algorithm setting on every low value host. Incremental, inventoried, tested changes outperform a single sweeping rewrite that nobody fully verified.

— Jesse Hart

Get an SSH Audit Instead of Guessing at Your Exposure

Reading a hardening checklist is one thing. Knowing which of your hosts actually violate it is another, and that gap is where our Technical Systems & Risk Review and Infrastructure Automation & Review engagements start: a direct audit of key inventory, authorized_keys hygiene, bastion configuration, and logging coverage across your actual fleet, not a generic template.

For teams that want to track key lifecycle actions, rotation schedules, and remediation steps as operational records rather than scattered notes, Project Relay ($49 per month) gives you a structured place to log that work over time. If your environment needs a full remediation plan built around your specific architecture, reach out through Technical Consulting to scope an engagement.

FAQ

What is the best hardening checklist to follow?

There is no single official checklist, but the strongest sources converge on the same priorities: key only authentication with Ed25519 or hardware backed keys, disabled root and password logins, modern cipher and key exchange selection, and centralized logging, as reflected in NIST IR 7966 and CISA identity guidance. We treat the checklist in this guide as a direct implementation of that consensus.

Is SSH obsolete?

No. SSH remains the standard protocol for remote administration of Linux and Unix systems, and the risk lies almost entirely in misconfiguration rather than the protocol itself. Properly hardened, with key based authentication, modern algorithms, and network controls, SSH continues to meet current security expectations.

Why is SSH considered a risky port?

Port 22 draws constant automated scanning and brute force attempts simply because it is the default, well known entry point for remote access on most servers. The risk comes from exposure and weak configuration, not from an inherent flaw in the protocol, and controls like key only authentication, rate limiting, and restricting source addresses address that exposure directly.

How do I harden my Linux system step by step?

Start with the prechecks: confirm your OpenSSH version, secure an out of band access path, and back up your configuration and keys. From there, move through authentication hardening, access restriction, algorithm selection, network rate limiting, and centralized logging, testing each change with sshd -t and a canary host before expanding it fleet wide.

Do I need multi factor authentication on top of SSH keys?

Multi factor authentication adds a meaningful layer for privileged accounts even when key based login is already enforced, since it protects against a stolen key paired with a stolen passphrase. We recommend enforcing it through AuthenticationMethods combined with a PAM module for any account with elevated or production access.

Sources

• Creating SSH keys — Caltech IMSS
• Security of Interactive and Automated Access Management Using Secure Shell (SSH) — NIST IR 7966
• OpenSSH release notes
• SSH client and key management hardening — Security Alliance
• Identity and access management: Recommended best practices for administrators — CISA