OSPF authentication: plain text, MD5, HMAC-SHA and TTL security

Why authenticate OSPF

OSPF trusts its neighbours. Any device that can send valid hellos on an OSPF-enabled segment can become a neighbour and inject routes, whether that is a misconfigured lab router or an attacker trying to redirect traffic. Passive interfaces close most user-facing ports, but on links between routers you also want proof that a neighbour is one of yours. OSPF authentication adds a shared secret to every packet.

Plain text and MD5

OSPFv2 defines three authentication types in its packet header: 0 (none), 1 (simple password) and 2 (cryptographic, originally MD5).

• Plain text puts the password itself in every packet. Anyone capturing traffic can read it. It only protects against accidents.
• MD5 sends a hash of the packet and key, plus a sequence number that guards against simple replay. The key never crosses the wire, but MD5 is now considered weak.

! Plain text on one interface (legacy)
interface GigabitEthernet0/1
 ip ospf authentication
 ip ospf authentication-key Lab0nly1
!
! MD5 on one interface (legacy)
interface GigabitEthernet0/2
 ip ospf authentication message-digest
 ip ospf message-digest-key 1 md5 Str0ngKey!2026
!
! Or turn MD5 on for every interface in an area
router ospf 1
 area 0 authentication message-digest

Area-wide authentication only sets the type; each interface still needs its key. Both ends of a link must agree on type, key ID and key, or the adjacency drops and a log message reports an authentication mismatch.

HMAC-SHA with key chains

Current IOS and IOS XE releases support HMAC-SHA authentication for OSPFv2 (described in RFC 5709) through a key chain. Key chains hold several keys with start and end times, so you can roll to a new key without an outage.

key chain OSPF-CORE
 key 1
  key-string C0re-Key-A
  cryptographic-algorithm hmac-sha-256
 key 2
  key-string C0re-Key-B
  cryptographic-algorithm hmac-sha-256
  send-lifetime 00:00:00 Jan 1 2027 infinite
!
interface GigabitEthernet0/1
 ip ospf authentication key-chain OSPF-CORE

To rotate keys, add the new key to every router first, set when it starts being sent, and remove the old key after all routers have moved. Keep router clocks in sync with NTP, otherwise key lifetimes start at different moments.

For OSPFv3 (IPv6), authentication uses IPsec or the newer authentication trailer instead of fields in the header.

TTL security and virtual links

OSPF packets between neighbours are sent with a TTL of 1 for multicast, but unicast packets can travel further. TTL security tells the router to accept OSPF packets only if their TTL shows they came from at most a set number of hops away. A forged packet from across the network arrives with a lower TTL and is dropped before it reaches the OSPF process.

router ospf 1
 ttl-security all-interfaces hops 1

Virtual links belong to area 0. If area 0 uses authentication, the virtual link must be configured with matching authentication too, or it will not come up.

Real-world example: a school trust's shared core

A trust of four schools shared one core network, and each school had a local technician with access to its own router. During a project, a technician connected a spare router to a core VLAN to test something, and it formed an OSPF adjacency and advertised an old default route. Half of one school lost internet access for an hour. The network manager then enabled HMAC-SHA-256 authentication through a key chain on every core link, stored the key in the trust's password vault, and added passive-interface default everywhere. A test router plugged in later simply logged an authentication mismatch and was ignored.