Scaling iBGP: full mesh, route reflectors, confederations, peer groups and synchronisation

The full mesh problem

Inside an AS, the AS_PATH does not change, so it cannot detect loops. BGP therefore forbids passing a route learned from one iBGP peer to another. Every iBGP router must peer with every other one: n(n-1)/2 sessions. Ten routers need 45 sessions; fifty need 1,225. Every new router means touching every existing one. Two designs fix this: route reflectors and confederations.

Route reflectors

A route reflector (RR), defined in RFC 4456, is allowed to pass iBGP routes on. Routers that peer with it are either clients or non-clients:

• A route from a client is reflected to all other clients and all non-clients.
• A route from a non-client is reflected to clients only.
• A route from an eBGP peer goes to everyone, as normal.

Clients need no special configuration; they simply peer with the RR. Two attributes prevent loops: ORIGINATOR_ID records the router that first introduced the route, and CLUSTER_LIST records each cluster the route passed through. A router ignores a route that carries its own ID or cluster.

router bgp 64500
 neighbor 10.255.0.11 remote-as 64500
 neighbor 10.255.0.11 update-source Loopback0
 neighbor 10.255.0.11 route-reflector-client
 neighbor 10.255.0.12 remote-as 64500
 neighbor 10.255.0.12 update-source Loopback0
 neighbor 10.255.0.12 route-reflector-client

Always deploy at least two reflectors so the RR is not a single point of failure. A reflector advertises only its own best path, so clients may lose visibility of alternative paths; BGP Add-Path can help, as described in BGP security and convergence.

Confederations

A confederation (RFC 5065) splits one AS into several member ASes, usually with private ASNs. Inside each member AS you still need a full mesh (or reflectors), but between member ASes the sessions behave like eBGP for loop prevention, while keeping next hop, local preference and MED as if they were iBGP. To the outside world the whole confederation looks like a single AS.

router bgp 65001
 bgp confederation identifier 64500
 bgp confederation peers 65002 65003

Confederations suit very large networks or mergers, but they are a bigger redesign, so most enterprises and many providers use route reflectors instead.

Peer groups, templates and synchronisation

Many neighbours often share the same settings. A peer group defines them once; IOS also generates updates once for the whole group. Peer templates are more flexible: session templates hold settings such as remote AS and update source, and policy templates hold filters, and templates can inherit from each other. Modern IOS builds update groups automatically, so peer groups are now mainly about tidy configuration.

router bgp 64500
 neighbor RR-CLIENTS peer-group
 neighbor RR-CLIENTS remote-as 64500
 neighbor RR-CLIENTS update-source Loopback0
 neighbor RR-CLIENTS route-reflector-client
 neighbor 10.255.0.13 peer-group RR-CLIENTS

BGP synchronisation was an old rule: do not use or advertise an iBGP route until the IGP also knows it. It made sense when not every router ran BGP, but it forced redistribution of BGP into the IGP. It has been disabled by default since early 2000s IOS releases, and modern designs run BGP on every router that needs external routes, or use MPLS so that core routers do not need them at all.

Real-world example: a regional provider grew from 6 to 30 routers running iBGP. Adding each new router meant editing every existing one, and a missed session once left a town without some internet routes for a morning. The team introduced two route reflectors in different data centres, made every other router a client of both, and used a peer group on the reflectors. Adding a new router now means configuring two sessions on the new router and two lines on each reflector.