How to automate Cisco datacenter BGP provisioning with NetBox and Ansible

How to automate Cisco datacenter BGP provisioning with NetBox and Ansible

Automating BGP in a datacenter is less about pushing config faster and more about making intent explicit, repeatable, and verifiable. The combination of netbox (as the source of truth) and Ansible (as the execution engine) can reliably provision and maintain BGP across Cisco IOS/XE and IOS-XR, if the NetBox data model is designed for automation and your Ansible playbooks are built for idempotency and safe change control.

This tutorial walks through an advanced, practical approach.


Design and prerequisites: required NetBox data model and BGP policy assumptions

Before touching Ansible, define the “shape” of the network and what BGP should look like everywhere. Most automation failures come from missing or ambiguous intent.

BGP policy assumptions (datacenter baseline)

This tutorial assumes a common datacenter BGP pattern:

  • Underlay uses eBGP between leaf–spine (or ToR–spine) with:
  • per-link /31 or /30 addressing
  • consistent BFD policy (optional)
  • consistent route-policy / route-map behavior
  • Overlay (optional) uses:
  • iBGP EVPN in a dedicated VRF (or default VRF depending on design)
  • route-reflectors or spines as RR (platform/design dependent)
  • ASNs are allocated:
  • per-device (common in eBGP underlay) or per-pod (less common)
  • Neighbors are derived from physical cabling:
  • interface -> cable -> remote device/interface -> IP addressing -> neighbor

Pick a consistent design. Automation should not infer topology from config.

NetBox objects you should model (minimum)

Devices and interfaces

  • Sites / Regions / Tenants: whatever you need for scoping (optional but helpful)
  • Device roles: spine, leaf (or rr, border)
  • Platforms: to drive OS-specific templates (iosxe, iosxr)
  • Interfaces:
  • names must match the device’s actual interface names
  • mark routed uplinks as L3 (NetBox interface type + IPs)
  • Cabling:
  • model spine/leaf cabling in NetBox so neighbors can be discovered deterministically

Prefixes and IP addresses

  • Prefix objects (per site/pod):
  • UNDERLAY_P2P pool(s) (e.g., /24 subdivided into /31s)
  • LOOPBACK pool(s) for router-ids / BGP update-sources
  • IP addresses assigned to interfaces:
  • each routed interface used for BGP must have IPs assigned in NetBox
  • Loopback interface IPs:
  • used for router-id and for overlay sessions (if applicable)

VRFs

  • Use VRFs for:
  • overlay tenant routing domains (typical)
  • optionally for underlay separation (less common; many keep underlay in default VRF)
  • Consistency matters more than the choice:
  • decide whether underlay is default VRF everywhere or a dedicated VRF everywhere

Custom fields (intent fields) you’ll likely need

NetBox has BGP-specific constructs via plugins, but many teams keep core netbox + custom fields to avoid coupling. A practical approach is to store enough intent to render config deterministically.

Recommended custom fields:

On Device (or Device Role):
- bgp_asn (integer): local ASN
- bgp_router_id (optional): if not derived from loopback IP
- bgp_underlay_enabled (bool)
- bgp_overlay_enabled (bool, optional)

On Interface:
- bgp_peer_group (string): e.g., UNDERLAY_EBGP
- bgp_bfd (bool)
- bgp_password (optional, ideally a reference to a secret name, not the secret itself)
- bgp_description (optional)

On VRF:
- bgp_rd / bgp_rt_import / bgp_rt_export (if doing EVPN/VXLAN; may live elsewhere in your design)

On Prefix:
- role (NetBox built-in) or custom tag like underlay-p2p, loopback, tenant-subnet

If you can compute something reliably (router-id from loopback0), prefer computation over storing a duplicate field.


NetBox best practices: modeling VRFs, peer-groups, route-maps, and intent

NetBox won’t enforce BGP correctness; your data model must.

VRFs: treat them as intent boundaries

Practical modeling rules:
- If a prefix belongs to a tenant, associate it to a VRF (NetBox supports VRF per prefix/IP).
- If an interface is a tenant SVI or routed L3VNI handoff, associate IPs with that VRF.
- Keep underlay addressing in the default VRF unless you have a strong reason not to.

Operationally useful fields for VRFs:
- vrf_id or l3vni (for EVPN)
- route-target import/export sets (even if they are generated later)

Peer-groups: encode policy, not topology

A peer-group should represent a standard policy bundle. Examples:
- UNDERLAY_EBGP_LEAF_TO_SPINE
- UNDERLAY_EBGP_SPINE_TO_LEAF (may differ by policy)
- OVERLAY_IBGP_EVPN_RR_CLIENT
- OVERLAY_IBGP_EVPN_RR

Where to store peer-group definitions:
- Store the peer-group name on the interface (or on a relationship), and keep the peer-group’s policy definition in Ansible group_vars or templates.
- This avoids trying to model route-maps and knobs in NetBox as arbitrary blobs.

Route-maps / route-policies: keep intent in NetBox, logic in Git

NetBox is a good place to hold:
- which policy should apply (name/reference)
- any business-level toggles (e.g., “allow default”)

NetBox is not a great place to hold:
- full route-map bodies as free-form text (hard to validate and diff)

Recommended pattern:
- In NetBox: fields like import_policy: RM-UNDERLAY-IN, export_policy: RM-UNDERLAY-OUT
- In Git: templates implementing those names per OS

Intent fields: build a small schema and stick to it

A small, stable schema is better than many one-off fields. Suggested “contract”:

  • Every BGP neighbor must be derivable from:
  • local device interface + remote device interface (via cable)
  • IPs assigned on those interfaces
  • bgp_peer_group and optional flags from interface custom fields
  • Every device must have:
  • a local ASN (bgp_asn)
  • router-id derivable from loopback0 (or a bgp_router_id override)

Ansible architecture: roles, NetBox dynamic inventory, templating, idempotency

Inventory: use NetBox as the dynamic inventory source

Use the netbox.netbox.nb_inventory inventory plugin to pull devices, platforms, sites, and custom fields.

Key practices:
- Group hosts by platform and/or device_role to select templates
- Filter out non-network devices (or devices without bgp_underlay_enabled)
- Use tags like bgp-managed in netbox for explicit inclusion

Roles: separate data collection, rendering, and deployment

A practical role layout:

  • roles/netbox_facts/
  • queries NetBox API for:
    • device interfaces + IPs
    • connected endpoints (cables)
    • loopbacks
    • VRFs (if needed)
  • produces normalized facts like bgp_neighbors[]
  • roles/bgp_render/
  • Jinja2 templates per OS:
    • iosxe_bgp.j2
    • iosxr_bgp.j2
  • generates candidate configs into files (one per device)
  • roles/bgp_deploy/
  • pushes config with idempotency and diff
  • handles staged deployment and rollback hooks
  • roles/bgp_validate/
  • runs post-checks:
    • session state, prefixes received/advertised, policy presence

Templating patterns for IOS/XE vs IOS-XR

Keep templates OS-specific but fed by the same normalized data model. Normalize once, render many.

Normalized neighbor structure example:
- neighbor_ip
- remote_as
- peer_group
- source_interface (loopback or physical)
- vrf (or default)
- bfd (bool)
- password_ref (not the secret itself)
- description

Idempotency techniques that actually work

  • Prefer resource modules where available, but BGP coverage is uneven across OSes.
  • When using *_config modules:
  • use diff: true
  • use match: line and controlled ordering
  • render deterministic config (stable ordering in Jinja2: sort neighbors)
  • Avoid “show | include” scraping to decide what to configure; use NetBox intent.

Safe deployment workflow: dry-run, config-diff, staged pushes, commits, rollback

Automation should make changes safer, not scarier.

Workflow checklist

  1. Pre-flight validation
  2. Ensure required NetBox fields exist (ASN, loopback, uplink IPs, cables)
  3. Ensure all neighbor links are symmetric and have addressing on both ends
  4. Render candidate config
  5. Store artifacts per run (timestamped directory)
  6. Dry-run / diff
  7. Use device-native diff features (IOS-XE “archive”, IOS-XR “commit replace” preview) where possible
  8. Staged push
  9. Spines first or leaves first depending on design (commonly spines first for underlay)
  10. Use serial/rolling changes (e.g., 10% of fabric at a time)
  11. Commit-like behavior
  12. IOS-XR: use commit confirmed
  13. IOS-XE: emulate with checkpoints + timed rollback playbook
  14. Post-change validation
  15. BGP up, routes present, no unexpected drops
  16. Reconciliation
  17. Any drift found becomes either:
    • corrected config (push), or
    • corrected intent (update netbox)

Example: NetBox inventory + staged deployment + diff (Ansible)

yaml
---
- name: Provision datacenter BGP from netbox intent
  hosts: tag_bgp_managed
  gather_facts: false
  serial: 5   # staged push
  vars:
    ansible_connection: network_cli

  tasks:
    - name: Build normalized BGP intent from netbox
      ansible.builtin.include_role:
        name: netbox_facts

    - name: Render BGP configuration (platform-specific)
      ansible.builtin.include_role:
        name: bgp_render

    - name: Push configuration with diff (IOS-XE/IOS)
      when: ansible_network_os in ["cisco.ios.ios", "cisco.iosxe.iosxe"]
      cisco.ios.ios_config:
        src: "{{ bgp_candidate_config_file }}"
        diff_against: running
        save_when: modified
      register: push_result

    - name: Push configuration with commit confirmed (IOS-XR)
      when: ansible_network_os in ["cisco.iosxr.iosxr"]
      cisco.iosxr.iosxr_config:
        src: "{{ bgp_candidate_config_file }}"
        diff_against: running
        commit: true
        commit_confirmed: true
        commit_confirm_timeout: 300
      register: push_result_xr

    - name: Post-change BGP session validation
      ansible.builtin.include_role:
        name: bgp_validate

Notes:
- tag_bgp_managed comes from the netbox dynamic inventory grouping by tag.
- bgp_candidate_config_file is produced by the render role (and should be deterministic).
- For IOS-XE, consider enabling archive and checkpoint features to support rollback procedures.

Transaction-like commits and rollback strategies

IOS-XR (best case):
- Use commit confirmed <seconds> so a loss of connectivity or failed validation can auto-revert.
- After validation passes, issue a final commit.

IOS/IOS-XE (practical approach):
- Enable config archive and checkpoints on devices.
- Automation flow:
- create a checkpoint
- apply changes
- validate
- if validation fails, revert to checkpoint

Even if rollback is not fully automated initially, at least generate and store:
- the exact candidate config pushed
- the before/after diff
- a “revert snippet” when feasible (or reference to checkpoint ID)


Validation and reconciliation: BGP session checks, route verification, telemetry back into NetBox

Provisioning isn’t done when the config loads. It’s done when the network converges to the intended state.

Automated BGP session checks

Typical checks per device:
- All expected neighbors exist
- Sessions are Established
- Remote AS matches intent
- For eBGP underlay: expected number of prefixes received/advertised (often small, e.g., loopbacks)
- For overlay EVPN: EVPN peers established, correct AFI/SAFI

Implement checks using Ansible network command modules:
- IOS-XE: show bgp ipv4 unicast summary, show bgp l2vpn evpn summary (platform-dependent)
- IOS-XR: show bgp ipv4 unicast summary, show bgp l2vpn evpn summary

Route verification (don’t skip it)

Session-up isn’t enough. Validate:
- Loopback reachability (underlay)
- Presence of expected routes in RIB/FIB
- No unexpected default route leaks (policy correctness)
- EVPN route types if using overlay (Type-2/Type-5 as applicable)

A practical approach:
- Define per-role expectations:
- leaves should see all spine loopbacks
- spines should see all leaf loopbacks
- Keep thresholds flexible but explicit (e.g., “>= N neighbors established” is not enough; match exact neighbor list from netbox intent).

Feeding operational state back into NetBox (and alerting on mismatches)

NetBox should remain the intent source, but it can also store derived operational info for visibility.

Options:
- NetBox custom fields for last validation status (bgp_last_ok, bgp_last_checked)
- NetBox journal entries for change logs and validation failures
- Tags like bgp-mismatch applied automatically when drift is detected
- Webhooks from automation pipeline to incident/alerting systems (Slack/email/Prometheus Alertmanager/etc.)

Reconciliation rule of thumb:
- If the device matches intent but intent is “wrong”, fix netbox.
- If intent is right but device drift exists, fix config via automation.
- If ambiguity exists, fail closed: don’t push changes until the data model is corrected.


Common pitfalls

  • Unmodeled cabling/topology in NetBox: without cables (or a reliable adjacency model), neighbor discovery becomes fragile.
  • Storing raw route-map bodies in netbox: difficult to validate and easy to drift; store references and render from templates.
  • Non-deterministic template output: unsorted neighbor lists cause perpetual diffs and churn.
  • Mixing underlay and overlay intent fields without clear scoping (VRF/AFI/SAFI): leads to applying policies in the wrong context.
  • No rollback plan for IOS/IOS-XE: treat “no commit” platforms as requiring extra safety (checkpoints, staged rollout, strict pre-checks).
  • Secrets in netbox custom fields: don’t store passwords directly; store secret references and pull from a vault system.
  • Relying on “whatever is on the device” as truth: makes drift permanent and breaks repeatability.

Key takeaways

  • Use netbox to model intent (devices, interfaces, IPs, VRFs, peer-group references), not raw vendor config.
  • Normalize BGP intent into a clean data structure once, then render OS-specific templates.
  • Build Ansible around a safe workflow: pre-checks, deterministic render, diff, staged deploy, rollback, post-validation.
  • Validate more than BGP session state: verify routes and policies, then reconcile drift explicitly.
  • Keep the schema small and stable; automation improves as your intent model becomes consistent.