[Part 8] Cisco SDWAN - Overlay Management Protocol

OMP Routes | TLOC Routes | Service Routes

·

16 min read

[Part 8] Cisco SDWAN - Overlay Management Protocol
Play this article

Overview

In the Cisco SD-WAN solution, the Overlay Management Protocol (OMP) serves as the routing protocol. However, OMP goes beyond just routing and provides several essential services within the control plane:

  • Facilitation of network communication: OMP enables data plane connectivity between sites in the SD-WAN fabric, including service chaining and multi-VPN topology information.

  • Distribution of data plane security information: OMP handles the distribution of encryption keys, ensuring secure communication within the fabric.

  • Best-path selection and routing policy advertisement: OMP determines the optimal paths for data traffic and communicates routing policies across the network.

    Read more

    [Part 6] Cisco SDWAN - vSmart Controller

    [Part 7] Cisco SDWAN - Control Plane Operations - OMP

WAN Edge routers will establish the OMP peering session with all of vSmart by default. The OMP update is protected via DTLS or TLS sessions.

The below example shows the OMP peers on WAN Edges

vEdge11 (Viptela OS) "show omp peers"

vEdge11# show omp peers

R -> routes received
I -> routes installed
S -> routes sent

                 DOMAIN   OVERLAY  SITE
PEER       TYPE    ID      ID       ID   STATE  UPTIME     R/I/S
---------------------------------------------------------------
9.9.9.30   vsmart  1       1        99    up   0:10:16:52   2/2/2

cEdge41 (IOS-XE) "show sdwan omp peers"

cEdge41#show sdwan omp peers

R -> routes received
I -> routes installed
S -> routes sent


                 DOMAIN  OVERLAY  SITE
PEER      TYPE   ID     ID        ID    STATE  UPTIME      R/I/S
---------------------------------------------------------------
9.9.9.30 vsmart  1      1         99    up    6:03:33:18    1/1/2

The below output shows the OMP peers from vSmart to WAN Edges.

vSmart1# show omp peers

R -> routes received
I -> routes installed
S -> routes sent

                  DOMAIN  OVERLAY SITE
PEER       TYPE   ID      ID      ID    STATE  UPTIME      R/I/S
--------------------------------------------------------------
1.1.1.11  vedge   1       1       10     up   0:10:20:15   2/0/2
1.1.1.40  vedge   1       1       40     up   6:08:41:00   2/0/1

OMP in the Cisco SD-WAN solution is responsible for advertising different types of routes between the vSmart controllers and WAN Edge routers. These routes include:

  • OMP routes (vRoutes): These routes represent network prefixes that enable connectivity services for data centers, branch offices, or any other endpoint within the SD-WAN fabric.

  • TLOC Routes: TLOCs serve as identifiers that associate an OMP route with a physical location. They are the only IP addresses that are known and reachable within the underlay network.

  • Service routes: Service routes identify network services within the SD-WAN overlay. These routes indicate the physical location of services such as firewalls, IPS, IDS, or any other device capable of processing network traffic. Service information is advertised through service routes and OMP routes.

1. OMP Routes (vRoutes)

In an SD-WAN setup, each WAN Edge device located at a site communicates with the central vSmart controllers by sharing information about the routes it can handle. These updates are similar to the way traditional routing updates work. They provide information about which network addresses (prefixes) the WAN Edge device can reach.

The Overlay Management Protocol (OMP) is responsible for sending these route updates to the vSmart controllers. OMP can advertise different types of routes, including routes that are directly connected to the WAN Edge device, static routes that are manually configured, and routes learned from traditional routing protocols like OSPF, EIGRP, and BGP.

In addition to the reachability information, several attributes are also shared when advertising routes. These attributes include:

  • TLOC (Transport Locator)

  • Origin

  • Originator

  • Preference

  • Service

  • Site ID

  • Tag

TLOC (Transport Location)

The TLOC identifier represents the next hop for the OMP route. It is similar to the BGP_NEXT_HOP attribute in BGP routing.

Within the TLOC, there are three values:

  • System IP Address: This is like a unique identifier for the WAN Edge device. It doesn't necessarily need to be a routable IP address but must be unique across all WAN Edges. The system IP address helps identify the original advertising WAN Edge device.

  • Color: This attribute is further explored in the data plane section. It is used to mark a specific WAN connection and can later be utilized to influence policies and topology construction.

  • Encapsulation Type: This value indicates the type of encapsulation used for the data plane tunnel, such as IPsec or GRE. It is advertised to specify the type of tunneling protocol being employed.

Origin

The origin attribute specifies the source of the route. As the route is advertised in the routing domain, the original source of the route is included in the update.

The source may indicate an identifier like BGP, OSPF, EIGRP, Connected, or Static, along with the protocol's original metric.

The origin attribute is also considered in the best-path selection for OMP routes. Similar to other attributes, it can be configured to influence how this information is used in policies and routing decisions.

Originator

The Originator attribute identifies the original source from which the route was learned. It is represented by the system IP address of the device that advertised the route. The network administrator can use this attribute to create policies that consider the route's origin.

Preference

The Preference attribute, sometimes called OMP Preference (not to be confused with TLOC Preference), can be modified to influence the criteria for selecting the best path for a route. Routes with higher preference values are preferred over those with lower values. This attribute operates similarly to LOCAL_PREF in BGP (Border Gateway Protocol).

Tips: With a larger omp preference value, the higher priority the prefix gets

Service

The Cisco SD-WAN solution supports service insertion, such as firewalls. If a service is associated with a route, it is indicated in the Service attribute. Further details about service routes will be discussed later in the chapter.

Site ID

The Site ID attribute is similar to a BGP autonomous system number (ASN). It is used for policy orchestration and to influence routing decisions. Each site in the network should have a unique Site ID. If there are multiple devices at a site, they should share the same Site ID to prevent routing loops.

Tag

The Tag attribute is optional and transitive. It can be applied to a route by an OMP peer and used in policies. However, when redistributing to or from OMP, the tag is not carried. This attribute functions similarly to a route tag in traditional routing protocols.

VPN

The VPN attribute identifies the specific VPN or VRF (Virtual Routing and Forwarding) from which the route was advertised. VPN tags allow the use of overlapping subnets as long as they belong to different VPNs or VRFs. This enables logical segmentation of the network into multiple data paths and separate routing instances per VPN or VRF. Note that in the Cisco SD-WAN solution, VPNs and VRFs are used interchangeably.

Read more: Cisco SDWAN VPN Segmentation or VRF?

Example of OMP Routes

The below example is the output of the OMP Route which include mentioned attributes and much other information.

vEdge11# show omp routes vpn 100

---------------------------------------------------
omp route entries for vpn 100 route 192.168.10.0/24
---------------------------------------------------
RECEIVED FROM:
peer            0.0.0.0
path-id         69
label           1002
status          C,Red,R
loss-reason     not set
lost-to-peer    not set
lost-to-path-id not set
    Attributes:
     originator       1.1.1.11
     type             installed
     tloc             1.1.1.11, public-internet, ipsec
     ultimate-tloc    not set
     domain-id        not set
     overlay-id        1
     site-id          10
     preference       not set
     tag              not set
     origin-proto     connected
     origin-metric    0
     as-path          not set
     unknown-attr-len not set

ADVERTISED TO:
peer    9.9.9.30

Above 192.168.10.0/24 is a directly connected network belonging to the internal network of vEdge11, so look at the Attributes originator above, it is the system-ip of vEdge11 (1.1.1.11)

Also, at the ADVERTISED TO: peer 9.9.9.30 (vSmart system-ip), it shows this prefix has been distributed to vSmart1 mentioned above.

Regarding the status of prefix 192.168.10.0/24 as output above, (C, Red, R):

  • C: chosen, it means the prefix chosen as the best path

  • Red: redistributed, means the prefix is redistributed from the IGP protocol, connected, ... into OMP. (In this case, look at the origin-proto, the value is connected, and it shows the origin protocol for this prefix is directly connected)

  • R: resolved, means tloc-nexthop available and reachability. (valid)

---------------------------------------------------
omp route entries for vpn 100 route 192.168.40.0/24
---------------------------------------------------

RECEIVED FROM:
peer            9.9.9.30
path-id         291
label           1002
status          C,I,R
loss-reason     not set
lost-to-peer    not set
lost-to-path-id not set
    Attributes:
     originator       1.1.1.40
     type             installed
     tloc             1.1.1.40, public-internet, ipsec
     ultimate-tloc    not set
     domain-id        not set
     overlay-id        1
     site-id          40
     preference       not set
     tag              not set
     origin-proto     connected
     origin-metric    0
     as-path          not set
     unknown-attr-len not set

Above 192.168.40.0/24 is the network prefix that vEdge11 received from vSmart (9.9.9.30), and you can see the originator is 1.1.1.40, the system-ip of cEdge41. Based on it, you can know this prefix comes from the service side of cEdge41.

In addition, the prefix 192.168.40.0/24 contains other attributes such as site-id 40 which tells us that cEdge41's site is 40, and TLOC information is mentioned in the above section. (system-ip, color, encapsulation type).

Regarding the status of prefix 192.168.40.0/24 as output above, (C, I, R):

  • I: installed, it means the prefix has been installed into the routing table.

2. TLOC Route

The TLOC (Transport Location Identifier) routes play a crucial role in identifying the physical location of a device within the transport network. These routes provide addressing that is routable in the underlying network infrastructure and serve as the endpoints for the data plane tunnels.

Simply, it is called the "Next-Hop of OMP Routes" and "WAN Edge's Site-to-Site VPN endpoint".

By using the TLOC with system IP addresses, the SD-WAN solution can maintain consistent and easily identifiable endpoints for data plane tunnels, even if IP addresses change dynamically.

A TLOC consists of three attributes explained above

  • System IP address

  • Transport color

  • Encapsulation type

If a WAN Edge device has multiple transports or interfaces, a separate TLOC route will be advertised for each interface to ensure proper routing and connectivity.

TLOC Color

The Color attribute of the TLOC route is crucial in identifying the transport being used. Each transport should ideally have a different color assigned to it. The Color attribute allows policies to be constructed to influence how the data plane is built within the SD-WAN network.

There are 22 predefined colors available to choose from, and they also define whether the underlying transport is private or public. This distinction determines the IP address that should be used when establishing a data plane tunnel to a remote site.

Note that the default color is not clarified in public or private color.

If no color is explicitly defined, the color "default" is used

PUBLIC COLORPRIVATE COLOR
public-internetmpls
biz-internetprivate1
3gprivate2
lteprivate3
blueprivate4
greenprivate5
redprivate6
bronze
silver
god
custom1
custom2
custom3

By default, WAN Edge devices will attempt to build data plane tunnels to every other site using every available color. (refer to the below Figure)

However, this may lead to inefficient routing, such as MPLS sites attempting to build tunnels to public Internet sites unintentionally.

To control this behavior, the "restrict" command and/or tunnel groups can be used. These mechanisms allow for fine-tuning of the data plane connectivity and routing decisions. Further details about this will be discussed in the data plane articles later.

When a TLOC route is advertised, it contains the following information:

  • TLOC private address: This attribute represents the private IP address derived from the physical interface of the WAN Edge device.

  • TLOC public address: The WAN Edge device receives notification via STUN (Session Traversal Utilities for NAT) that it may be behind a NAT (Network Address Translation) device. This attribute contains the publicly routable or outside IP address assigned to the WAN Edge, enabling data plane connectivity across a NAT boundary. If the public and private addresses match in a TLOC route, it indicates that the device is not behind a NAT.

  • Color: This attribute corresponds to the defined color of the transport being used. The list of colors is mentioned in the above table. If no color is explicitly defined, the color "default" is used.

  • Encapsulation type: This attribute specifies the type of tunnel encapsulation used for the data plane. The available options are IPsec and GRE. Both sides of the tunnel must match in terms of the encapsulation type for data plane connectivity.

  • Preference: Similar to OMP Preference, this attribute allows the network administrator to indicate a preference for one TLOC over another when comparing the same OMP route.

  • Site ID: This value identifies the originator of the TLOC route and is used to control how data plane tunnels are established.

  • Tag: Similar to route tags and OMP tags, this attribute allows the definition of a value that can control how prefixes are exchanged and influence the flow of traffic.

  • Weight: This attribute functions similarly to BGP Weight. It is a path selection method, locally significant, and indicates a preference for one path over another. A higher weight value is preferred over a lower one.

These attributes provide detailed information and control over the TLOC routes, allowing for efficient data plane connectivity and routing decisions within the SD-WAN network.

3. Service Routes

Service routes are used to advertise specific services within the SD-WAN overlay network. Service chaining policies can then be applied to direct data traffic through one or more of these services before reaching its original destination.

Service chaining is beneficial when certain data traffic needs to pass through additional security or optimization services, such as firewalls, load balancers, or IDPs (Intrusion Detection and Prevention systems). These services can be applied on a per-VPN basis, allowing for granular control and flexibility.

For example, network services will be deployed according to the Hub-Spokes scheme, and services are connected at the central site (Hub) so that traffic from other sites is rerouted to the network service site (like Firewall) and then continues to forward to the original destination.

To enable service chaining in the overlay network, the following workflow is typically followed:

  • The network administrator defines the service using a feature template. This template outlines the configuration settings and parameters for the specific service.

  • WAN Edge routers within the network advertise the availability of these services to the central vSmart controllers. Multiple WAN Edge routers can advertise the same service for redundancy purposes.

  • In addition to service advertisements, WAN Edge routers also advertise their OMP (Overlay Management Protocol) and TLOC routes to provide information about reachability and routing.

  • The network administrator then applies a policy that specifies which traffic should flow through the advertised service(s). This policy ensures that the identified traffic is processed by the designated service(s) before being forwarded to its final destination.

It's important to note that for service chaining to work, devices providing the services must be Layer 2 adjacent to the WAN Edge device.

This means there should be no intermediate hops between the WAN Edge device and the service device. Layer 2 adjacency can be achieved using IPsec or GRE tunnels, which facilitate the required connectivity.

In order to offer a service within the SD-WAN network, the site serving as the hub or service provider will advertise a service route using a Subsequent Address Family Identifier (SAFI) in the OMP (Overlay Management Protocol) Network Layer Reachability Information (NLRI). This information is then communicated to the vSmart controller and propagated to the WAN Edge devices. The service route update contains the following details:

  • VPN ID: This attribute specifies the VPN (Virtual Private Network) to which the service applies. It associates the service with a specific VPN within the SD-WAN network.

  • Service ID: The Service ID defines the type of service being advertised. There are seven predefined service types available:

    • FW: Represents a firewall service (mapped to svc-id 1). (above example)

    • IDS: Represents an intrusion detection system service (mapped to svc-id 2).

    • IDP: Represents an Identity Provider service (mapped to svc-id 3).

    • netsvc1, netsvc2, netsvc3, and netsvc4: These service types are reserved for custom services and correspond to the service values of svc-id 4, svc-id 5, svc-id 6, and svc-id 7, respectively.

  • Label: OMP routes that require traffic to flow through this service will have their Label field replaced with this specific label. It helps in identifying the service associated with the route.

  • Originator ID: This attribute represents the system IP address of the node or device advertising the service. It identifies the source of the service advertisement.

  • TLOC: The Transport Location address specifies where the service is located within the network. It identifies the physical location or endpoint associated with the service.

  • Path ID: The Path ID serves as an identifier for the OMP path. It helps in distinguishing and tracking different paths within the SD-WAN network.

Configure Service Routes

Following the above example diagram, the service route will be configured on vEdge3 at site 3.

## vEdge3 ###
vpn 10
  service FW address 3.3.3.3
!

To enforce and drive the traffic between site 1 (LAN-A) and site 2 (LAN-B) through the Firewall (Services) at site 3, the Centralized Policy has to be done via vSmart.

policy
  lists 
    site-list BRANCHES
      site-id 1, 2
  !
  control-policy FIREWALL-SERVICE
    sequence 10
      match route
        site-id 3
      action accept
        set service FW vpn 10
    default-action accept
  !
apply-policy
  site-list BRANCHES control-policy FIREWALL-SERVICE out
!

Tips:

VPN 10 associates the Firewall service with a specific VPN10 within the fabric.

Verify the Service Routes

In order to verify the service route is configured correctly, use "show omp services" on Viptela OS or "show sdwan omp services" on IOS-XE.

vEdge3# show omp services

ADDRESS                                            PATH
FAMILY   VPN    SERVICE   ORIGINATOR   FROM PEER   ID    LABEL  STATUS
--------------------------------------------------------------------
ipv4     10     FW        3.3.3.31      9.9.9.30    52   1006   C,I,R

WRAP-UP

Through the article, we learn about Overlay Management Protocol that is used in the Cisco SDWAN Control Plane.

There are three kinds of routes in OMP:

  • OMP Routes, like BGP prefixes.

  • TLOC Routes provide addressing that is routable in the underlying network infrastructure and serve as the endpoints for the data plane tunnels.

  • Service Routes are used to advertise specific services and the policies can then be applied to direct data traffic through one or more of these services before reaching its original destination.

My name is Nam who loves to talk and share knowledge related to Networking, Automation, and so on. More about me: nam-nguyen.me

Hope you enjoy the blog and don't forget to join the Tech-Learner-Hub to get more and more valuable content.

I really recommend this Cisco SD-WAN Book for the SD-WAN starter:
(This is sponsored by Amazon Program)