Cisco SD-WAN for ENCOR and ENSLD Candidates
I recently completed my CCNP Enterprise Certification by taking the 350-401 Enterprise Core and 300-420 Enterprise Design exams. Both of the blueprints for these exams included topics on Cisco’s SD-WAN solution. In this article I’d like to cover Cisco SD-WAN for candidates looking to take either of those exams. Join me!
For those of you that need a primer on the topics of WAN and SD-WAN I invite you to checkout my friend, and podcast co-host, Aaron Engineered‘s blog on these subjects:
https://www.aaronengineered.com/blog/wan-for-dummies
http://aaronengineered.com/blog/sdwan-for-dummies
A quick word…
This blog article is not exhaustive. Do not consider this the end all be all or only resource you should use to understand SD-WAN. I think I do a great job here, but the ENCOR and Design exams will morph overtime. So depending on when you take the exam you may get/have a different experience than I did.
You will not see any screen shots of vManage or demos or configuration because as of this writing you’re not asked to do that on either the 350-401 ENCOR or the 300-420 Enterprise Design exams.
The 3 Planes of Networking
Almost sounds like the Four Horsemen of the Apocalypse but it’s much better than that, I promise. This is important to understand because SD-WAN changes the game using these three planes, Management, Control, and Data.
In short – the management plane is where process are managed, polices created and implemented. The Control plane is where things like routing protocols live, the IP Routing table is formed, it’s also responsible for sending and receiving traffic relating to routing like forming neighbor-ships and routing protocol updates. Lastly, the data plane is the raw forwarding of traffic it receives that transits the router.
The Big Picture
What makes SD-WAN different is that the transport does not matter. MPLS, Cable, DSL, DIA, fiber, copper, wireless, LTE – it doesn’t matter.
Why doesn’t it matter? Well let’s go back to those 3 planes of networking, Management, Control, and Data.
vBond – this is the Orchestrator and should be reachable by a public DNS entry. So, when new devices are added to your SD-WAN domain all you need to do is tell it about your vBond, and once it checks in vBond will take care of the rest. vBond will tell a new vEdge, for example, how to reach vSmart Controllers and vManage.
vManage is the Management plane of SD-WAN. As the Network Admin/Operator/Engineer you interact with SD-WAN via vMange GUI or API – that’s right, no. CLI here. That’s not to say you can’t SSH into the router, because you can, but don’t expect to be able to modify the configuration from there once the router is connected to vManage. The GUI and API are the new defacto methods for managing your SD-WAN routers.
vSmart Controllers are in the Control Plane. They communicate with vManage and the Edges. They use a routing protocol made specifically for SD-WAN called Overlay Management Protocol, or OMP. The Edges use OMP to communicate route information with the vSmart Controllers, not each other. Then, based on policies created on the vManage, the vSmart controller will “leak” routes via OMP back to the Edges. vSmart controller utilizes DTLS to securely communicate with other devices.
vEdges and cEdges – these are the data plane. They are the routers sitting in Data Centers, Branch locations, colo’s, moving data through the SD-WAN fabric. Edges use IPsec and GRE tunnels to connect to each other.
When the devices come online and report to the vBond the vBond simply keeps track of all of the devices in the SD-WAN domain and informs them about the others within. For this reason, the vBond is the only device that needs to be reachable directly on the internet and not NAT’d behind a firewall or other device. It always needs to be reachable as devices can come online at any time. The vBond is a rather basic device, comparatively speaking. It’s just a vEdge configured as a vBond.
SD-WAN Security
As mentioned previously Cisco SD-WAN utilizes DTLS for secure communication throughout the fabric. DTLS and TLS are certificate-based forms of authentication. If you’re using Cisco SD-WAN hosted controllers they will always be equipped with appropriate certificates as will the Edges devices. However, if you choose to host your own SD-WAN environment on-premises or in your own private/public cloud, or build a lab, you’ll need to manage your own certificates through an Enterprise Certificate Authority (CA).
More on OMP
OMP runs over TLS and DTLS tunnels. OMP is used to securely communicate route and related information between vSmart Controllers and Edges in the Cisco SD-WAN fabric. Besides route information, vSmart controllers all exchange crypto keys used to secured communications and policy information all via OMP.
There are three different kinds of routes advertised by OMP:
OMP Routes – These are routes advertised by the local Edge devices and includes static routes, OSPF, and now EIGRP, and BGP routing protocols. EIGRP functionality in Cisco SD-WAN is still relatively new.
TLOC Routes – TLOC is short for Tunnel Location, and are logical tunnel termination points on Edge devices located throughout the Secure SD-WAN fabric underlay.
Services Routes – These are routes to services such as firewall, VPN, application optimization, web filtering, and more.
OMP Route Attributes
Like other routing protocols OMP has many different attributes. The engineers that designed OMP have likened it to BGP. Here are the different attributes of OMP Routes:
TLOC – this is used to identify the next hop for the OMP route and is very similar to the BGP NEXT_HOP attribute. More on TLOCs in a little bit.
Origin – this is the source of the original route. Was it static, OSPF, EIGRP, BGP, etc.
Originator – the identiy of the Edge device that originated the route.
Preference – this is the preference for one OMP route over another. Like most other things in Cisco land a higher preference is more preferred.
Site ID – this is the site ID of the site as it is defined in vManage.
VPN – this identifies the VPN that the route belongs to. When it comes to Cisco SD-WAN think of VPN more like a VRF.
Additionally, you can use an optional path attribute called a Tag. A Tag is used by an OMP router to control routing info that it accepts, redistributes, etc.
TLOC Routes
TLOC Routes contain some specific information about the transport.
Private Address – the private (RFC 1918 space) address of the Edge.
Public Address – this is the NAT translated address.
Carrier – is this a public transport like DIA, Broadband, LTE, or a private transport like MPLS.
Color – There are many different colors in SD-WAN. Some are pre-defined and some are user define-able. The colors identify the route.
Encapsulation type – whether the encapsulation is GRE or IPsec.
Weight – This is a tie breaker of sorts for routes reachable through two, or more, TLOCs.
Just like OMP Routes, TLOC Routes can also use Tags.
Provisioning
One of the major benefits of SD-WAN is the deployment of new devices. There are two options for deployment and onboarding of new Edges: Manual and Zero Touch Provisioning. ZTP allows anyone, literally anyone, can deploy a router with ZTP. Just power and a network cable, plugged into the correct port, and ZTP does the rest. This means you don’t have to send someone from IT, or hire expensive contractors, to do a new site turn up.
Despite the name there is some initial setup required for ZTP. But, you only have to do it once, or unless you move to a different vBond.
When you first buy SD-WAN devices and licenses you’ll be provisioned a Tenant that’s associates with your Smart Account. Devices will be placed in the plug and play (PnP) portal located at software.cisco.com. You upload information about your vBond into the PnP which will produce a provisioning file that you then take and upload to your designated vManage.
ZTP
ZTP has a few minimum requirements. The Edge devices must be able to reach the public internet and preferably Google Public DNS of 8.8.8.8 and 8.8.4.4. Additionally the connection should have DHCP address, zero configuration needs to be done to the Router.
Depending on the device you have you will need to use specific interfaces for WAN under ZTP. Check production documentation for your vEdge or cEdge.
In Cisco SD-WAN it’s all about templates. Templates are devices specific. So, before you can provision a device you must go into vManage and build templates for the device models you have purchased.
Once you have these things in place you can plug in and power up your Edge devices. They’ll reach out and be dropped into your tenant and then get provisioned. You can then assign profiles to them, like if it’s a Data Center device or a Branch device. You can deploy various overlays and security policies.
Manual Provisioning
Manual provisioning will require you to configure some information onto the device. This info includes basic interface IP information so the router has connectivity, including a default gateway or route, DNS servers, the vBond hostname, the Organization name, system IP address, and site ID. A word on the Org Name. This is case sensitive and MUST match how it is defined on the vManage servers. If it is off even the slightest then manual provisioning will not work.
Bootstrapping IOS-XE SD-WAN Devices
One additional option for deployment of Cisco IOS-XE SD-WAN devices is called Bootstrapping. This is the process of putting a basic configuration file onto a USB drive. The file named ciscosdwan.cfg is generated by vManage. You download the file from vManage, copy it to a USB drive and insert into the router prior to power up. The boot process will look for the ciscosdwan.cfg file on a USB drive and use it if it finds it. More on the bootstrap process here.
More on Security
So previously mentioned we talked about security between the planes, now let’s talk about securing Cisco SD-WAN.
vManage uses RBAC (Role Based Authentication) and out of the box has three main user groups:
Basic – view interface and device information on Edges
Operator – view information through Cisco SD-WAN, including on vManage
Netadmin – Full control, read and write, everywhere.
Additional groups can be created and customized as the organization see fit.
Users can be authenticated via AAA, using RADIUS, TACACS+, or SAML SSO, so SD-WAN administrators can continue to use their Enterprise credentials to authenticate and administer the environment.
To further protect vManage a list of allowed IP addresses can be configured so only those IPs can be allowed access to your vManage. If you’re using Cisco hosted controllers you’ll need to work with TAC to configure this list.
SD-WAN Design
Let’s change gears a little bit and discuss design.
Controllers
The biggest design consideration is scale, how many vEdges/cEdges will be in your SD-WAN environment. If you need more capacity to on-board lots and lots of Edges then you’ll need additional vBonds and vSmart Controllers. vManage can be clustered for High Availability, but as of this writing it has it’s limitations. For example, you can only stretch vManage clusters across data centers that have less than 5ms round trip time between them.
High Availability
HA can be find at in many different ports of the SD-WAN solution, from the data center, WAN connectivity, and even on the WAN side.
Multiple SD-WAN Edges can be configured per site and weights are used to prioritize which device is designated as primary and secondary. You can also dedicate devices to certain transports, for example one could be dedicated to connect to your MPLS cloud and the other can be connected directly to the internet.
On the LAN side Edge devices can use VRRP to protect the gateway IP address, or you can peer with another L3 device using BGP, OSPF, and now EIGRP.
VPN Topologies
These are the overlays that you can apply within your Cisco SD-WAN network. Traditionally speaking, this was previously a decision that was made from the beginning and then you just lived with it, until you decided to change it. Will the WAN be hub and spoke, full mesh or some combination in between. Well with SD-WAN you can have these different topologies co-exist and be reserved for different types of traffic.
For example you could deploy a full mesh VPN for voice/collab applications to provided the lowest latency between sites. Then you could have a hub and spoke topology for internally hosted resources since traffic would need to flow through a centralized data center. You could also do regional hubs for larger networks that have regional data centers.
Application Aware Routing
We can’t come all this way and NOT discuss application aware routing. This, in my opinion, is one of the many reasons that the Cisco SD-WAN solution stands apart from others, and is a key part of SD-WAN, period.
Application aware routing is the ability to identify application traffic and route it accordingly. Rules can be created based on the application traffic. For example, you can configure Application Aware Routing policies that say any traffic destined to SaaS will exit the branch via the local internet connection and go directly to that SaaS destination. And why not? The traffic is trusted, its probably secured, so why send it back through a centralized point when you can just send it directly to it’s destination and provide what is likely the best past for it.
Within Cisco SD-WAN you can natively select from a list of applications and create rules right within vManage. If you’re using a home grown app you can even create custom rules to help the Edges identify the traffic and deal with it accordingly.
Resources
If you want more Cisco SD-WAN, and by now I’m sure you do, here is the only link you’ll ever need:
Someone has already taken the time to curate a very comprehensive list of SD-WAN related resources. From Cisco Live On-demand presentations, to Cisco Verified Designs for SD-WAN, links to downloads, it’s all here. In fact, I know I’ve said it before, if you’re starting in on any new Cisco topic I always recommend checking Cisco Live On Demand presentations. I have yet to be disappointed. It’s a very good, and very free, resource.
Additionally, here is a full list of resources I used to pass the ENCOR and ENSLD exams
350-401 Official Cert Guide
300-420 Official Cert Guide
Pluralsight Cisco ENSLD Course – Absolutely great design course on there that was extremely helpful. In fact, between the book and that course was all I needed. If you’re interested in a Pluralsight subscription you can use my referral code to save on either your first month or first year
Cisco Live On Demand – search for DNA Center, LISP, VxLAN
Labs, lots of labs – while the exam I got did not have any configuration examples or sims, labbing this stuff out just gave me a much better understanding.
Cisco DevNet – For all the API and Automation topics
If you’re looking for a study partner or group don’t forget to check out the Discord group – It’s All about the Journey! Lots of great stuff happening there. One of our members just did a great class on Cisco’s SD-Access for the CCNP study group, and we just did our first video happy hour too, where members shared their stories on how they broke into IT.
I hope you found this useful. If you have any questions please do not hesitate to reach out to me!
No Blinky Blinky 