In an era of cloud-based services, deployment of SD-WAN services is an increasingly important requirement for SD-WAN Managed Service Providers (MSPs). Enterprises opting for managed services — which constitute the vast majority of installations over the past year — expect smooth, rapid service rollouts and upgrades. Problems experienced during deployment could reframe the overall customer experience.
To address the double-digit SD-WAN growth, MSPs have strived to systematically design and deploy new managed SD-WAN services. While MSPs typically qualify one or two primary SD-WAN vendors, enterprise customers are becoming increasingly specific about which products are best suited to their environments. This has increased the number of vendors and products that MSPs are compelled to support (see Figure 1), as well as their carrying costs. Exacerbating this situation is the proprietary nature of the SD-WAN products and MSPs lack of experience with them.
MEF sought to address these challenges through the introduction of the industry’s first SD-WAN service standard. In the first year post ratification, half of the U.S. SD-WAN MSPs market leaders have become MEF SD-WAN certified. Growth in adoption has also catalyzed new standards activity, including a new standard to facilitate multivendor SD-WAN deployment, MEF SD-WAN Service Readiness Testing (SRT).
SD-WAN SRT specifies performance monitoring and test requirements that are targeted towards enhancing SD-WAN deployment. It is complementary to a similar set of MEF standards, Service Activation Testing (SAT), that have been widely deployed for standard IP and Carrier Ethernet services. Because the standard has yet to be ratified or released as a Draft Standard, the working draft is only available to MEF members.
The intent is to use SRT and SAT in tandem to provide comprehensive verification that an SD-WAN service is performing and functioning as expected, prior to activation. Enterprise end users are particularly sensitive to first impressions and many baulk at serving as a guinea pig for relatively new managed services like SD-WAN. SRT can help build end-user confidence that service instances are thoroughly exercised before trusting business-critical applications to the network.
Figure 1: Security is among the top benefits of using SD-WAN managed services
Source: Managed SD-WAN Services Survey, January, 2021, Heavy Reading
Deployment Challenges
Over the years, Communications Service Providers (CSPs) have accumulated experience working across multiple customers, applications, and environments. Such deep hands-on knowledge has provided the basis for a series of operational best practices that streamline the deployment and maintenance of the network.
SD-WAN is not just another connectivity service; it is the first software-driven, dynamic, multilayer service explicitly intended to support cloud environments. Yet significantly increased complexity, the influence of cloud and the on-demand mindset of many enterprises pose new challenges for MSPs. Enterprises expect new services to be available on demand, highly agile, and devoid of the details.
Because of the relatively short timescales — gone are the days of new services provisioned and forgotten — MSPs have automated their operations to the maximum extent feasible. Managing multilayer networks, however, requires tools capable of rapidly isolating and correlating problems, up and down the stack, to minimize downtime.
A recent survey conducted by HeavyReading points to a number of SD-WAN challenges, including maintaining performance levels over a diverse set of underlays, verification of new services throughout the lifecycle, multilayer troubleshooting and fault isolation, and more. (see Figure 2)
Such challenges accentuate the role of testing throughout the service lifecycle. DevOps, NetOps and SecOps are built upon constant testing. However, the sheer mass of telecommunications infrastructure limits the ability to introduce fully automated DevOps methodologies, even though most operators have strived to automate and orchestrate their service delivery platforms.
The introduction of standard test methods across multiple vendors and multiple WAN technologies with a diverse set of applications are essential to sustain the growth of SD-WAN. As are purpose-built tools to realize the testing.
Service Activation Testing
MEF Service Readiness Testing is based on fundamental concepts introduced by MEF Service Activation Testing standards. SAT was originally released to enhance Carrier Ethernet service deployment and offers service providers a number of benefits:
- Enhance the customer experience by ensuring the service performs as expected prior to activation.
- Verify proper configuration.
- Validate expected performance levels.
- Define a vendor-neutral testing methodology that can be utilized for any conformant equipment.
- Document the results of the deployment-time SAT for compliance.
- Enable testing to be proactively executed (and re-executed) for any change, upgrade, etc.
The MEF SAT standard for Carrier Ethernet (MEF 48.1) was originally introduced in 2014 and has been widely deployed by many MSPs and on vendor equipment. The success of this standard prompted the MEF community to move up a layer and provide the IP Services SAT (MEF 67), introduced in late 2020.
For each SAT standard, MEF defined a common test methodology and distinct test requirements for each service type. Multiple vendors have also implemented MEF CE test suites, most of whom have enhanced their offerings with additional test cases.
MEF Service Readiness Testing
The MEF SRT standard’s project complements SAT by up-leveling the test coverage from Layer 2-3 to the SD-WAN overlay. The standard addresses two distinct areas:
- SD-WAN performance monitoring
- SD-WAN service readiness testing
Figure 2: SD-WAN Managed Service Providers: Major Challenges
Source: Managed SD-WAN Services Survey, January, 2021, Heavy Reading
Each area includes a distinct testing methodology and functional testing requirements that are inherently vendor- and implementation-independent. It is also assumed that the results of the tests may be made available to support a wide range of portals, GUIs, etc.
Performance monitoring enables applications flows to be assessed for latency, delay variation, packet loss, and information rates. While SAT only provides network segment testing at Layer 2, this process allows for end-to-end performance assessments at Layer 3.
The duration of performance testing is intended to enable rapid deployment time testing. The test requirements have been selected to provide adequate coverage in 30 seconds. Threshold Crossing Alerts (TCA) are also supported, providing service providers with an indication when a performance metric is detected outside of its expected range.
The MEF working draft also defines SRT, which provides end-to-end testing over each SD-WAN Underlay Connectivity Service (UCS) or WAN connection and includes test requirements designed to verify that the SD-WAN service is ready to be utilized.
Overcoming SD-WAN Deployment Challenges with SRT/SAT
Standards serve as the catalyst for change but require commitment and investment to yield the tangible solutions that SD-WAN MSPs are seeking.
Multilayer validation is an essential step to integrate and operationalize test methods optimized for each layer of the stack. MEF SRT and SAT offer the basis for a multilayer testing approach capable of exercising the diverse WAN technologies deployed, comprehensive IP protocols, and adjacent technologies active in today’s SD-WAN networks.
While testing plays a central role in streamlining SD-WAN deployments, additional capabilities are required to analyze and correlate the results to rapidly isolate, expose, and mitigate problems. Additional MEF activities are under consideration to facilitate integration of testing as operators migrate to a DevOps-driven infrastructure.
Once the SRT standard is ratified by MEF — and operators gain experience with multilayer testing — we will revisit this area in a future post.
Other posts in the “Insights into MEF’s SD-WAN Standard” series:
- Part One: Overview (July 31, 2019)
- Part Two: Technical Approach (August 14, 2019)
- Part Three: Application Security Project (January 14, 2020)
- Part Four: MEF SD-WAN Certification Program (March 23, 2020)
- Part Five: MEF SD-WAN Certification Turns 1: Lessons Learned from the Industry’s First SD-WAN Standard (Feb 1, 2021)
- Part Six: MEF Jumps on the SASE Bandwagon (March 11, 2021)