Chapter 3: Provider Bridging: 802.1ad (QinQ) and Metro Ethernet

Introduction

In the intricate landscape of modern networking, especially within service provider environments and large enterprises, the standard IEEE 802.1Q VLAN often falls short of meeting the demands for extensive customer isolation and flexible service delivery. This is where Provider Bridging, defined by IEEE 802.1ad (commonly known as QinQ or Q-in-Q for “Q-in-Q”), steps in. QinQ allows for the encapsulation of a customer’s 802.1Q tagged frame within another 802.1Q tag, effectively creating a “double-tagged” frame. This mechanism is fundamental to delivering Metro Ethernet services, enabling service providers to extend customer VLANs transparently across their infrastructure while maintaining strict separation between different customers.

This chapter will provide a comprehensive understanding of 802.1ad (QinQ) and its application in Metro Ethernet. We will delve into the technical concepts, architectural implications, and practical configurations across multiple vendors. You will learn how to implement, secure, automate, troubleshoot, and optimize QinQ deployments to build robust and scalable provider networks.

By the end of this chapter, you will be able to:

• Understand the core principles and frame format of IEEE 802.1ad (QinQ).
• Differentiate between Customer VLAN (C-VLAN) and Service VLAN (S-VLAN).
• Design and implement Metro Ethernet services like E-Line, E-LAN, and E-Tree using QinQ.
• Configure QinQ on Cisco, Juniper, and Arista network devices.
• Automate QinQ deployments using Python and Ansible.
• Identify and mitigate security risks associated with QinQ.
• Troubleshoot common QinQ implementation issues.
• Optimize QinQ deployments for performance and scalability.

Technical Concepts

The Need for Provider Bridging

Standard 802.1Q VLANs provide 4094 unique VLAN IDs, which is sufficient for many enterprise networks. However, a service provider offering Layer 2 VPN services to hundreds or thousands of customers, where each customer might use their full range of 4094 VLANs, quickly exhausts this limit. Moreover, customer VLAN IDs must be kept isolated from each other and from the provider’s internal VLANs.

Provider Bridging (802.1ad) addresses this by introducing a second layer of 802.1Q tagging. The customer’s original 802.1Q tag (C-VLAN) is preserved and encapsulated within a service provider’s tag (S-VLAN). This allows the service provider to use a single S-VLAN to transport traffic for a specific customer, regardless of the customer’s internal C-VLANs, across their network.

802.1ad (QinQ) Protocol Specifications

IEEE 802.1ad is an amendment to IEEE 802.1Q that defines “Provider Bridges.” The key mechanism is the addition of a second 802.1Q tag, known as the Service VLAN (S-VLAN) tag, to an Ethernet frame.

• Customer VLAN (C-VLAN): This is the original 802.1Q tag used by the customer within their network.
• Service VLAN (S-VLAN): This is the outer 802.1Q tag added by the service provider’s equipment. It identifies the customer or service within the provider’s network.

When a customer frame, which may or may not already be 802.1Q tagged (C-tagged), enters the provider network, the provider edge device adds an S-tag. When the frame exits the provider network on the customer-facing side, the S-tag is removed. The internal provider network only needs to be aware of the S-VLAN tags for forwarding decisions.

Reference: IEEE Std 802.1ad-2005 (now incorporated into 802.1Q).

QinQ Frame Format

The following packet diagram illustrates the structure of an Ethernet frame with an S-VLAN tag encapsulating a C-VLAN tag.

packetdiag {
  colwidth = 64
  0-47: Destination MAC Address
  48-95: Source MAC Address
  96-111: S-Tag EtherType (0x88A8)
  112-114: S-PCP (Priority)
  115: S-DEI (Drop Eligibility)
  116-127: S-VLAN ID
  128-143: C-Tag EtherType (0x8100)
  144-146: C-PCP (Priority)
  147: C-DEI (Drop Eligibility)
  148-159: C-VLAN ID
  160-175: Length/Type (e.g., IP, ARP)
  176-N: Payload (Customer Data)
  N-N+31: Frame Check Sequence (FCS)
}

Explanation:

• S-Tag EtherType (0x88A8): This indicates that the following tag is a Service VLAN tag. This EtherType differentiates 802.1ad frames from standard 802.1Q (0x8100) frames, allowing devices to properly identify and process the outer tag.
• S-PCP, S-DEI, S-VLAN ID: These fields are for the Service Provider’s VLAN tag.
• C-Tag EtherType (0x8100): This indicates the presence of the Customer VLAN tag.
• C-PCP, C-DEI, C-VLAN ID: These fields are for the Customer’s original VLAN tag.

Architecture and Design: Provider Bridging

Provider bridging involves specific roles for network devices:

• Customer Edge (CE) Device: The customer’s router or switch connected to the provider network. It sends standard 802.1Q or untagged frames.
• Provider Edge (PE) Device: The service provider’s device directly connected to the CE device. It performs the S-tag push and pop operations. It’s often referred to as a “UNI” (User Network Interface) port.
• Provider (P) Device: Core switches or routers within the service provider’s network that forward traffic based on S-VLAN tags. They are unaware of C-VLANs.

nwdiag {
  network Customer_Site_A {
    address = "10.0.1.0/24"
    CE_A [label = "Customer Edge A\n(Router/Switch)"];
  }
  network Customer_Site_B {
    address = "10.0.2.0/24"
    CE_B [label = "Customer Edge B\n(Router/Switch)"];
  }

  network Provider_Network {
    address = "172.16.0.0/20"
    PE_A [label = "Provider Edge A\n(UNI, S-Tag Push/Pop)"];
    P1 [label = "Provider Core Switch 1\n(S-Tag Forwarding)"];
    P2 [label = "Provider Core Switch 2\n(S-Tag Forwarding)"];
    PE_B [label = "Provider Edge B\n(UNI, S-Tag Push/Pop)"];
  }

  // Connections
  CE_A -- PE_A;
  PE_A -- P1;
  P1 -- P2;
  P2 -- PE_B;
  PE_B -- CE_B;

  // Visual separation
  PE_A -- P1 [label = "Internal S-VLANs"];
  P1 -- P2 [label = "Internal S-VLANs"];
  P2 -- PE_B [label = "Internal S-VLANs"];

  CE_A -- PE_A [label = "C-VLANs/Untagged"];
  PE_B -- CE_B [label = "C-VLANs/Untagged"];
}

Metro Ethernet Services with QinQ

QinQ is a foundational technology for delivering various Metro Ethernet Forum (MEF) services:

• E-Line (Ethernet Virtual Private Line - EVPL): Point-to-point service connecting two UNI ports, acting as a dedicated Ethernet link. QinQ is ideal for this, as a single S-VLAN can map to a specific customer’s E-Line service. Customer VLANs are transparently carried.

  @startuml
  !theme mars
  
  ' Define elements
  cloud "Internet" as internet_cloud
  rectangle "Customer A Site 1" as CustA_S1
  rectangle "Customer A Site 2" as CustA_S2
  cloud "Service Provider Core" as SP_Core {
      node "PE Router 1" as PE1
      node "PE Router 2" as PE2
      component "P Switches/Routers" as P_Core
  }
  
  ' Connections for E-Line
  CustA_S1 -[hidden]right-> PE1 : C-VLANs (802.1Q)
  PE1 -[bold]-> P_Core : QinQ (S-VLAN 101)
  P_Core -[bold]-> PE2 : QinQ (S-VLAN 101)
  PE2 -[hidden]left-> CustA_S2 : C-VLANs (802.1Q)
  
  ' Labeling the service
  note right of PE1 : S-Tag 101 pushed for Customer A
  note left of PE2 : S-Tag 101 popped for Customer A
  
  link CustA_S1 to PE1 {
      label="C-VLANs"
      color=blue
  }
  link PE1 to P_Core {
      label="S-VLAN 101"
      color=green
  }
  link P_Core to PE2 {
      label="S-VLAN 101"
      color=green
  }
  link PE2 to CustA_S2 {
      label="C-VLANs"
      color=blue
  }
  
  @enduml
  

• E-LAN (Ethernet Virtual Private LAN - EVPLAN): Point-to-multipoint or multipoint-to-multipoint service connecting multiple UNI ports, creating a shared Ethernet broadcast domain. Multiple S-VLANs or specific S-VLAN policies can segment different customers within the E-LAN.

• E-Tree (Ethernet Virtual Private Tree): A rooted multipoint service where traffic can flow from a root to leaves, but not directly between leaves.

Control Plane vs. Data Plane

• Data Plane: QinQ primarily operates in the data plane by modifying the Ethernet frame header. Provider devices forward traffic based on the outer S-VLAN tag. Original C-VLAN tags are opaque to the provider core.
• Control Plane: The provider’s internal control plane (e.g., STP, routing protocols) operates on the S-VLANs. Customer BPDU (Bridge Protocol Data Unit) frames (from customer STP) are typically tunneled transparently through the provider network, meaning the provider’s STP domain does not interact with the customer’s STP domain. This is achieved through BPDU tunneling mechanisms or by simply pushing an S-tag onto the customer’s STP frames.

State Machines and Workflows

The fundamental workflow for QinQ on a provider edge port (UNI) involves:

1. Ingress (Customer to Provider):
   • An Ethernet frame arrives on the UNI port.
   • The device checks if the frame is already 802.1Q tagged (C-tagged) or untagged.
   • The configured S-VLAN tag (e.g., dot1q tunnel on Cisco) is pushed onto the frame.
   • If the frame was untagged, it may become double-tagged or the C-VLAN ID can be set to a default value (e.g., 0).
   • The double-tagged frame is then forwarded into the provider network based on its S-VLAN ID.
2. Egress (Provider to Customer):
   • A double-tagged frame (S-tagged and C-tagged) arrives on the UNI port from the provider network.
   • The S-VLAN tag is popped (removed).
   • The remaining C-tagged frame (or untagged if it was originally untagged and a default C-VLAN was used) is forwarded out to the customer’s CE device.

digraph QinQ_Traffic_Flow {
    rankdir=LR;
    node [shape=box];

    subgraph cluster_customer_a {
        label="Customer A Network";
        color=lightgrey;
        style=filled;
        CE_A [label="Customer Edge A"];
        Host_A [label="Host A\n(VLAN 10)"];
        Host_A -> CE_A [label="Ethernet Frame\n(C-VLAN 10)"];
    }

    subgraph cluster_provider_network {
        label="Service Provider Network";
        color=lightblue;
        style=filled;
        PE_A [label="Provider Edge A"];
        P_Core [label="Provider Core"];
        PE_B [label="Provider Edge B"];
        PE_A -> P_Core [label="Double-Tagged Frame\n(S-VLAN 100, C-VLAN 10)"];
        P_Core -> PE_B [label="Double-Tagged Frame\n(S-VLAN 100, C-VLAN 10)"];
    }

    subgraph cluster_customer_b {
        label="Customer B Network";
        color=lightgrey;
        style=filled;
        CE_B [label="Customer Edge B"];
        Host_B [label="Host B\n(VLAN 10)"];
        PE_B -> CE_B [label="Ethernet Frame\n(C-VLAN 10)"];
        CE_B -> Host_B [label="Ethernet Frame\n(C-VLAN 10)"];
    }

    CE_A -> PE_A [label="Ingress: S-Tag Push\n(S-VLAN 100)"];
}

Configuration Examples

Provider bridging configurations vary across vendors, but the underlying principle of adding/removing an S-VLAN tag remains consistent. We will demonstrate basic QinQ configurations for Cisco, Juniper, and Arista.

Scenario: Customer A wants to extend their VLANs (e.g., VLAN 10, 20) between two sites over the provider’s network. The provider will use S-VLAN 100 for Customer A’s service.

Cisco IOS XE/IOS Configuration

On Cisco devices, dot1q tunnel is used to configure an interface as a QinQ port (UNI). This command configures the interface to accept all incoming 802.1Q tagged frames and encapsulates them with the specified S-VLAN tag. Untagged frames entering a dot1q tunnel port are typically dropped by default unless explicitly configured with a native VLAN.

! Interface on Provider Edge A connected to Customer Edge A
interface GigabitEthernet0/1
 description Connected to Customer A Site 1
 switchport mode dot1q-tunnel
 switchport access vlan 100
 no cdp enable
 spanning-tree bpdufilter enable
end

! Verification commands
show interfaces GigabitEthernet0/1 switchport
show vlan id 100

Explanation:

• switchport mode dot1q-tunnel: This command enables QinQ tunneling on the interface. It turns the port into a UNI.
• switchport access vlan 100: This assigns the S-VLAN ID 100 to all customer traffic entering this port. All C-VLANs (and untagged frames, depending on specific IOS version and configuration) received on this interface will be encapsulated with S-VLAN 100.
• no cdp enable: It’s a best practice to disable CDP on customer-facing ports to prevent information disclosure.
• spanning-tree bpdufilter enable: Prevents customer BPDUs from interfering with the provider’s STP domain. The customer’s BPDUs are effectively dropped or tunneled transparently (depending on specific configuration and device capabilities).

Security Warning: spanning-tree bpdufilter enable should be used cautiously. It prevents the port from sending or receiving BPDUs, which could lead to STP loops in the customer’s network if not properly designed. Often, spanning-tree bpduguard enable is preferred if loops are to be prevented, or BPDU tunneling (e.g., l2protocol-tunnel) if customer STP needs to function across the provider network.

Juniper JunOS Configuration

On Juniper devices, QinQ is often configured using vlan-tagging and qinq-ether within an ethernet-switching family interface or a dedicated bridge domain. The input-vlan-map push and output-vlan-map pop commands define the tag manipulation.

# Interface on Provider Edge A connected to Customer Edge A
interfaces {
    ge-0/0/1 {
        description "Connected to Customer A Site 1";
        flexible-vlan-tagging;
        encapsulation flexible-ethernet-services;
        unit 0 {
            family ethernet-switching {
                vlan-id-list 1-4094; # Accept all C-VLANs
                input-vlan-map {
                    push;
                    vlan-id 100; # Push S-VLAN 100
                    ether-type 0x88a8; # Specify S-Tag EtherType
                }
                output-vlan-map {
                    pop; # Pop S-VLAN on egress
                }
            }
        }
    }
}

# Add S-VLAN 100 to the provider core facing interfaces/bridge domain
bridge-domains {
    CUSTOMER_A_SERVICE {
        domain-type bridge;
        vlan-id 100; # S-VLAN ID
        interface ge-0/0/1.0; # Customer-facing interface unit
        interface xe-0/1/0.0; # Provider core facing interface unit
    }
}

# Verification commands
show interfaces ge-0/0/1
show bridge domain CUSTOMER_A_SERVICE

Explanation:

• flexible-vlan-tagging and encapsulation flexible-ethernet-services: Enable flexible VLAN processing on the interface.
• unit 0 { family ethernet-switching { ... } }: Configures the logical unit for Layer 2 switching.
• vlan-id-list 1-4094: Configures the UNI port to accept all possible C-VLAN IDs.
• input-vlan-map { push; vlan-id 100; ether-type 0x88a8; }: On ingress, pushes S-VLAN 100 with EtherType 0x88A8 (802.1ad).
• output-vlan-map { pop; }: On egress, removes the outermost S-VLAN tag.
• bridge-domains: Defines the Layer 2 forwarding instance for the S-VLAN.

Arista EOS Configuration

Arista EOS uses a more explicit configuration for VLAN translation/tagging, often leveraging vlan-tag-stacking or service-vlan on trunk ports.

! Interface on Provider Edge A connected to Customer Edge A
interface Ethernet1
  description Connected to Customer A Site 1
  switchport mode trunk
  switchport trunk allowed vlan 100
  switchport vlan-tag-stacking
  switchport dot1q ether-type 0x88a8 # Specify S-Tag EtherType if needed
  service-vlan 100
  no spanning-tree portfast
  spanning-tree bpduguard enable
end

! Verification commands
show interfaces Ethernet1 switchport
show vlan id 100

Explanation:

• switchport mode trunk: The port is configured as a trunk, carrying the S-VLAN.
• switchport trunk allowed vlan 100: Only S-VLAN 100 is allowed on this trunk.
• switchport vlan-tag-stacking: Enables QinQ functionality on the port.
• service-vlan 100: This explicitly defines the S-VLAN tag (100) that will be pushed/popped on this port for customer traffic.
• switchport dot1q ether-type 0x88a8: Configures the EtherType for the outer S-tag.
• spanning-tree bpduguard enable: A common best practice to protect the provider network from customer STP misconfigurations.

Automation Examples

Automating QinQ configurations ensures consistency, reduces human error, and speeds up service deployment.

Ansible Playbook for QinQ Configuration

This Ansible playbook configures a Cisco IOS XE device for QinQ, pushing S-VLAN 100 on GigabitEthernet0/1.

---
- name: Configure QinQ on Cisco IOS XE devices
  hosts: cisco_iosxe_devices
  gather_facts: no
  connection: network_cli
  vars:
    ansible_network_os: ios
    customer_service_vlan: 100
    customer_facing_interface: GigabitEthernet0/1

  tasks:
    - name: Ensure QinQ interface is configured
      ios_config:
        lines:
          - "switchport mode dot1q-tunnel"
          - "switchport access vlan "
          - "no cdp enable"
          - "spanning-tree bpdufilter enable"
        parents: "interface "
        backup: yes
      register: config_output

    - name: Show running config for the interface
      ios_command:
        commands: "show running-config interface "
      register: show_run_interface

    - name: Display configuration changes and running config
      debug:
        msg: |
          Configuration changes applied: 
          Current interface config:
          

Python Script (Netmiko) for QinQ Configuration

This Python script uses Netmiko to connect to a Cisco IOS XE device and configure a QinQ interface.

import os
from netmiko import ConnectHandler
from getpass import getpass

# Device details
DEVICE = {
    "device_type": "cisco_ios",
    "host": "192.168.1.10", # Replace with your device IP
    "username": "admin",
    "password": os.getenv("NETMIKO_PASSWORD") or getpass("Enter password: "),
}

# Configuration parameters
CUSTOMER_SERVICE_VLAN = 100
CUSTOMER_FACING_INTERFACE = "GigabitEthernet0/1"

CONFIG_COMMANDS = [
    f"interface {CUSTOMER_FACING_INTERFACE}",
    "description Connected to Customer A Site 1 (QinQ)",
    "switchport mode dot1q-tunnel",
    f"switchport access vlan {CUSTOMER_SERVICE_VLAN}",
    "no cdp enable",
    "spanning-tree bpdufilter enable",
    "end"
]

VERIFY_COMMANDS = [
    f"show interfaces {CUSTOMER_FACING_INTERFACE} switchport",
    f"show vlan id {CUSTOMER_SERVICE_VLAN}"
]

if __name__ == "__main__":
    net_connect = None
    try:
        print(f"Connecting to {DEVICE['host']}...")
        net_connect = ConnectHandler(**DEVICE)
        print("Connection successful.")

        print(f"\nApplying QinQ configuration to {CUSTOMER_FACING_INTERFACE}...")
        output = net_connect.send_config_set(CONFIG_COMMANDS)
        print("Configuration applied:\n", output)

        print("\nVerifying configuration...")
        for cmd in VERIFY_COMMANDS:
            verify_output = net_connect.send_command(cmd)
            print(f"\n--- Output of '{cmd}' ---\n{verify_output}")

    except Exception as e:
        print(f"An error occurred: {e}")
    finally:
        if net_connect:
            net_connect.disconnect()
            print("Disconnected from device.")

Note: For production use, manage credentials securely (e.g., environment variables, Vault).

Security Considerations

While QinQ enhances service isolation, it also introduces specific security considerations.

• VLAN Hopping Attacks: QinQ inherently mitigates traditional VLAN hopping attacks that rely on a single 802.1Q tag. Since all customer traffic is encapsulated in a provider S-VLAN, an attacker within one customer’s C-VLAN cannot directly “hop” to another customer’s C-VLAN or the provider’s management VLANs simply by manipulating the C-VLAN tag. However, misconfigurations in the QinQ implementation (e.g., incorrect S-VLAN assignment, allowing native VLAN traffic to mix) could still expose vulnerabilities.

  • Mitigation: Ensure strict S-VLAN assignment per customer. Implement strong access control lists (ACLs) or VLAN Access Maps (VACLs) on provider edge interfaces to filter unwanted traffic or control inter-customer communication if E-LAN services are provided.

• MAC Address Table Overflows: Each customer’s MAC addresses are learned within the S-VLAN. A malicious customer could flood the UNI port with a large number of spoofed MAC addresses, potentially overflowing the MAC address table of the PE device or provider core switches. This can lead to traffic being flooded to all ports within the S-VLAN, causing performance degradation and potential information leakage.

  • Mitigation: Implement port-security on UNI ports to limit the number of MAC addresses learned per port. Configure MAC address aging timers.

• BPDU Flooding/Hijacking: If customer BPDUs are tunneled transparently, a misconfigured customer STP instance could generate excessive BPDUs, consuming provider network resources. If BPDU tunneling is not used, and BPDUs are simply forwarded, a malicious customer could attempt to influence the provider’s STP domain (highly unlikely with proper QinQ, but possible with misconfigurations).

  • Mitigation:
    • BPDU Guard: Shuts down the port if a BPDU is received, preventing customer BPDUs from affecting the provider’s STP. (Cisco: spanning-tree bpduguard enable)
    • BPDU Filter: Prevents BPDUs from being sent or received on a port. (Cisco: spanning-tree bpdufilter enable)
    • L2 Protocol Tunneling: Encapsulates and tunnels L2 protocol frames (like BPDUs, CDP, LLDP) transparently across the provider network, allowing customer STP to function end-to-end without impacting the provider’s STP. This is generally the preferred method for EVPL/EVPLAN if customer STP interoperability is required. (Cisco: l2protocol-tunnel cdp, l2protocol-tunnel stp)

• Service Isolation: The primary goal of QinQ is service isolation. Ensure that S-VLANs are unique per customer or service type to prevent unintentional mixing of traffic.

  • Mitigation: Rigorous S-VLAN allocation and documentation. Use automation to enforce S-VLAN assignments.

Security Configuration Example (Cisco):

! Enhanced security for a QinQ UNI port
interface GigabitEthernet0/1
 description Connected to Customer A Site 1 (Secure QinQ)
 switchport mode dot1q-tunnel
 switchport access vlan 100
 no cdp enable
 l2protocol-tunnel cdp
 l2protocol-tunnel stp
 l2protocol-tunnel vtp
 spanning-tree portfast
 spanning-tree bpduguard enable
 switchport port-security
 switchport port-security maximum 50 ! Limit MACs to 50
 switchport port-security violation restrict
 switchport port-security aging time 2
end

Warning: Ensure spanning-tree portfast is enabled only on access ports connected to end devices or in this case, a UNI port that should not cause loops. bpduguard will then protect against unexpected BPDUs. l2protocol-tunnel is generally safer than bpdufilter as it allows customer L2 protocols to function.

Verification & Troubleshooting

Troubleshooting QinQ issues requires a systematic approach, starting from the physical layer and moving up.

Verification Commands

These commands help confirm the QinQ configuration and operational status.

Cisco IOS XE/IOS:

show interfaces GigabitEthernet0/1 switchport
show running-config interface GigabitEthernet0/1
show vlan brief
show mac address-table interface GigabitEthernet0/1

Juniper JunOS:

show interfaces ge-0/0/1 terse
show interfaces ge-0/0/1 detail
show bridge domain CUSTOMER_A_SERVICE
show ethernet-switching table interface ge-0/0/1.0

Arista EOS:

show interfaces Ethernet1 switchport
show interfaces status | include Eth1
show vlan
show mac address-table address interface Ethernet1

Expected Output (Cisco Example)

# show interfaces GigabitEthernet0/1 switchport
Name: Gi0/1
Switchport: Enabled
Administrative Mode: dot1q-tunnel
Operational Mode: dot1q-tunnel
Administrative Trunking Encapsulation: dot1q
Operational Trunking Encapsulation: dot1q
Negotiation of Trunking: Disabled
Access Mode VLAN: 100 (Service_VLAN_Cust_A)  <-- Confirms S-VLAN
Trunking Native Mode VLAN: 1 (default)
...
Protected: false
Host-port: No
...

Troubleshooting Common Issues

Debug Commands (Cisco Example)

Use debug commands cautiously in production as they can impact device performance.

debug ethernet-services interface GigabitEthernet0/1 detail
debug vlan packet

Performance Optimization

Optimizing QinQ deployments ensures efficient use of network resources and reliable service delivery.

• MTU Configuration: As discussed, QinQ adds 4 or 8 bytes to the Ethernet frame. Ensure that the Maximum Transmission Unit (MTU) is appropriately increased across all devices in the provider network that handle QinQ traffic (PE devices, P devices, and inter-switch links). A common practice is to configure jumbo frames (e.g., MTU 9000 bytes) on provider infrastructure to accommodate all overheads.
• Hardware Capabilities: Modern network ASICs (Application-Specific Integrated Circuits) are designed for efficient hardware-based QinQ tagging and forwarding. Ensure that your PE and P devices have sufficient hardware capabilities to process double-tagged frames at line rate without software forwarding.
• VLAN Pruning: Implement VLAN pruning (e.g., VTP pruning for Cisco, GMRP for Generic VLAN Registration Protocol) within the provider network to prevent S-VLAN traffic from being sent over trunks where it’s not needed. This reduces unnecessary broadcast/multicast traffic and optimizes bandwidth.
• Quality of Service (QoS): QinQ supports the preservation of customer’s 802.1p CoS (Class of Service) values (C-PCP). The provider can also remark the S-PCP field based on service agreements. Implement QoS policies to prioritize critical customer traffic within the provider network, particularly if different service tiers are offered (e.g., gold, silver, bronze services).
• Link Aggregation (LAG/EtherChannel): Bundle multiple physical links into a single logical link to increase bandwidth and provide redundancy for S-VLAN traffic, both on UNI links and within the provider core.
• Monitoring and Analytics: Implement robust monitoring solutions to track S-VLAN bandwidth utilization, errors, and latency. Tools that can decode and analyze double-tagged frames are essential for deep visibility into customer traffic.

Hands-On Lab

This lab guides you through configuring a basic E-Line service using QinQ on Cisco IOS XE devices, connecting two customer sites.

Lab Topology

nwdiag {
  network Customer_Site_A_LAN {
    address = "192.168.10.0/24"
    CE_A [label="Customer A Router"];
    Host_A [label="Host A\n(192.168.10.10)"];
    Host_A -- CE_A;
  }

  network Customer_Site_B_LAN {
    address = "192.168.20.0/24"
    CE_B [label="Customer B Router"];
    Host_B [label="Host B\n(192.168.20.10)"];
    Host_B -- CE_B;
  }

  network Provider_Network_Core {
    address = "172.16.0.0/24"
    PE_A [label="Provider Edge A"];
    P1 [label="Provider Core Switch"];
    PE_B [label="Provider Edge B"];
  }

  CE_A -- PE_A [label="GigabitEthernet0/1"];
  PE_A -- P1 [label="GigabitEthernet0/2"];
  P1 -- PE_B [label="GigabitEthernet0/3"];
  PE_B -- CE_B [label="GigabitEthernet0/1"];

  // Annotations
  PE_A -- P1 [label = "S-VLAN 100"];
  P1 -- PE_B [label = "S-VLAN 100"];
  CE_A -- PE_A [label = "C-VLANs (10/20)"];
  PE_B -- CE_B [label = "C-VLANs (10/20)"];
}

Devices:

• CE_A (Customer A Router): Cisco IOS XE (e.g., CSR1000V)
• PE_A (Provider Edge A): Cisco IOS XE (e.g., CSR1000V)
• P1 (Provider Core Switch): Cisco IOS XE (e.g., CSR1000V or 3750/9300 switch)
• PE_B (Provider Edge B): Cisco IOS XE (e.g., CSR1000V)
• CE_B (Customer B Router): Cisco IOS XE (e.g., CSR1000V)
• Host_A, Host_B: Simple end-devices (e.g., PCs, servers, or even loopback interfaces on CE routers)

Assumptions:

• All devices have basic connectivity and management IP addresses.
• Customer A wants to extend VLAN 10 and VLAN 20 from Site A to Site B.
• The provider uses S-VLAN 100 for Customer A’s service.

Objectives

1. Configure CE_A and CE_B to use VLAN 10 and VLAN 20.
2. Configure PE_A and PE_B as QinQ UNI ports for S-VLAN 100.
3. Configure P1 to transparently forward S-VLAN 100.
4. Verify end-to-end connectivity between Host_A and Host_B for both VLANs.

Step-by-Step Configuration

1. Configure Customer Edge Routers (CE_A and CE_B)

CE_A Configuration:

! CE_A
interface GigabitEthernet0/1
 no ip address
 negotiation auto
!
interface GigabitEthernet0/1.10
 encapsulation dot1Q 10
 ip address 192.168.10.1 255.255.255.0
!
interface GigabitEthernet0/1.20
 encapsulation dot1Q 20
 ip address 192.168.20.1 255.255.255.0
!
! (Optional) Host A simulation
interface Loopback0
 ip address 192.168.10.10 255.255.255.0
!
interface Loopback1
 ip address 192.168.20.10 255.255.255.0

CE_B Configuration:

! CE_B
interface GigabitEthernet0/1
 no ip address
 negotiation auto
!
interface GigabitEthernet0/1.10
 encapsulation dot1Q 10
 ip address 192.168.10.2 255.255.255.0
!
interface GigabitEthernet0/1.20
 encapsulation dot1Q 20
 ip address 192.168.20.2 255.255.255.0
!
! (Optional) Host B simulation
interface Loopback0
 ip address 192.168.10.11 255.255.255.0
!
interface Loopback1
 ip address 192.168.20.11 255.255.255.0

2. Configure Provider Edge Routers (PE_A and PE_B)

PE_A Configuration:

! PE_A
! Interface connected to CE_A (UNI port)
interface GigabitEthernet0/1
 description Customer A Site 1 UNI
 switchport mode dot1q-tunnel
 switchport access vlan 100
 no cdp enable
 l2protocol-tunnel cdp
 l2protocol-tunnel stp
 spanning-tree portfast
 spanning-tree bpduguard enable
!
! Interface connected to P1 (NMI port - Network to Network Interface)
interface GigabitEthernet0/2
 description Link to P1
 switchport mode trunk
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 100
 no cdp enable
!
end

PE_B Configuration:

! PE_B
! Interface connected to CE_B (UNI port)
interface GigabitEthernet0/1
 description Customer A Site 2 UNI
 switchport mode dot1q-tunnel
 switchport access vlan 100
 no cdp enable
 l2protocol-tunnel cdp
 l2protocol-tunnel stp
 spanning-tree portfast
 spanning-tree bpduguard enable
!
! Interface connected to P1 (NMI port)
interface GigabitEthernet0/3
 description Link to P1
 switchport mode trunk
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 100
 no cdp enable
!
end

3. Configure Provider Core Switch (P1)

P1 Configuration:

! P1
vlan 100
 name S-VLAN_Customer_A
!
interface GigabitEthernet0/2
 description Link to PE_A
 switchport mode trunk
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 100
 no cdp enable
!
interface GigabitEthernet0/3
 description Link to PE_B
 switchport mode trunk
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 100
 no cdp enable
!
end

Verification Steps

1. Verify QinQ on PE devices: On PE_A and PE_B:

   show interfaces GigabitEthernet0/1 switchport
   show vlan brief
   

   Confirm Operational Mode is dot1q-tunnel and Access Mode VLAN is 100.

2. Verify S-VLAN on P1: On P1:

   show vlan brief
   show interfaces trunk
   

   Confirm VLAN 100 exists and is allowed on both GigabitEthernet0/2 and GigabitEthernet0/3.

3. End-to-End Connectivity (Host A to Host B): From CE_A (or Host A if separate):

   ping 192.168.10.11 source Loopback0  ! For VLAN 10 connectivity
   ping 192.168.20.11 source Loopback1  ! For VLAN 20 connectivity
   

   From CE_B (or Host B if separate):

   ping 192.168.10.10 source Loopback0  ! For VLAN 10 connectivity
   ping 192.168.20.10 source Loopback1  ! For VLAN 20 connectivity
   

   All pings should be successful.

Challenge Exercises

1. Add a new customer (Customer B): Configure another E-Line service for Customer B using S-VLAN 200, extending their VLAN 30 between two new CE devices.
2. Troubleshoot an MTU issue: Intentionally lower the MTU on one of the provider core links (P1 to PE_B) and observe the impact on larger packets (e.g., ping with large data size). Then, correct the MTU and verify.
3. Implement QoS: Configure a simple QoS policy on PE_A to mark customer traffic from C-VLAN 10 with a higher S-PCP value than C-VLAN 20, and verify with packet capture if possible.

Best Practices Checklist

• S-VLAN Allocation: Allocate S-VLANs systematically, dedicating specific ranges or IDs per customer or service. Document all S-VLAN assignments.
• MTU Configuration: Ensure all devices forwarding QinQ traffic (PE and P devices, and inter-device links) have their MTU settings increased to accommodate the additional 4 or 8 bytes.
• BPDU Handling: Use l2protocol-tunnel for transparently forwarding customer L2 control plane traffic (STP, CDP, VTP) where required. Otherwise, use spanning-tree bpduguard enable with spanning-tree portfast on UNI ports to protect the provider network from customer STP. Avoid bpdufilter unless its implications are fully understood.
• Port Security: Implement port-security on UNI ports to limit the number of MAC addresses learned, preventing MAC table overflows.
• Traffic Filtering: Consider applying ACLs or VACLs on UNI ports to enforce specific traffic policies or rate-limit customer traffic.
• CDP/LLDP Disablement: Disable discovery protocols (CDP/LLDP) on customer-facing UNI ports to prevent information leakage.
• VLAN Pruning: Enable VLAN pruning within the provider core to optimize bandwidth utilization by preventing unnecessary S-VLAN traffic from traversing links where it’s not required.
• QoS Implementation: Design and implement QoS policies to preserve customer CoS markings (C-PCP) and apply appropriate queuing and prioritization based on service level agreements (SLAs).
• Redundancy: Design for redundancy using Link Aggregation Groups (LAGs) for UNI connections and within the provider core.
• Automation: Automate the provisioning and modification of QinQ services using tools like Ansible, Python, or network orchestration platforms to minimize manual errors and accelerate deployment.
• Monitoring: Implement comprehensive monitoring for S-VLAN utilization, errors, and performance metrics across the provider network.
• Documentation: Maintain detailed documentation of QinQ designs, configurations, S-VLAN assignments, and service mappings.
• Change Management: Implement a rigorous change management process for QinQ configurations, especially for multi-customer environments.

Reference Links

• IEEE 802.1Q (including 802.1ad): The official standard for VLANs and Provider Bridging. While 802.1ad was a separate amendment, its content is now integrated into the main 802.1Q standard. The latest versions can be found on the IEEE 802.1 working group website.
  • https://1.ieee802.org/
  • https://standards.ieee.org/ieee/802.1Q/10323/
• Metro Ethernet Forum (MEF): Defines standards for Metro Ethernet services.
  • https://mef.net/
• Cisco Dot1q Tunneling:
  • https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3750/software/release/12-2_55_se/configuration/guide/scg/swvlan.html (Example for older IOS, but concepts apply)
• Juniper QinQ Configuration:
  • https://www.juniper.net/documentation/us/en/software/junos/vlan/topics/topic-map/qinq-ethernet-overview.html
• Arista EOS VLAN Tagging:
  • Refer to the Arista EOS documentation for your specific switch model and software version.
• Ansible Network Automation:
  • https://docs.ansible.com/ansible/latest/network/index.html
• Netmiko Documentation:
  • https://github.com/ktbyers/netmiko

What’s Next

This chapter has equipped you with a solid foundation in Provider Bridging (802.1ad QinQ) and its pivotal role in delivering Metro Ethernet services. You’ve learned about the double-tagging mechanism, multi-vendor configurations, automation techniques, critical security considerations, and effective troubleshooting strategies.

In the next chapter, we will expand our discussion to Virtual Extensible LAN (VXLAN). While VLANs and QinQ address Layer 2 segmentation within a confined domain, VXLAN provides a scalable solution for extending Layer 2 networks over a Layer 3 underlay, which is crucial for modern data centers and multi-cloud environments. We will explore VXLAN’s architecture, its benefits over traditional VLANs, configuration examples, and its integration into software-defined networking (SDN) solutions.