Chapter 16: Hybrid Cloud VLAN Integration: AWS, Azure, On-Prem

Introduction

Modern enterprise IT landscapes are increasingly embracing hybrid cloud strategies, leveraging the scalability and flexibility of public clouds like Amazon Web Services (AWS) and Microsoft Azure while retaining critical workloads and data on-premises. A fundamental challenge in these hybrid architectures is the seamless and secure integration of Virtual Local Area Networks (VLANs) from the traditional on-premises environment with the virtualized networking constructs of the cloud.

This chapter is designed to be a comprehensive guide for network engineers navigating the complexities of hybrid cloud VLAN integration. We will delve into the underlying technical concepts, explore multi-vendor configuration examples, demonstrate automation techniques, address critical security considerations, and provide robust troubleshooting methodologies.

After completing this chapter, you will be able to:

• Understand the architectural considerations for extending on-premises VLANs into AWS and Azure.
• Configure various connectivity options, including IPsec VPNs, AWS Direct Connect, and Azure ExpressRoute, for seamless integration.
• Implement network automation solutions using Terraform and Ansible for hybrid cloud networking.
• Apply best practices for securing hybrid cloud connections and segmenting traffic.
• Effectively troubleshoot common issues encountered during hybrid cloud VLAN integration.

Technical Concepts

Integrating on-premises VLANs with cloud environments is primarily a Layer 3 (IP) routing challenge, as public cloud providers do not inherently support Layer 2 (Ethernet) VLAN extension in the same way traditional enterprise networks do. Instead, cloud environments offer virtualized network constructs that achieve similar isolation and segmentation at Layer 3.

1. VLAN Fundamentals (On-Premises Context)

At the core of on-premises network segmentation lies the IEEE 802.1Q standard for VLAN tagging. This standard allows a single physical network infrastructure to be logically segmented into multiple broadcast domains.

• IEEE 802.1Q (Virtual Bridged Local Area Networks): Defines the architecture for VLANs, including the 4-byte VLAN tag inserted into Ethernet frames. This tag includes a 12-bit VLAN Identifier (VID), allowing for 4094 distinct VLANs.
• IEEE 802.1ad (Provider Bridges / QinQ): An amendment to 802.1Q that enables “stacked VLANs” or “QinQ,” where an additional 802.1Q tag (the “S-tag” for Service Provider) is inserted outside the customer’s 802.1Q tag (the “C-tag” for Customer). This is primarily used by service providers to carry customer VLANs across their backbone while keeping them logically separate. While not directly used for on-prem to cloud integration in most enterprise scenarios, it’s relevant if a managed service provider acts as an intermediary.

Reference: IEEE 802.1Q-2022, IEEE 802.1ad

2. Hybrid Cloud Connectivity Models

The bridge between your on-premises VLANs and cloud networks is established through various connectivity models, primarily at Layer 3.

a. Site-to-Site IPsec VPN

This is the most common and often the simplest way to establish secure connectivity. It uses IPsec to encrypt traffic between your on-premises edge device and a cloud VPN Gateway. Your on-premises VLANs’ IP subnets are routed across this encrypted tunnel.

packetdiag {
  colwidth = 32
  0-31: IP Header (Outer)
  32-63: ESP Header
  64-95: IP Header (Inner)
  96-127: TCP Header
  128-255: Data
  256-287: ESP Trailer
  288-319: ESP Authentication
}

Figure 16.1: Generic IPsec ESP Packet Structure

b. Direct Connect (AWS) / ExpressRoute (Azure)

These are dedicated private connections between your on-premises data center and the cloud provider’s network. They offer higher bandwidth, lower latency, and more consistent network performance than VPNs over the public internet. While they are private, the traffic is still Layer 3 (IP) and BGP is used for route exchange.

c. Software-Defined WAN (SD-WAN)

SD-WAN solutions can provide an overlay network that extends from on-premises to cloud, abstracting the underlying transport (IPsec VPNs, Direct Connect, MPLS). They offer centralized management, intelligent path selection, and application-aware routing.

@startuml
!theme mars

' Define all elements first
cloud "AWS Region" as AWS
cloud "Azure Region" as Azure
rectangle "On-Premises Data Center" as OnPrem {
    component "On-Premises Edge Router" as OnPremRouter
    rectangle "On-Prem VLANs" as OnPremVLANs
}

component "AWS Direct Connect / VPN Gateway" as AWSCnx
component "Azure ExpressRoute / VPN Gateway" as AzureCnx

' Then connect them
OnPremVLANs -- OnPremRouter
OnPremRouter -- AWSCnx : Direct Connect / IPsec VPN
OnPremRouter -- AzureCnx : ExpressRoute / IPsec VPN

AWSCnx -- AWS : Virtual Private Cloud (VPC)
AzureCnx -- Azure : Virtual Network (VNet)

AWS -- Azure : Optional: Cloud-to-Cloud Interconnect

@enduml

Figure 16.2: High-Level Hybrid Cloud Connectivity Architecture

3. Cloud Networking Primitives for Integration

Public cloud providers do not use traditional VLANs directly within their virtual networks. Instead, they use virtualized constructs.

a. AWS Networking Concepts

• Virtual Private Cloud (VPC): A logically isolated virtual network where you launch AWS resources.
• Subnets: Divisions within a VPC, analogous to subnets on-premises, associated with an Availability Zone.
• Transit Gateway (TGW): A network transit hub that connects VPCs and on-premises networks through a central gateway. It simplifies network architecture and provides a single point for routing. On-premises VPNs and Direct Connect connections terminate here.
• Direct Connect Gateway (DXGW): An aggregation point for AWS Direct Connect connections, allowing connectivity to multiple VPCs in different regions via Transit Gateways.
• Virtual Private Gateway (VPG): Used for terminating IPsec VPN connections from on-premises and connecting to a single VPC.

nwdiag {
  network "On-Premises Network" {
    address = "10.0.0.0/16"
    router "On-Prem Edge" {
      address = "10.0.0.1";
    }
  }

  network "Direct Connect" {
    description = "Dedicated Circuit"
    router "On-Prem Edge";
    router "AWS DX Gateway" {
      address = "169.254.1.1";
    }
  }

  network "AWS Region" {
    address = "192.168.0.0/16"
    router "AWS DX Gateway";
    router "AWS Transit Gateway" {
      address = "192.168.1.1";
    }
    network "AWS VPC A" {
      address = "192.168.10.0/24"
      router "AWS Transit Gateway";
      device "EC2 Instance A" {
        address = "192.168.10.10";
      }
    }
    network "AWS VPC B" {
      address = "192.168.20.0/24"
      router "AWS Transit Gateway";
      device "EC2 Instance B" {
        address = "192.168.20.10";
      }
    }
  }
}

Figure 16.3: AWS Direct Connect and Transit Gateway Topology

b. Azure Networking Concepts

• Virtual Network (VNet): A logically isolated network in Azure, similar to a VPC.
• Subnets: Divisions within a VNet.
• Virtual Network Gateway (VNG): Provides VPN (IPsec) or ExpressRoute connectivity between VNets and on-premises networks.
• ExpressRoute Circuit: The dedicated private connection for Azure, analogous to AWS Direct Connect.
• Azure Virtual WAN: A unified networking, security, and routing solution that brings many networking, security, and routing functionalities together into a single operational interface. Similar to AWS Transit Gateway, but with more integrated services.
• Azure Route Server: Simplifies dynamic routing (BGP) between your network virtual appliances (NVAs) or VPN/ExpressRoute gateways and the VNet.

OnPrem: On-Premises Data Center {
  shape: rectangle
  router: Edge Router {
    on_prem_lan: On-Prem LAN (10.0.0.0/16)
  }
}

Azure: Azure Cloud {
  shape: cloud
  vnet_hub: Azure Virtual WAN Hub {
    expressroute: ExpressRoute Gateway
    vpn: VPN Gateway
  }
  vnet_spoke1: VNet Spoke 1 (10.10.0.0/16) {
    vm1: VM 1 (10.10.0.10)
  }
  vnet_spoke2: VNet Spoke 2 (10.20.0.0/16) {
    vm2: VM 2 (10.20.0.10)
  }
}

OnPrem.router -> Azure.expressroute: ExpressRoute Circuit
OnPrem.router -> Azure.vpn: IPsec VPN Tunnel

Azure.vnet_hub.expressroute -> Azure.vnet_spoke1: VNet Peering
Azure.vnet_hub.expressroute -> Azure.vnet_spoke2: VNet Peering
Azure.vnet_hub.vpn -> Azure.vnet_spoke1: VNet Peering
Azure.vnet_hub.vpn -> Azure.vnet_spoke2: VNet Peering

Figure 16.4: Azure Virtual WAN Hub Topology

4. IP Address Space Management

A critical aspect of hybrid cloud integration is meticulous IP address planning. Overlapping IP addresses between on-premises and cloud networks will cause routing failures. This necessitates:

• Unique IP Address Spaces: Each on-premises VLAN/subnet and cloud VNet/VPC must have a globally unique, non-overlapping IP address range.
• Route Aggregation: Summarize routes where possible to reduce routing table size, especially with BGP.
• NAT (Network Address Translation): Can be used as a last resort for overlapping IP spaces, but adds complexity and latency.

5. Control Plane vs. Data Plane

• Control Plane: Manages the routing information. In hybrid cloud, this is primarily BGP (Border Gateway Protocol) which exchanges network prefixes between on-premises routers and cloud gateways.
• Data Plane: Carries the actual user traffic. This traffic is encapsulated (e.g., in IPsec tunnels) or transmitted directly over private circuits.

digraph BGP_Route_Exchange {
    rankdir=LR;
    node [shape=box, style="rounded,filled", fillcolor="#F0F4FF", fontname="Arial", fontsize=11];
    edge [color="#555555", arrowsize=0.8];

    OnPrem_Router [label="On-Premises Edge Router"];
    Cloud_GW [label="Cloud VPN/DX/ER Gateway"];
    Cloud_TG_VW [label="Cloud Transit/Virtual WAN"];
    Cloud_VPC_VNET [label="Cloud VPC/VNet"];

    OnPrem_Router -> Cloud_GW [label="BGP Peer (e.g., eBGP)\nAdvertise On-Prem Routes", color="blue"];
    Cloud_GW -> OnPrem_Router [label="BGP Peer\nReceive Cloud Routes", color="green"];

    Cloud_GW -> Cloud_TG_VW [label="Internal Routing/Peering"];
    Cloud_TG_VW -> Cloud_VPC_VNET [label="VPC/VNet Attachment\nRoute Propagation"];

    Cloud_VPC_VNET -> Cloud_TG_VW [label="Advertise VPC/VNet Routes"];
    Cloud_TG_VW -> Cloud_GW [label="Internal Routing/Peering"];
}

Figure 16.5: Hybrid Cloud BGP Control Plane Flow

Configuration Examples

This section provides practical configuration examples for integrating on-premises VLANs (represented by their routed subnets) with AWS and Azure cloud environments using common connectivity methods.

Scenario:

• On-Premises Network: VLAN 10 (10.10.10.0/24), VLAN 20 (10.10.20.0/24)
• AWS VPC: 172.16.0.0/20, Subnet A (172.16.1.0/24)
• Azure VNet: 10.1.0.0/16, Subnet A (10.1.1.0/24)

1. On-Premises Cisco IOS-XE Edge Router (IPsec VPN to AWS)

This configuration sets up an IPsec Site-to-Site VPN from an on-premises Cisco router to an AWS Virtual Private Gateway (VPG). Security Warning: Pre-shared keys should be generated securely and kept confidential. Stronger encryption and hashing algorithms are recommended in production.

! Cisco IOS-XE configuration
! Interface connecting to the Internet
interface GigabitEthernet0/0/1
 ip address 203.0.113.1 255.255.255.252
 no shutdown
! Interface with on-premises VLANs (SVI for routing)
interface Vlan10
 ip address 10.10.10.1 255.255.255.0
 no shutdown
interface Vlan20
 ip address 10.10.20.1 255.255.255.0
 no shutdown

! Define interesting traffic for the VPN tunnel (on-prem to AWS)
ip access-list extended AWS_VPN_TRAFFIC
 permit ip 10.10.10.0 0.0.0.255 172.16.1.0 0.0.0.255
 permit ip 10.10.20.0 0.0.0.255 172.16.1.0 0.0.0.255
! Add other required cloud subnets as needed

! ISAKMP (IKEv1) Policy - Phase 1
crypto isakmp policy 10
  authentication pre-share
  encryption aes-256
  hash sha256
  group 2
  lifetime 86400
crypto isakmp key YourAWSPSK address 52.XX.XX.XX no-xauth ! Replace 52.XX.XX.XX with AWS VPN Public IP

! IPsec Transform Set - Phase 2
crypto ipsec transform-set AWS_TS esp-aes 256 esp-sha256-hmac
 mode tunnel

! IPsec Profile (optional, but good practice for tunnel protection)
crypto ipsec profile AWS_IPSEC_PROFILE
 set transform-set AWS_TS
 set pfs group2

! Crypto Map
crypto map AWS_CM 10 ipsec-isakmp
 set peer 52.XX.XX.XX           ! AWS VPN Public IP
 set transform-set AWS_TS
 match address AWS_VPN_TRAFFIC
 set security-association lifetime seconds 3600
 set pfs group2

! Apply crypto map to the external interface
interface GigabitEthernet0/0/1
 crypto map AWS_CM

! Static route to AWS VPC subnet via the tunnel's next-hop (the peer)
! For redundancy, you'd typically have two tunnels and dynamic routing (BGP)
ip route 172.16.1.0 255.255.255.0 52.XX.XX.XX permanent

! For BGP instead of static routes (more complex, but recommended for production)
! router bgp 65000 (Your AS)
!  neighbor 52.XX.XX.XX remote-as 64512 (AWS AS)
!  neighbor 52.XX.XX.XX ebgp-multihop 255
!  neighbor 52.XX.XX.XX update-source GigabitEthernet0/0/1
!  address-family ipv4
!   network 10.10.10.0 mask 255.255.255.0
!   network 10.10.20.0 mask 255.255.255.0
!   exit-address-family

! Verification Commands:
! show crypto isakmp sa
! show crypto ipsec sa
! show ip route 172.16.1.0
! ping 172.16.1.10 source Vlan10 (assuming 172.16.1.10 is an EC2 instance)

2. AWS CLI Configuration (Corresponding IPsec VPN)

This demonstrates the AWS CLI commands to create the necessary components for the VPN connection. Note: customer-gateway-id, vpn-gateway-id, vpn-connection-id are outputs from previous commands.

# AWS CLI configuration (Conceptual, replace IDs and IPs)

# 1. Create a Customer Gateway (representing your on-premises router)
aws ec2 create-customer-gateway \
    --bgp-asn 65000 \
    --public-ip 203.0.113.1 \
    --type ipsec.1
# Output will include CustomerGatewayId: cg-xxxxxxxxxxxxxxxxx

# 2. Create a Virtual Private Gateway (for the VPC to connect to)
aws ec2 create-vpn-gateway \
    --type ipsec.1 \
    --amazon-side-asn 64512
# Output will include VpnGatewayId: vpg-xxxxxxxxxxxxxxxxx

# 3. Attach the VPG to your VPC (assuming VPC ID: vpc-abcdefg12345)
aws ec2 attach-vpn-gateway \
    --vpn-gateway-id vpg-xxxxxxxxxxxxxxxxx \
    --vpc-id vpc-abcdefg12345

# 4. Create the VPN Connection (Site-to-Site VPN)
aws ec2 create-vpn-connection \
    --customer-gateway-id cg-xxxxxxxxxxxxxxxxx \
    --vpn-gateway-id vpg-xxxxxxxxxxxxxxxxx \
    --type ipsec.1 \
    --options '{"StaticRoutesOnly":true}' \
    --static-routes-only \
    --tunnel-inside-ip-addresses '{"TunnelInsideIpAddress1":"169.254.10.0/30","TunnelInsideIpAddress2":"169.254.11.0/30"}' \
    --tunnel-bgp-asns '{"TunnelBgpAsn1":64512,"TunnelBgpAsn2":64512}' \
    --tunnel-pre-shared-keys '{"TunnelPreSharedKey1":"YourAWSPSK","TunnelPreSharedKey2":"YourAWSPSK2"}'
# Output will include VpnConnectionId: vpn-xxxxxxxxxxxxxxxxx
# You will get two tunnels. Configure your on-prem router for both for redundancy.

# If using BGP: remove --static-routes-only and add --tunnel-bgp-asns

# 5. Enable Route Propagation on your VPC Route Tables (for subnets needing on-prem access)
# Assuming your VPC has a route table rt-xxxxxxxxxxxxxxxxx associated with 172.16.1.0/24
aws ec2 enable-vpn-gateway-route-propagation \
    --route-table-id rt-xxxxxxxxxxxxxxxxx \
    --vpn-gateway-id vpg-xxxxxxxxxxxxxxxxx

# Verification Commands:
# aws ec2 describe-vpn-connections --vpn-connection-ids vpn-xxxxxxxxxxxxxxxxx
# aws ec2 describe-customer-gateways --customer-gateway-ids cg-xxxxxxxxxxxxxxxxx
# aws ec2 describe-vpn-gateways --vpn-gateway-ids vpg-xxxxxxxxxxxxxxxxx
# aws ec2 describe-route-tables --route-table-ids rt-xxxxxxxxxxxxxxxxx

3. On-Premises Juniper JunOS Edge Router (IPsec VPN to Azure)

This configuration sets up an IPsec Site-to-Site VPN from an on-premises Juniper router to an Azure Virtual Network Gateway.

# Juniper JunOS configuration
interfaces {
    ge-0/0/1 {
        unit 0 {
            family inet {
                address 203.0.113.2/30;
            }
        }
    }
    unit 10 {
        vlan-id 10;
        family inet {
            address 10.10.10.1/24;
        }
    }
    unit 20 {
        vlan-id 20;
        family inet {
            address 10.10.20.1/24;
        }
    }
}

routing-options {
    static {
        route 10.1.1.0/24 next-hop st0.0; # Route to Azure VNet via tunnel interface
    }
    router-id 1.1.1.1; # Your router ID
    # For BGP, would configure 'protocols bgp' here.
}

security {
    ike {
        gateway azure-vpn-gw {
            ike-version 2;
            address 52.XX.XX.XX; # Azure VPN Gateway Public IP
            dead-peer-detection {
                interval 10;
                threshold 3;
            }
            local-identity ipv4-only-addr 203.0.113.2; # Your router's public IP
            remote-identity ipv4-only-addr 52.XX.XX.XX;
            external-interface ge-0/0/1.0;
            policy ike-policy;
        }
        policy ike-policy {
            mode main;
            proposals [ ike-proposal ];
            pre-shared-key ascii-text "YourAzurePSK";
        }
        proposal ike-proposal {
            authentication pre-share;
            encryption aes-256-gcm; # Match Azure requirements
            hash sha256;
            dh-group group14; # Match Azure requirements
            lifetime seconds 28800; # Match Azure requirements
        }
    }
    ipsec {
        vpn azure-ipsec-vpn {
            bind-interface st0.0;
            ike {
                gateway azure-vpn-gw;
                ipsec-policy ipsec-policy;
            }
            df-bit clear;
            traffic-selector ts-1 {
                local-ip 10.10.10.0/24;
                local-ip 10.10.20.0/24;
                remote-ip 10.1.1.0/24;
            }
        }
        policy ipsec-policy {
            proposals [ ipsec-proposal ];
            perfect-forward-secrecy {
                group14; # Match Azure requirements
            }
            lifetime seconds 3600; # Match Azure requirements
        }
        proposal ipsec-proposal {
            protocol esp;
            authentication hmac-sha256-96; # Match Azure requirements
            encryption aes-256-gcm; # Match Azure requirements
        }
    }
    zones {
        security-zone untrust {
            host-inbound-traffic {
                system-services {
                    ike;
                }
            }
            interfaces {
                ge-0/0/1.0;
            }
        }
        security-zone trust {
            interfaces {
                vlan.10;
                vlan.20;
            }
        }
        security-zone vpn-zone {
            interfaces {
                st0.0;
            }
        }
    }
    policies {
        from-zone trust to-zone vpn-zone {
            policy allow-trust-vpn {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    permit;
                }
            }
        }
        from-zone vpn-zone to-zone trust {
            policy allow-vpn-trust {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    permit;
                }
            }
        }
    }
}

# Verification Commands:
# show security ike security-associations
# show security ipsec security-associations
# show interfaces st0.0
# show route 10.1.1.0/24
# ping 10.1.1.10 source 10.10.10.1 (assuming 10.1.1.10 is an Azure VM)

4. Azure CLI Configuration (Corresponding IPsec VPN)

This demonstrates the Azure CLI commands to create the necessary components for the VPN connection.

# Azure CLI configuration (Conceptual, replace resource names/IDs)

# Variables
RESOURCE_GROUP="HybridVLANIntegrationRG"
LOCATION="eastus"
VNET_NAME="OnPremVNet"
VNET_CIDR="10.1.0.0/16"
SUBNET_NAME="default"
SUBNET_CIDR="10.1.1.0/24"
GW_SUBNET_NAME="GatewaySubnet" # Required name for gateway subnet
GW_SUBNET_CIDR="10.1.255.0/27" # Minimum /27
VPN_GW_NAME="OnPremVNetGW"
PUBLIC_IP_NAME="OnPremVNetGWPIP"
LOCAL_GW_NAME="OnPremLocalGW"
CONNECTION_NAME="OnPremVNetConnection"
ONPREM_PUBLIC_IP="203.0.113.2" # Your Juniper router public IP
ONPREM_IP_PREFIXES="10.10.10.0/24 10.10.20.0/24"
PSK="YourAzurePSK"

# 1. Create Resource Group
az group create --name $RESOURCE_GROUP --location $LOCATION

# 2. Create Virtual Network
az network vnet create \
    --name $VNET_NAME \
    --resource-group $RESOURCE_GROUP \
    --location $LOCATION \
    --address-prefixes $VNET_CIDR

# 3. Create Gateway Subnet (MUST be named GatewaySubnet)
az network vnet subnet create \
    --name $GW_SUBNET_NAME \
    --address-prefixes $GW_SUBNET_CIDR \
    --vnet-name $VNET_NAME \
    --resource-group $RESOURCE_GROUP

# 4. Create Public IP for VPN Gateway
az network public-ip create \
    --name $PUBLIC_IP_NAME \
    --resource-group $RESOURCE_GROUP \
    --location $LOCATION \
    --allocation-method Dynamic \
    --sku Standard

# 5. Create VPN Gateway
az network vnet-gateway create \
    --name $VPN_GW_NAME \
    --resource-group $RESOURCE_GROUP \
    --location $LOCATION \
    --public-ip-address $PUBLIC_IP_NAME \
    --vnet $VNET_NAME \
    --gateway-type Vpn \
    --vpn-type RouteBased \
    --sku VpnGw1 \
    --no-wait

# Wait for VPN Gateway creation to complete before proceeding

# 6. Create Local Network Gateway (representing your on-premises router)
az network local-gateway create \
    --name $LOCAL_GW_NAME \
    --resource-group $RESOURCE_GROUP \
    --gateway-ip-address $ONPREM_PUBLIC_IP \
    --local-network-address-prefixes $ONPREM_IP_PREFIXES

# 7. Create VPN Connection
az network vnet-gateway connection create \
    --name $CONNECTION_NAME \
    --resource-group $RESOURCE_GROUP \
    --vnet-gateway1 $VPN_GW_NAME \
    --local-gateway2 $LOCAL_GW_NAME \
    --connection-type IPsec \
    --shared-key $PSK \
    --tunnel-connection-mode Default

# Verification Commands:
# az network vnet-gateway show --name $VPN_GW_NAME --resource-group $RESOURCE_GROUP
# az network vnet-gateway connection show --name $CONNECTION_NAME --resource-group $RESOURCE_GROUP
# az network local-gateway show --name $LOCAL_GW_NAME --resource-group $RESOURCE_GROUP
# az network nic show --name vm1-nic --resource-group $RESOURCE_GROUP # Check VM NICs for effective routes

Automation Examples

Automating hybrid cloud VLAN integration streamlines deployments, reduces human error, and ensures consistency. We’ll use Terraform for cloud infrastructure provisioning and Ansible for on-premises network device configuration.

1. Terraform for AWS VPC and VPN Gateway

This Terraform configuration provisions an AWS VPC, a subnet, a Virtual Private Gateway, and a Customer Gateway, and creates the VPN connection.

# main.tf for AWS VPN setup

# Configure AWS Provider
provider "aws" {
  region = "us-east-1"
}

# 1. Create a VPC
resource "aws_vpc" "hybrid_vpc" {
  cidr_block = "172.16.0.0/20"
  tags = {
    Name = "HybridVLAN-VPC"
  }
}

# 2. Create a Subnet
resource "aws_subnet" "app_subnet" {
  vpc_id            = aws_vpc.hybrid_vpc.id
  cidr_block        = "172.16.1.0/24"
  availability_zone = "us-east-1a"
  tags = {
    Name = "HybridVLAN-AppSubnet"
  }
}

# 3. Create a Virtual Private Gateway
resource "aws_vpn_gateway" "onprem_vpg" {
  vpc_id          = aws_vpc.hybrid_vpc.id
  amazon_side_asn = "64512" # AWS AS for BGP
  tags = {
    Name = "OnPrem-VPG"
  }
}

# 4. Create a Customer Gateway (representing your on-premises router)
resource "aws_customer_gateway" "onprem_cgw" {
  bgp_asn    = "65000" # Your on-premises AS
  ip_address = "203.0.113.1" # Your on-premises public IP
  type       = "ipsec.1"
  tags = {
    Name = "OnPrem-CGW"
  }
}

# 5. Create the VPN Connection (Site-to-Site VPN)
resource "aws_vpn_connection" "onprem_vpn" {
  vpn_gateway_id      = aws_vpn_gateway.onprem_vpg.id
  customer_gateway_id = aws_customer_gateway.onprem_cgw.id
  type                = "ipsec.1"
  static_routes_only  = true # Set to false if using BGP
  tunnel1_preshared_key = "YourStrongPresharedKey1"
  tunnel2_preshared_key = "YourStrongPresharedKey2"
  # tunnel1_dpd_timeout_action = "clear" # Optional DPD settings
  # tunnel1_dpd_timeout_seconds = 10 # Optional DPD settings
  tags = {
    Name = "OnPrem-VPN-Connection"
  }
}

# 6. Propagate routes from the VPG to the default route table of the VPC
resource "aws_vpc_main_route_table_association" "default_rt_association" {
  vpc_id         = aws_vpc.hybrid_vpc.id
  route_table_id = aws_vpc.hybrid_vpc.main_route_table_id
}

resource "aws_ec2_transit_gateway_route_table_association" "example" {
  count                  = var.use_transit_gateway ? 1 : 0
  transit_gateway_attachment_id = aws_ec2_transit_gateway_vpc_attachment.example[0].id
  transit_gateway_route_table_id = aws_ec2_transit_gateway.example[0].association_default_route_table_id
}

# Example of enabling route propagation for other route tables (if not using main_route_table_id)
# data "aws_route_table" "main" {
#   vpc_id = aws_vpc.hybrid_vpc.id
#   filter {
#     name = "association.main"
#     values = ["true"]
#   }
# }
# resource "aws_ec2_transit_gateway_route_table_propagation" "vpn_propagation" {
#   route_table_id        = data.aws_route_table.main.id
#   transit_gateway_attachment_id = aws_vpn_gateway.onprem_vpg.id
# }

# Outputs
output "vpn_connection_id" {
  value = aws_vpn_connection.onprem_vpn.id
}
output "tunnel1_outside_ip_address" {
  value = aws_vpn_connection.onprem_vpn.tunnel1_outside_ip_address
}
output "tunnel2_outside_ip_address" {
  value = aws_vpn_connection.onprem_vpn.tunnel2_outside_ip_address
}
output "customer_gateway_config" {
  value     = aws_vpn_connection.onprem_vpn.customer_gateway_configuration
  sensitive = true # Mark as sensitive to prevent PSKs from being printed
}

2. Ansible for Cisco IOS-XE On-Premises Router

This Ansible playbook configures the Cisco IOS-XE router for the IPsec VPN to AWS. It dynamically pulls cloud VPN endpoint IPs and PSKs from Terraform outputs if integrated.

# playbook-cisco-aws-vpn.yaml

- name: Configure Cisco IOS-XE for AWS VPN
  hosts: cisco_routers
  gather_facts: no
  connection: network_cli

  vars:
    aws_tunnel1_ip: ""
    aws_tunnel2_ip: ""
    aws_psk1: ""
    aws_psk2: "" # Assuming two PSKs in output

  tasks:
    - name: Ensure interfaces are configured
      cisco.ios.ios_config:
        lines:
          - "ip access-list extended AWS_VPN_TRAFFIC"
          - "  permit ip 10.10.10.0 0.0.0.255 172.16.1.0 0.0.0.255"
          - "  permit ip 10.10.20.0 0.0.0.255 172.16.1.0 0.0.0.255"
          - "interface GigabitEthernet0/0/1"
          - "  ip address 203.0.113.1 255.255.255.252"
          - "  no shutdown"
          - "interface Vlan10"
          - "  ip address 10.10.10.1 255.255.255.0"
          - "  no shutdown"
          - "interface Vlan20"
          - "  ip address 10.10.20.1 255.255.255.0"
          - "  no shutdown"
        save_when: changed

    - name: Configure ISAKMP (IKEv1) policy
      cisco.ios.ios_config:
        lines:
          - "crypto isakmp policy 10"
          - "  authentication pre-share"
          - "  encryption aes-256"
          - "  hash sha256"
          - "  group 2"
          - "  lifetime 86400"
        save_when: changed

    - name: Configure ISAKMP keys for Tunnel 1
      cisco.ios.ios_config:
        lines:
          - "crypto isakmp key  address  no-xauth"
        save_when: changed

    - name: Configure ISAKMP keys for Tunnel 2
      cisco.ios.ios_config:
        lines:
          - "crypto isakmp key  address  no-xauth"
        save_when: changed

    - name: Configure IPsec transform set
      cisco.ios.ios_config:
        lines:
          - "crypto ipsec transform-set AWS_TS esp-aes 256 esp-sha256-hmac"
          - "  mode tunnel"
        save_when: changed

    - name: Configure Crypto Map for Tunnel 1
      cisco.ios.ios_config:
        lines:
          - "crypto map AWS_CM 10 ipsec-isakmp"
          - "  set peer "
          - "  set transform-set AWS_TS"
          - "  match address AWS_VPN_TRAFFIC"
          - "  set security-association lifetime seconds 3600"
          - "  set pfs group2"
        save_when: changed

    - name: Configure Crypto Map for Tunnel 2 (assuming separate crypto maps for simplicity, or use VTI)
      cisco.ios.ios_config:
        lines:
          - "crypto map AWS_CM 20 ipsec-isakmp" # Use a different sequence number
          - "  set peer "
          - "  set transform-set AWS_TS"
          - "  match address AWS_VPN_TRAFFIC"
          - "  set security-association lifetime seconds 3600"
          - "  set pfs group2"
        save_when: changed

    - name: Apply Crypto Map to external interface
      cisco.ios.ios_config:
        lines:
          - "interface GigabitEthernet0/0/1"
          - "  crypto map AWS_CM"
        save_when: changed

    - name: Configure static routes to AWS VPC
      cisco.ios.ios_config:
        lines:
          - "ip route 172.16.1.0 255.255.255.0  permanent" # Route to AWS via Tunnel 1
          # For redundancy, consider Equal-Cost Multi-Path (ECMP) with two static routes or BGP
        save_when: changed

Note: For Ansible to dynamically pull Terraform outputs, you would typically use a local-exec provisioner in Terraform to run an Ansible playbook, or use dynamic inventory from Terraform state. The hostvars['localhost']['terraform_output'] assumes Terraform output is available to Ansible via some mechanism (e.g., a local file or an orchestrator).

Security Considerations

Security is paramount in hybrid cloud environments, especially when integrating VLANs and IP subnets across disparate infrastructures. Misconfigurations can lead to significant vulnerabilities.

1. Attack Vectors

• VLAN Hopping (On-Premises): While VLANs provide segmentation, improper switch port configurations (e.g., leaving unused ports in trunking mode or default VLAN 1, enabling Dynamic Trunking Protocol (DTP)) can allow an attacker to gain access to unauthorized VLANs.
• VPN Vulnerabilities: Weak IPsec policies, easily guessable pre-shared keys (PSKs), or outdated cryptographic algorithms can expose VPN tunnels to eavesdropping or man-in-the-middle attacks.
• Misconfigured Route Tables/Security Policies: Incorrect routing in either on-premises or cloud environments can lead to unintended access to sensitive resources. Open firewall rules or overly permissive cloud security groups/network security groups (NSGs) are common culprits.
• Identity and Access Management (IAM) Compromise: Compromised cloud IAM roles or on-premises credentials with access to network configurations can lead to full network control.
• Denial of Service (DoS): Overloading VPN tunnels or Direct Connect/ExpressRoute circuits can disrupt connectivity.

2. Mitigation Strategies and Best Practices

• Strict VLAN Configuration (On-Premises):
  • Disable DTP: Manually configure trunk ports (switchport mode trunk) and access ports (switchport mode access).
  • Change Native VLAN: Do not use VLAN 1 as the native VLAN. Assign an unused VLAN ID and prune it from all trunks where it’s not explicitly needed.
  • Prune Unused VLANs: Remove unused VLANs from trunk links to reduce broadcast domains and potential attack surface.
  • Implement Port Security: Limit the number of MAC addresses allowed on an access port.
  • Private VLANs (PVLANs): Use PVLANs for further isolation within a VLAN, preventing devices in the same subnet from communicating directly.
• Strong VPN Cryptography:
  • Use strong, unique, and long pre-shared keys (or certificates for more robust authentication).
  • Enforce modern encryption (e.g., AES-256 GCM), hashing (e.g., SHA-256), and Diffie-Hellman (DH) groups (e.g., Group 14 or higher) in IPsec policies.
  • Regularly review and update cryptographic settings.
• Least Privilege for Network Access:
  • Implement network segmentation based on the principle of least privilege. On-premises, use ACLs on SVIs or firewalls. In the cloud, use Security Groups (AWS), Network Security Groups (Azure), and Azure Firewall/AWS Network Firewall.
  • Only allow necessary ports and protocols between specific source and destination IP ranges.
  • Enforce strong IAM policies for cloud resources and network device access credentials.
• Traffic Inspection:
  • Deploy firewall appliances (physical or virtual) at the on-premises edge and within cloud VPCs/VNets to inspect and filter all traffic crossing the hybrid boundary.
  • Leverage cloud-native firewall services (e.g., AWS Network Firewall, Azure Firewall).
• DDoS Protection:
  • Utilize cloud provider DDoS protection services (e.g., AWS Shield, Azure DDoS Protection) for cloud-facing applications.
• Monitoring and Logging:
  • Implement robust logging for all network devices and cloud networking components.
  • Use network monitoring tools (on-prem) and cloud-native tools (e.g., AWS CloudWatch, Azure Monitor, VPC Flow Logs, NSG Flow Logs) to detect anomalies and potential security incidents.
• Regular Audits:
  • Periodically audit network configurations and security policies in both on-premises and cloud environments.
  • Conduct penetration testing and vulnerability assessments.

3. Security Configuration Examples (Cloud Firewall)

AWS Security Group Example (Allowing On-Premise VLANs)

{
  "Description": "Allow access from on-premises network via VPN",
  "GroupName": "OnPrem-Access-SG",
  "IpPermissions": [
    {
      "IpProtocol": "tcp",
      "FromPort": 22,
      "ToPort": 22,
      "IpRanges": [
        { "CidrIp": "10.10.10.0/24", "Description": "On-prem VLAN 10" },
        { "CidrIp": "10.10.20.0/24", "Description": "On-prem VLAN 20" }
      ]
    },
    {
      "IpProtocol": "tcp",
      "FromPort": 80,
      "ToPort": 80,
      "IpRanges": [
        { "CidrIp": "10.10.10.0/24", "Description": "On-prem VLAN 10" },
        { "CidrIp": "10.10.20.0/24", "Description": "On-prem VLAN 20" }
      ]
    },
    {
      "IpProtocol": "tcp",
      "FromPort": 443,
      "ToPort": 443,
      "IpRanges": [
        { "CidrIp": "10.10.10.0/24", "Description": "On-prem VLAN 10" },
        { "CidrIp": "10.10.20.0/24", "Description": "On-prem VLAN 20" }
      ]
    },
    {
      "IpProtocol": "icmp",
      "FromPort": -1,
      "ToPort": -1,
      "IpRanges": [
        { "CidrIp": "10.10.10.0/24", "Description": "On-prem VLAN 10" },
        { "CidrIp": "10.10.20.0/24", "Description": "On-prem VLAN 20" }
      ]
    }
  ]
}

Azure Network Security Group (NSG) Rule Example (Allowing On-Premise VLANs)

[
  {
    "name": "AllowOnPremSSH",
    "priority": 100,
    "direction": "Inbound",
    "access": "Allow",
    "protocol": "Tcp",
    "sourcePortRange": "*",
    "destinationPortRange": "22",
    "sourceAddressPrefixes": ["10.10.10.0/24", "10.10.20.0/24"],
    "destinationAddressPrefix": "*",
    "description": "Allow SSH from on-premises VLANs"
  },
  {
    "name": "AllowOnPremHTTP",
    "priority": 110,
    "direction": "Inbound",
    "access": "Allow",
    "protocol": "Tcp",
    "sourcePortRange": "*",
    "destinationPortRange": "80",
    "sourceAddressPrefixes": ["10.10.10.0/24", "10.10.20.0/24"],
    "destinationAddressPrefix": "*",
    "description": "Allow HTTP from on-premises VLANs"
  },
  {
    "name": "AllowOnPremHTTPS",
    "priority": 120,
    "direction": "Inbound",
    "access": "Allow",
    "protocol": "Tcp",
    "sourcePortRange": "*",
    "destinationPortRange": "443",
    "sourceAddressPrefixes": ["10.10.10.0/24", "10.10.20.0/24"],
    "destinationAddressPrefix": "*",
    "description": "Allow HTTPS from on-premises VLANs"
  },
  {
    "name": "AllowOnPremICMP",
    "priority": 130,
    "direction": "Inbound",
    "access": "Allow",
    "protocol": "Icmp",
    "sourcePortRange": "*",
    "destinationPortRange": "*",
    "sourceAddressPrefixes": ["10.10.10.0/24", "10.10.20.0/24"],
    "destinationAddressPrefix": "*",
    "description": "Allow ICMP from on-premises VLANs"
  }
]

Verification & Troubleshooting

Effective verification and troubleshooting are crucial for maintaining stable hybrid cloud connectivity. Issues can arise from misconfigurations on either side of the connection (on-premises or cloud) or from underlying network problems.

1. Verification Commands

On-Premises Cisco IOS-XE

# Check interface status and IP addressing for your external interface and SVIs
show ip interface brief

# Verify the IPsec ISAKMP (Phase 1) Security Association (SA)
show crypto isakmp sa

# Verify the IPsec (Phase 2) Security Association (SA)
show crypto ipsec sa

# Check crypto map configuration
show crypto map

# Verify routing table entries for cloud subnets
show ip route <cloud_subnet_ip>

# Test reachability to a cloud instance
ping <cloud_instance_ip> source <on_prem_vlan_interface_ip>
traceroute <cloud_instance_ip>

On-Premises Juniper JunOS

# Check interface status and IP addressing
show interfaces terse

# Verify the IKE (Phase 1) Security Association (SA)
show security ike security-associations

# Verify the IPsec (Phase 2) Security Association (SA)
show security ipsec security-associations

# Check routing table entries for cloud subnets
show route <cloud_subnet_ip>

# Test reachability to a cloud instance
ping <cloud_instance_ip> routing-instance default source <on_prem_vlan_interface_ip>
traceroute <cloud_instance_ip> source <on_prem_vlan_interface_ip>

AWS CLI

# Check VPN Connection status
aws ec2 describe-vpn-connections --vpn-connection-ids vpn-xxxxxxxxxxxxxxxxx --query "VpnConnections[*].State"

# Get detailed VPN connection information, including tunnel status
aws ec2 describe-vpn-connections --vpn-connection-ids vpn-xxxxxxxxxxxxxxxxx

# Check route table propagation status
aws ec2 describe-route-tables --route-table-ids rt-xxxxxxxxxxxxxxxxx --query "RouteTables[*].PropagatingVgws"

# Check effective route table for a specific network interface (e.g., EC2 instance)
aws ec2 describe-network-interfaces --network-interface-ids eni-xxxxxxxxxxxxxxxxx --query "NetworkInterfaces[*].TagSet"
# (Then manually check the associated route table and its entries)

Azure CLI

# Check VPN Gateway Connection status
az network vnet-gateway connection show --name <connection_name> --resource-group <resource_group_name> --query "connectionStatus"

# Get detailed VPN Gateway connection information
az network vnet-gateway connection show --name <connection_name> --resource-group <resource_group_name>

# Check effective routes for a network interface (e.g., VM NIC)
az network nic show-effective-route-table --name <nic_name> --resource-group <resource_group_name> -o table

2. Common Issues and Resolution Steps

Performance Optimization

Optimizing performance in a hybrid cloud environment is about minimizing latency, maximizing throughput, and ensuring consistent service delivery.

• Choose the Right Connectivity:
  • IPsec VPN: Cost-effective for non-critical, lower-bandwidth workloads. Performance can be inconsistent due to internet variability.
  • Direct Connect/ExpressRoute: Essential for high-throughput, low-latency, and consistent performance requirements (e.g., database synchronization, large file transfers, real-time applications).
• Bandwidth Planning:
  • Right-size your Direct Connect/ExpressRoute circuit bandwidth based on current and projected needs. Monitor utilization to prevent saturation.
  • Consider multiple circuits for redundancy and increased aggregate bandwidth.
• Latency Management:
  • Proximity: Deploy cloud resources in regions geographically closest to your on-premises data center.
  • AWS Direct Connect Gateway / Azure Virtual WAN: Use these services to simplify routing and reduce latency by avoiding unnecessary hops.
  • Traffic Engineering: Employ BGP local preference or AS-path prepending to influence traffic paths if you have multiple hybrid connections.
• Jumbo Frames:
  • Enable Jumbo Frames (MTU up to 9001 bytes in AWS, 9000 bytes in Azure) on Direct Connect/ExpressRoute and connected network devices to reduce packet fragmentation and CPU overhead for large data transfers. Ensure end-to-end support.
• Accelerated Networking (Cloud-Native):
  • Leverage cloud-specific features like AWS Enhanced Networking (ENA) or Azure Accelerated Networking for VMs to improve network throughput and reduce CPU utilization.
• Quality of Service (QoS):
  • Implement QoS policies on on-premises edge devices to prioritize critical hybrid cloud traffic (e.g., VoIP, video conferencing) over less critical traffic.
• Monitoring and Baselines:
  • Continuously monitor bandwidth utilization, latency, and packet loss across hybrid connections.
  • Establish performance baselines to quickly identify deviations and potential bottlenecks. Use cloud-native tools (CloudWatch, Azure Monitor) and on-premises NMS.

Hands-On Lab

This lab will guide you through setting up a basic IPsec VPN tunnel between a simulated on-premises router (e.g., a Cisco CSR1000V in a lab environment or a Linux VM with strongSwan) and an AWS VPC.

Lab Topology

nwdiag {
  network "Internet" {
    cloud01;
  }

  network "On-Premises DMZ" {
    address = "203.0.113.0/30"
    router "Cisco_Router" {
      address = "203.0.113.1";
      description = "On-Prem Edge Router";
    }
    cloud01;
  }

  network "On-Premises LAN" {
    address = "10.10.10.0/24"
    router "Cisco_Router";
    device "OnPrem_Host" {
      address = "10.10.10.10";
      description = "Simulated Workstation";
    }
  }

  network "AWS Region (us-east-1)" {
    address = "172.16.0.0/20"
    device "AWS_VPG" {
      address = "52.XX.XX.XX"; # Public IP for Tunnel 1
      description = "Virtual Private Gateway";
    }
    network "AWS VPC Subnet" {
      address = "172.16.1.0/24"
      device "AWS_VPG";
      device "EC2_WebSvr" {
        address = "172.16.1.10";
        description = "Web Server";
      }
    }
  }

  Cisco_Router -- AWS_VPG : "IPsec VPN Tunnel"
}

Figure 16.6: Hands-On Lab Topology (On-Prem to AWS VPN)

Objectives

1. Deploy an AWS VPC, subnet, Virtual Private Gateway, and Customer Gateway.
2. Configure an IPsec Site-to-Site VPN connection in AWS.
3. Extract AWS VPN configuration details.
4. Configure the simulated on-premises Cisco router (or StrongSwan on Linux) to establish the VPN tunnel.
5. Deploy a test EC2 instance in the AWS VPC.
6. Verify end-to-end connectivity from the on-premises host to the AWS EC2 instance.

Step-by-Step Configuration (Using Terraform and Ansible)

Prerequisite:

• AWS Account with CLI configured.
• Terraform installed.
• Ansible installed.
• Simulated Cisco IOS-XE router (e.g., CSR1000V) or a Linux VM with strongSwan for the on-premises edge, accessible via SSH.
• An on-premises public IP address that your simulated router can use.

Step 1: Terraform Deployment of AWS Infrastructure

1. Create main.tf with the Terraform configuration provided in the “Automation Examples” section (AWS VPC and VPN Gateway).
2. Adjust bgp_asn and ip_address in aws_customer_gateway to match your on-premises router’s public IP and ASN.
3. Run:

   terraform init
   terraform plan -out tfplan
   terraform apply "tfplan"
   

4. Note the tunnel1_outside_ip_address, tunnel2_outside_ip_address, and customer_gateway_config from the Terraform output. You’ll need the public IP(s) and PSK(s).

Step 2: On-Premises Cisco Router Configuration (via Ansible)

1. Create an Ansible inventory file (inventory.ini):

   [cisco_routers]
   your_cisco_router_ip_address ansible_user=admin ansible_password=YourPassword ansible_network_os=ios
   

2. Create a group_vars/all.yml file (or provide variables via CLI/Secrets Manager):

   # These values would typically come from a secrets manager or dynamic lookup
   # For this lab, manually paste them from Terraform output.
   terraform_output:
     tunnel1_outside_ip_address:
       value: "52.XX.XX.XX" # Replace with actual AWS VPN Tunnel 1 Public IP
     tunnel2_outside_ip_address:
       value: "52.YY.YY.YY" # Replace with actual AWS VPN Tunnel 2 Public IP
     customer_gateway_config:
       value: |
         # This is a snippet of the full configuration string from AWS.
         # Extract PSK values from here.
         Pre-Shared-Key: YourStrongPresharedKey1
         Pre-Shared-Key: YourStrongPresharedKey2
   

3. Modify the playbook-cisco-aws-vpn.yaml from the “Automation Examples” section. Crucially, ensure you accurately parse the customer_gateway_config to get the PSKs, as the regex example is simplified.
4. Run the Ansible playbook:

   ansible-playbook -i inventory.ini playbook-cisco-aws-vpn.yaml
   

5. Log into your Cisco router and verify the VPN tunnel status using show crypto isakmp sa and show crypto ipsec sa. Both should show “UP”.

Step 3: Deploy and Verify EC2 Instance in AWS

1. Manually or via Terraform, deploy a simple EC2 instance (e.g., Amazon Linux 2) into the 172.16.1.0/24 subnet created by Terraform.
2. Ensure its Security Group allows ICMP and SSH (TCP 22) from your on-premises VLANs (10.10.10.0/24).
3. Log into your OnPrem_Host (10.10.10.10) and try to ping the EC2 instance’s private IP (172.16.1.10):

   ping 172.16.1.10
   

4. Attempt to SSH into the EC2 instance:

   ssh ec2-user@172.16.1.10
   

5. If successful, your hybrid cloud VPN integration is working!

Challenge Exercises

1. Modify the Terraform and Ansible configurations to use BGP for dynamic route exchange instead of static routes.
2. Implement a second VPN tunnel (using tunnel2 details from AWS) and configure it on the Cisco router for redundancy. Verify failover by simulating a tunnel failure.
3. Add an Azure VNet and VPN Gateway (using Terraform) and configure the Juniper router (using Ansible) to create an IPsec VPN to Azure.
4. Configure an AWS Transit Gateway in the Terraform setup and connect your VPC to it.

Best Practices Checklist

Adhering to best practices ensures a robust, secure, and scalable hybrid cloud networking environment.

• IP Address Management:
  • Use a clearly defined and non-overlapping IP addressing scheme for on-premises and all cloud VPCs/VNets.
  • Document all IP ranges and their allocations meticulously.
• Connectivity Strategy:
  • For critical production workloads, prioritize dedicated private connections (AWS Direct Connect/Azure ExpressRoute) over IPsec VPNs.
  • Implement redundant connections (multiple VPN tunnels, multiple Direct Connect/ExpressRoute circuits).
• Network Segmentation:
  • Apply strong segmentation on-premises using VLANs and router ACLs.
  • Leverage cloud-native security groups (AWS) or network security groups (Azure) for micro-segmentation within cloud networks.
  • Deploy virtual firewalls or cloud-native firewall services at the hybrid edge for deeper packet inspection.
• Security Hardening:
  • Use strong, unique, and rotated pre-shared keys or certificate-based authentication for VPNs.
  • Enforce modern cryptographic standards (e.g., AES-256, SHA-256, DH Group 14+) for IPsec.
  • Disable insecure protocols and unused features on network devices.
  • Implement the principle of least privilege for all network access and IAM roles.
  • Regularly patch and update all network hardware and software.
• Automation (NetDevOps):
  • Use Infrastructure as Code (IaC) tools like Terraform for provisioning cloud networking resources.
  • Automate on-premises network device configurations using tools like Ansible.
  • Integrate automation into CI/CD pipelines for consistent deployments and changes.
• Monitoring and Logging:
  • Centralize logging for all on-premises network devices and cloud networking components.
  • Configure alerts for critical events (e.g., VPN tunnel down, high bandwidth utilization, security breaches).
  • Establish performance baselines and monitor for anomalies in latency, throughput, and packet loss.
• Documentation:
  • Maintain up-to-date network diagrams for both on-premises and cloud environments.
  • Document all configurations, IP addressing, security policies, and troubleshooting procedures.
• Change Management:
  • Implement a formal change management process for all network modifications, especially in hybrid environments.
  • Test changes in non-production environments before deploying to production.
• Compliance:
  • Ensure all hybrid network configurations comply with relevant industry regulations (e.g., HIPAA, GDPR, PCI DSS).

Reference Links

• IEEE 802.1Q Standard: https://standards.ieee.org/ieee/802.1Q/10323/
• IEEE 802.1ad (QinQ): https://en.wikipedia.org/wiki/IEEE_802.1ad
• AWS Direct Connect Documentation: https://docs.aws.amazon.com/directconnect/latest/UserGuide/Welcome.html
• AWS Site-to-Site VPN Documentation: https://docs.aws.amazon.com/vpn/latest/s2svpn/VPC_VPN.html
• AWS Transit Gateway Documentation: https://docs.aws.amazon.com/vpc/latest/tgw/what-is-transit-gateway.html
• Azure ExpressRoute Documentation: https://learn.microsoft.com/en-us/azure/expressroute/expressroute-introduction
• Azure VPN Gateway Documentation: https://learn.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-about-vpngateways
• Azure Virtual WAN Documentation: https://learn.microsoft.com/en-us/azure/virtual-wan/virtual-wan-about
• Terraform AWS Provider Documentation: https://registry.terraform.io/providers/hashicorp/aws/latest/docs
• Ansible Network Modules (Cisco IOS): https://docs.ansible.com/ansible/latest/collections/cisco/ios/index.html
• Network Diagram Tools (nwdiag, PlantUML, D2, Graphviz):
  • nwdiag: http://blockdiag.com/en/nwdiag/
  • PlantUML: https://plantuml.com/
  • D2: https://d2lang.com/
  • Graphviz: https://graphviz.org/
• VLAN Security Best Practices: https://www.upguard.com/blog/network-segmentation-best-practices

What’s Next

This chapter has equipped you with a foundational understanding and practical skills for integrating on-premises VLANs (via routed subnets) with AWS and Azure cloud environments. We covered essential technical concepts, detailed configuration examples for multi-vendor environments, and explored the power of automation to streamline deployments. We also emphasized the critical importance of security and provided a structured approach to verification and troubleshooting.

In the next chapter, we will expand on these concepts by diving deeper into Advanced Cloud Routing Architectures. This will include topics such as inter-region connectivity, multi-cloud networking, advanced SD-WAN integration strategies, and optimizing routing for containerized workloads. By mastering these advanced topics, you will be well-prepared to design and manage highly complex and resilient cloud and hybrid network infrastructures.