Cisco ACI Special Considerations

Cisco ACI Special Considerations

(1) TEP Pool and Infrastructure VLAN

1. TEP Pool
   • Tunnel endpoint address pool
   • IP addresses from this pool are assigned to all fabric elements to communicate over the infrastructure VLAN
   • “This subnet should not overlap with any other routed subnets in your network”
     • If this subnet does overlap with another subnet
     • Change this subnet to a different /16 subnet
     • The recommended minimum mask is /19
   • If you want to change this later on, the procedure is very 
2. Infrastructure VLAN
   • In-band VLAN
   • Allows APIC to communicate with leafs and spines
3. Changing TEP Pool and Infra VLAN after the initial provisioning
   • Changing either the infrastructure IP address (TEP IP pool) range or the infra VLAN after the initial provisioning setup process is not possible without rebuilding the fabric.
   • Procedures (reinitialize the APICs and the leafs):
     • Login to the Fabric, and take the backup (save it locally)
     • Delete all the configuration
     • Reinitialize the Fabric again (APICs and the leafs)
     • Set the correct value for TEP Pool and Infra VLAN
     • Restore the backup
4. Infrastructure VLAN and TEP Pool will be extended
   • Hypervisor integrations, such as OpenStack, Kubernetes, etc
     • Because a lot of these platforms don’t implement VRF
   • MultiPOD design
5. OpenStack integration example

• You might have connectivity issue if the Infra VLAN segment does overlap with another segment in your network
• In this case, the Openstack compute server won’t be able to reach NTP Server 10.0.0.35
  • Because default route 0.0.0.0/0 will lose against 10.0.0.0/16 and there is no VRF implementation

(2) UCS-B connectivity with VMware

• Each Fabric Interconnect has a port-channel (vPC) towards the ACI leafs
  • We can use switch dependendent protocol, such as LACP
• Fabric interconnects are connected for clustering
  • No data traffic is on the links
  • For example: can’t form vPC peer link using this links
• The hypervisor running on a blade has 2 independent connections
  • No switch dependent procotols can be used
    • > Using LACP IP-hash algorithms will cause MAC flaps on the UCS FI’s and N5K;s
  • Use switch independent protocol instead
    • > MAC Pinning-Physical-NIC Load for vSwitch policy