Configure cluster egress with outbound types in Azure Kubernetes Service
By: Date: 03/02/2024 Categories: azure Tags:

An AKS cluster’s egress can be tailored to meet certain needs. AKS will automatically supply a standard SKU load balancer for egress setup and use. However, if public IP addresses are blocked or extra hops are needed for egress, the default configuration might not be sufficient for every circumstance.

Limitations

Setting outboundType requires AKS clusters with a vm-set-type of VirtualMachineScaleSets and load-balancer-sku of Standard.

AKS types in Outbound

The following outbound kinds can be used to configure an AKS cluster: load balancer, NAT gateway, or user-defined routing. The outgoing type solely affects your cluster’s egress traffic.

LoadBalancer types of Outbound

The load balancer is used for egress through an AKS-assigned public IP. An outbound type of loadBalancer supports Kubernetes services of type loadBalancer, which expect egress out of the load balancer created by the AKS resource provider.

If loadBalancer is set, AKS automatically completes the following configuration:

  • A public IP address is provisioned for cluster egress.
  • The public IP address is assigned to the load balancer resource.
  • Backend pools for the load balancer are set up for agent nodes in the cluster.

Updating outboundType after cluster creation

Changing the outbound type after cluster creation will deploy or remove resources as required to put the cluster into the new egress configuration.

The following tables show the supported migration paths between outbound types for managed and BYO virtual networks.

Supported Migration Paths for Managed VNet

Managed VNetloadBalancermanagedNATGatewayuserAssignedNATGatewayuserDefinedRouting
loadBalancerN/ASupportedNot SupportedSupported
managedNATGatewaySupportedN/ANot SupportedSupported
userAssignedNATGatewayNot SupportedNot SupportedN/ANot Supported
userDefinedRoutingSupportedSupportedNot SupportedN/A

Supported Migration Paths for BYO VNet

BYO VNetloadBalancermanagedNATGatewayuserAssignedNATGatewayuserDefinedRouting
loadBalancerN/ANot SupportedSupportedSupported
managedNATGatewayNot SupportedN/ANot SupportedNot Supported
userAssignedNATGatewaySupportedNot SupportedN/ASupported
userDefinedRoutingSupportedNot SupportedSupportedN/A

Migration is only supported between loadBalancermanagedNATGateway (if using a managed virtual network), userAssignedNATGateway and userDefinedRouting (if using a custom virtual network).

Update cluster from loadbalancer to managedNATGateway

az aks update -g <resourceGroup> -n <clusterName> --outbound-type managedNATGateway --nat-gateway-managed-outbound-ip-count <number of managed outbound ip>

Update cluster from managedNATGateway to loadbalancer
az aks update -g <resourceGroup> -n <clusterName> --outbound-type loadBalancer <--load-balancer-managed-outbound-ip-count <number of managed outbound ip>| --load-balancer-outbound-ips <outbound ip ids> | --load-balancer-outbound-ip-prefixes <outbound ip prefix ids> >

Update cluster from managedNATGateway to userDefinedRouting

Add route 0.0.0.0/0 to default route table.

az aks update -g <resourceGroup> -n <clusterName> --outbound-type userDefinedRouting

Refer- Customize cluster egress with outbound in AKS