Node Auto-Provisioning (NAP) in Azure Kubernetes Service (AKS) allows you to define AKSNodeClass resources to dynamically provision node pools based on workload demands. Below is a detailed, step-by-step guide to configure AKSNodeClass resources for NAP.
Prerequisites
- Azure Subscription with access to AKS and NAP preview features.
- Azure CLI installed (
azversion ≥ 2.51.0). - kubectl configured for your AKS cluster.
- An existing AKS cluster (or create one during the process).
- Resource Group for your AKS cluster and NAP resources.
- Managed Identity (system-assigned or user-assigned) for AKS node management.
Step 1: Enable NAP Feature in Your Azure Subscription
NAP is currently in preview. Enable the feature flag for your subscription.
# Register the NAP feature
az feature register --namespace Microsoft.ContainerService --name AKSNodeAutoProvisioning
# Wait ~15 minutes for the feature to propagate
az feature list -n AKSNodeAutoProvisioning --query "features[?name=='AKSNodeAutoProvisioning'].properties.state" -o json
# Enable NAP for your AKS cluster during creation or update
Step 2: Create a Managed Identity for Node Provisioning
AKS requires a managed identity to provision resources (e.g., VM Scale Sets, NICs).
# Create a user-assigned managed identity
az identity create --resource-group --name akSNPIdentity
# Get the identity principal ID
IDENTITY_PRINCIPAL_ID=$(az identity show --resource-group --name akSNPIdentity --query principalId -o tsv)
Step 3: Grant Required Permissions to the Managed Identity
Assign Contributor and Network Contributor roles to the identity.
# Assign Contributor role to manage compute resources
az role assignment create --assignee $IDENTITY_PRINCIPAL_ID \
--role "Contributor" \
--scope /subscriptions//resourceGroups/
# Assign Network Contributor role for VNet/subnet access
az role assignment create --assignee $IDENTITY_PRINCIPAL_ID \
--role "Network Contributor" \
--scope /subscriptions//resourceGroups//providers/Microsoft.Network/virtualNetworks/
Step 4: Prepare Your AKS Cluster
Ensure your AKS cluster is NAP-enabled. Create a new cluster or update an existing one:
Option A: Create a New NAP-Enabled AKS Cluster
az aks create \
--resource-group \
--name \
--node-auto-provisioning true \
--node-identity-client-id \ # Optional: if using user-assigned identity
--node-identity-object-id $IDENTITY_PRINCIPAL_ID \
--enable-oidc-issuer \
--network-plugin azure \
--vnet-subnet-id /subscriptions//resourceGroups//providers/Microsoft.Network/virtualNetworks//subnets/
Option B: Update an Existing Cluster
az aks update \
--resource-group \
--name \
--enable-node-auto-provisioning \
--node-identity-client-id \
--node-identity-object-id $IDENTITY_PRINCIPAL_ID
Step 5: Create a Virtual Network and Subnet (If Not Existing)
Ensure you have a subnet for node auto-provisioning.
# Example: Create a VNet and subnet
az network vnet create \
--resource-group \
--name myAKSVnet \
--address-prefix 10.0.0.0/16 \
--subnet-name node-subnet \
--subnet-address-prefix 10.0.0.0/24
Note the subnet ID for later use.
Step 6: Define an AKSNodeClass Resource
Create a AKSNodeClass YAML file to specify node configuration (e.g., VM size, OS disk, scaling limits).
Example aksnodeclass.yaml
apiVersion: autoscaling.microsoft.k8s.io/v1beta1
kind: AKSNodeClass
metadata:
name: standard-d2s-v3-nodeclass
spec:
properties:
# VM SKU (e.g., Standard_D2s_v3)
vmSize: Standard_D2s_v3
# OS disk size (MiB)
osDiskSizeMiB: 102400
# Subnet for nodes
subnetId: /subscriptions//resourceGroups//providers/Microsoft.Network/virtualNetworks//subnets/
# Optional: Enable public IP for nodes (default: false)
enablePublicIP: false
# Optional: Max pods per node
maxPods: 30
# Optional: Node labels/taints
labels:
node-class: standard-d2s-v3
taints: []
Apply the configuration:
kubectl apply -f aksnodeclass.yaml
Step 7: Create a NodePool Linked to AKSNodeClass
Define a NodePool resource to map the AKSNodeClass to autoscaling parameters.
Example nodepool.yaml
apiVersion: autoscaling.microsoft.k8s.io/v1beta1
kind: NodePool
metadata:
name: standard-d2s-v3-pool
spec:
aKSNodeClass: standard-d2s-v3-nodeclass # References the AKSNodeClass name
minNodeCount: 1
maxNodeCount: 5
nodeCount: 1
# Optional: Node labels/taints inherited from AKSNodeClass
labels:
node-pool: standard-d2s-v3-pool
Apply the configuration:
kubectl apply -f nodepool.yaml
Step 8: Verify Node Auto-Provisioning
- Check Nodes:
kubectl get nodesYou should see nodes from the new node pool. - Check AKSNodeClass and NodePool Status:
kubectl get aksnodeclass kubectl get nodepool - Test Scaling: Deploy a workload that exceeds current node capacity. The AKS autoscaler will provision new nodes based on the
minNodeCount/maxNodeCountdefined in theNodePool.
Step 9: Clean Up (Optional)
To avoid costs, delete resources when done:
az aks delete --resource-group --name --yes
az identity delete --resource-group --name akSNPIdentity
Key Notes
- NAP is Preview: Feature availability may vary by region.
- RBAC: Ensure the managed identity has Contributor and Network Contributor roles.
- Subnet Considerations: Nodes must reside in a dedicated subnet with sufficient IP addresses.
- Scaling Limits: Adjust
minNodeCount/maxNodeCountbased on workload demands. - Monitoring: Use Azure Monitor or Kubernetes events to troubleshoot provisioning issues.
By following these steps, you’ll enable dynamic node auto-provisioning in AKS using AKSNodeClass resources, optimizing cluster scalability and cost efficiency.
Configure AKSNodeClass resources for node auto-provisioning (NAP) in Azure Kubernetes Service (AKS)