PureDevOps Community

EKS LoadBalancer (Ingress) is not to able to auto-discover subnets

Ensure that --cluster-name in the aws-load-balancer-controller deployment is correct configured.

Use

kubectl get deployment -n kube-system aws-load-balancer-controller -oyaml |grep "cluster-name"

to get the cluster name in the deployment.

If it isn’t correct, edit deployment with next command and rename it:

kubectl edit deployment -n kube-system aws-load-balancer-controller

In other case, it was because I hadn’t labeled the AWS subnets with the correct resource tags. https://kubernetes-sigs.github.io/aws-load-balancer-controller/guide/controller/subnet_discovery/

Edit - 5/28/2021

Public Subnets should be resource tagged with: kubernetes.io/role/elb: 1

Private Subnets should be tagged with: kubernetes.io/role/internal-elb: 1

Both private and public subnets should be tagged with: kubernetes.io/cluster/${your-cluster-name}: owned

or if the subnets are also used by non-EKS resources kubernetes.io/cluster/${your-cluster-name}: shared

Source: Subnet Discovery - AWS LoadBalancer Controller

How can I automatically discover the subnets used by my Application Load Balancer in Amazon EKS?

Last updated: 2021-08-03

I want to automatically discover the subnets used by my Application Load Balancer (ALB) in Amazon Elastic Kubernetes Service (Amazon EKS).

Short description

You can tag your AWS subnets to allow the AWS Load Balancer controller to auto discover subnets used for Application Load Balancers.

Resolution

  1. Deploy the AWS Load Balancer Controller for your Amazon EKS cluster.

  2. Verify that the AWS Load Balancer Controller is installed:

kubectl get deployment -n kube-system aws-load-balancer-controller

Note: If the Deployment is deployed in a different namespace, then replace -n kube-system with the appropriate namespace.

  1. Create a Kubernetes Ingress resource on your cluster with the following annotation:
annotations:
    kubernetes.io/ingress.class: alb

Note: The AWS Load Balancer Controller creates load balancers. The Ingress resource configures the Application Load Balancer to route HTTP(S) traffic to different pods within your cluster.

  1. Add either an internal or internet-facing annotation to specify where you want the Ingress to create your load balancer:
alb.ingress.kubernetes.io/scheme: internal

-or-

alb.ingress.kubernetes.io/scheme: internet-facing

Note: Choose internal to create an internal load balancer, or internet-facing to create a public load balancer.

  1. Use tags to allow the Application Load Balancer Ingress Controller to create a load balancer using auto-discovery. For example:
kubernetes.io/role/internal-elb                Set to 1 or empty tag value for internal load balancers
kubernetes.io/role/elb                         Set to 1 or empty tag value for internet-facing load balancers

Note: You can use tags for auto-discovery instead of the manual alb.ingress.kubernetes.io/subnets annotation.

Example of a subnet with the correct tags for a cluster with an internal load balancer:

kubernetes.io/role/internal-elb          1

Example of a subnet with the correct tags for a cluster with a public load balancer:

kubernetes.io/role/elb                     1

Note: For cluster versions 1.18 and earlier, Amazon EKS adds the following tag to all subnets passed in during cluster creation. The tag isn’t added to version 1.19 clusters. If you’re using the tag and you update to cluster version 1.19 from an earlier version, then you don’t have to add the tag again. The tag stays on your subnet. You can use the following tag to control where an Application Load Balancer is provisioned. Use this tag in addition to the subnet tags required for automatically provisioning an Application Load Balancer.

kubernetes.io/cluster/$CLUSTER_NAME    shared

Important: The AWS Load Balancer Controller workflow checks subnet tags for the value of " " (empty string) and 1. For private subnets, set the value of the kubernetes.io/role/internal-elb tag to an empty string or 1. For public subnets, set the value of the kubernetes.io/role/elb tag to an empty string or 1. These tags allow your subnets to be auto-discovered from the Amazon EKS VPC subnets of your Application Load Balancer.

  1. Validate that your Amazon EKS VPC subnets have the correct tags:
aws ec2 describe-subnets --subnet-ids your-subnet-xxxxxxxxxxxxxxxxx
  1. Deploy a sample application to verify that the AWS Load Balancer Controller creates an Application Load Balancer as a result of the Ingress object:
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/examples/2048/2048_full.yaml
  1. Verify that the Ingress resource gets created and has an associated Application Load Balancer:
kubectl get ingress/2048-ingress -n 2048-game

Either an internal or internet-facing load balancer is created, depending on the annotations (alb.ingress.kubernetes.io/scheme:) that you defined in the Ingress object and subnets.

Ref: Automatically discover subnets used by Application Load Balancers in Amazon EKS