Automating AWS Infrastructure with Jenkins, Terraform, and Kubernetes (EKS)

If you’re looking to automate the deployment and management of AWS resources using Jenkins, Terraform, and Kubernetes (EKS), you’re in the right place! In this guide, we’ll walk you through setting up Jenkins to provision an EKS cluster, deploy applications using Helm, and manage the entire infrastructure lifecycle — all with a few simple clicks.

Key Steps to Automate AWS Infrastructure with Jenkins, Terraform & Kubernetes

1. Setting Up Jenkins:
   • First, we install and configure all necessary tools like Terraform, kubectl, and AWS CLI on the Jenkins server.
   • We then create a Jenkins Freestyle Project where we’ll configure parameters for the region, VPC ID, cluster name, and Terraform action (apply or destroy).
2. GitHub Integration:
   • You’ll connect Jenkins to a GitHub repository containing your Terraform configuration.
   • This repo will hold the infrastructure code that Jenkins will execute (make sure the branch is set to main instead of master).
3. Terraform Configuration:
   • In Jenkins, we define the Terraform variables such as region, VPC ID, and cluster name, which will be used in our infrastructure provisioning script.
   • We also ensure that any resources created (like an EKS cluster) are stored in an S3 backend for persistence.
4. Install Rebuild Plugin:
   • To simplify running the Jenkins job multiple times without re-entering values, we install the Rebuild Plugin. This plugin allows us to quickly re-run jobs with the same parameters.
5. IAM Role and Permissions:
   • If you’re getting access errors (like No valid credential source for S3 backend), don’t worry. You’ll need to set up an IAM role for your Jenkins EC2 instance to ensure Terraform has the required permissions to create and manage AWS resources.
6. Deploying Infrastructure with Terraform:
   • Once the IAM permissions are configured, you can trigger Jenkins to run the Terraform job. The job will provision the necessary AWS resources (like an EKS cluster and node groups), ensuring a seamless deployment.
7. Helm Deployment on EKS:
   • Once the cluster is created, we use Helm to install applications like nginx and AWS Load Balancer Controllers in the Kubernetes cluster.
   • Jenkins runs Helm commands, and once complete, you can verify the deployments using kubectl.
8. Troubleshooting & Common Errors:
   • If you encounter the kubectl command not found error, it’s likely because kubectl is installed under the EC2 user, not the Jenkins user. A simple fix: ensure kubectl is installed and available in the Jenkins user’s environment.
9. Destroying Infrastructure:
   • Once the deployment is complete, the next step is to clean up. You can re-run the Jenkins job with the destroy action to tear down the entire infrastructure (EKS cluster, EC2 instances, VPC, etc.).
   • Don’t forget to manually delete any load balancers created to avoid unnecessary charges.

Conclusion:

This guide provides a detailed approach to automating the entire lifecycle of AWS infrastructure — from provisioning to deployment and clean-up — using Jenkins, Terraform, and Kubernetes. Whether you’re new to these tools or looking to streamline your infrastructure processes, this step-by-step guide is sure to help you get started.

Stay tuned for more Kubernetes and EKS-related content in our upcoming videos! If you have any questions or run into any issues, feel free to leave a comment, and I’ll do my best to help you out.

Happy automating! 🚀