Taming the Ingress Nightmare: How we secured our AWS EKS Cluster against critical vulnerabilities

DevOps team remediated critical "IngressNightmare" vulnerabilities in AWS EKS within 48 hours with zero downtime through assessment, planning, execution, and verification.

Ali

When our DevOps team received an AWS Security Notification about critical vulnerabilities in the ingress-nginx controller on March 24th, we initiated our AWS incident response plan immediately. Dubbed "IngressNightmare," these vulnerabilities could potentially allow attackers to execute arbitrary code and compromise our entire Amazon EKS cluster security posture.

Here's how we identified, assessed, and remediated this critical AWS security finding within 48 hours – with zero service disruption and full AWS audit compliance.

The threat: five critical vulnerabilities in AWS EKS

The most severe vulnerability, CVE-2025-1974 (CVSS 9.8), affected the ingress-nginx Validating Admission Controller in our AWS EKS environment. If exploited, it would allow attackers with pod network access to:

  • Inject malicious NGINX configurations
  • Execute arbitrary code on the controller
  • Potentially compromise all AWS EKS cluster secrets
  • Create a compliance violation in our AWS Security Hub findings


Four additional vulnerabilities completed the "IngressNightmare" package: CVE-2025-1098, CVE-2025-1097, CVE-2025-24514, and CVE-2025-24513, with CVSS scores ranging from 4.8 to 8.8.

Our AWS-Compliant four-step response process

1. AWS Security assessment and intelligence gathering

First, we confirmed our exposure by checking our ingress-nginx version in our AWS EKS cluster:

kubectl get pods --all-namespaces -l app.kubernetes.io/name=ingress-nginx -o jsonpath='{.items[*].spec.containers[*].image}'


We were running version 1.11.3 – vulnerable to all five CVEs and flagged in our AWS Security Hub.

Consulting authoritative sources like AWS Security Bulletins, Wiz.io, Devoriales, and the Kubernetes Blog, we learned that these vulnerabilities affected a staggering 43% of cloud environments, including many AWS EKS deployments. The official patches were available in versions 1.11.5 and 1.12.1.

2. AWS-Compliant remediation planning

Following AWS Well-Architected Framework security best practices, we decided to upgrade directly to version 1.12.1, which addressed all vulnerabilities by:

  • Disabling unsafe NGINX configuration validation
  • Patching the injection vulnerabilities in the admission controller
  • Creating detailed AWS CloudTrail logs for audit purposes

3. Execution with zero downtime on AWS EKS

Our AWS EKS remediation took just three commands, with detailed logging enabled for AWS CloudTrail and AWS Config auditing:

# Add the official Helm repository
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

# Upgrade to the patched version with admission webhooks temporarily disabled
helm upgrade ingress-nginx ingress-nginx/ingress-nginx \
  --version 4.11.5 \
  --set controller.image.tag="v1.12.1" \
  --set controller.admissionWebhooks.enabled=false \
  --namespace ingress-nginx

We used AWS EKS's native rolling update strategy to ensure zero downtime during the upgrade and minimize impact on our AWS application reliability.

4. AWS security verification and re-enabling security features

After confirming the upgrade was successful in our AWS EKS cluster, we re-enabled the admission webhook – an important feature that warns users about incorrect Ingress configurations:

helm upgrade ingress-nginx ingress-nginx/ingress-nginx \
  --version 4.11.5 \
  --set controller.image.tag="v1.12.1" \
  --set controller.admissionWebhooks.enabled=true \
  --namespace ingress-nginx


Finally, we ran vulnerability scans against our public endpoints to verify the patch and ensure AWS compliance:

nuclei -u https://development.dashboard.partytrick.com/ -t nginx.yaml


No alerts were detected, confirming successful remediation. We updated our AWS Security Hub findings to mark the vulnerability as resolved, ensuring our AWS audit trail was complete.


Key takeaways for AWS engineering leaders

  1. Leverage AWS Security Notifications – Our prompt response was possible because we had properly configured AWS Security Hub alerts and AWS SNS notifications.

  2. Follow AWS Well-Architected Framework security best practices – Having a defined process aligned with AWS security standards allowed us to move from notification to remediation in under 48 hours.

  3. Use AWS EKS rolling updates for zero downtime – By leveraging Kubernetes' native deployment strategies on AWS EKS, we secured our infrastructure without service interruption.

  4. Maintain AWS audit compliance – We documented all steps in AWS CloudTrail and created detailed remediation notes for our next AWS security audit.

This incident reinforced that in today's AWS cloud-native world, security vulnerabilities are inevitable – but with the right process and AWS security tools, you can respond swiftly and minimize risk to your organization while maintaining AWS compliance.

Has your team faced similar challenges with AWS EKS security? We'd love to hear about your AWS security response processes in the comments.

Interested in learning more?

Opening Motion Calendar...

Congrats, you may be CDAP grant eligible!
Book a consultation with our founder on another tab.
Facing issues? Click here to try again.

Successfully submitted.

Oops! Something went wrong while submitting the form.