Event Timeslots (1)
Track 2 – 2024
-
Presenter: Sushanth Kothapally and Senthil Kamala Rathinam
Abstract: As enterprises grow, their demand for IP addresses often exceeds what their corporate network can provide. While networks are designed to tackle future needs, evolving enterprises eventually outgrow them. This often possess constraint of running multiple AWS Glue jobs in parallel that require number of database connections. To enable more connections, organizations can obtain additional IP addresses from their corporate pool, either unique or overlapping with the existing network. Overlapping IP addresses require extra network management like private NAT gateways, AWS PrivateLink, or self-managed NAT appliances to establish connectivity. This presentation discusses two strategies to scale AWS Glue jobs within IP-constrained networks:
Optimize IP address usage by right-sizing Data Processing Units, using AWS Glue Auto Scaling, and fine-tuning jobs
Expand network capacity by adding a non-routable CIDR range with a private NAT gateway.
Implementing these strategies allows companies to maximize their use of IP addresses to support more data processing workloads using AWS Glue
AWS Services: AWS Glue, AWS Transit Gateway, AWS Transit Gateway, Amazon VPC
Audience: Advanced
angelo