Our applications often rely on external services to provide pieces of functionality, such as cloud compute services, DNS services and information sources for information-heavy services like weather and sports. Gumby is no stranger to this. Among the IAAS providers that Gumby supports, we have Amazon EC2, with upcoming support for Microsoft Azure. Gumby also relies on Amazon’s Route53 DNS service to provide DNS record management to deployed services.
Service providers often impose an upper limit on the average or instantaneous rate of requests to an API by source host or tenant. Hence, when calling into external services such as Amazon’s AWS, there is a need to consciously manage the rate at which the APIs are targeted. In a Scala/Akka actor-based application like Gumby, this can be achieved very elegantly using a “throttler”. A number of such throttler implementations have been written, some of which are now part of the akka standard library for Scala/Java. Of particular interest is the TimerBasedThrottler, which provides the ability to set a maximum rate at which messages can pass through it. However, it suffers from a known limitation, namely, it only imposes the rate measured over regular intervals bounded by the start time of the timer. This makes its functionality non-deterministic when it comes to satisfying external rate limits. This talk will highlight this limitation and present the enhancement that addresses this limitation and makes the throttler more generally and deterministically satisfy its requirements.
Vikas holds a Masters in Aerospace Engineering from the University of Texas and an MBA from San Francisco State University, but his heart surely lies in the craftsmanship and delivery of software. He has been practicing the art of software engineering for 2 decades – most often speaking C, C++ and Java and using OO paradigms, but more recently transitioning to Scala and functional paradigms and loving it. He is passionate about continuous deployment and delivery and is part of the team developing tools and infrastructure to facilitate just that for software development teams at Comcast.