The Inner Workings of Section’s Adaptive Edge Engine

We recently highlighted the results of a series of tests proving the superior performance of Section’s Adaptive Edge Engine (AEE) over a major cloud provider. Across the series of tests, Section’s Edge proved to be consistently faster with less extreme risk than the Cloud deployment. The average latency for the slowest 1% of Section Edge users was equal to the average latency of all users in the Cloud case.

In this post, we will be uncovering the inner workings of the AEE that are delivering these compelling results.

What is the Adaptive Edge Engine?

Without automated workload orchestration, edge compute will be prohibitively expensive for many workloads. The AEE (patent pending) is a dynamic, context-aware workload scheduling and traffic-routing technology.

In setting out to build this technology, our goal was to facilitate the creation of systems that effectively manage lower latency and cost optimization for edge workloads. We also want to avoid the limitations of traditional content delivery solution architectures, including single datacenter hosting, single datacenter hosting with CDN, multi datacenter hosting and CDN with serverless, all of which limit end-user performance due to network latency issues and computational cost parameters.

Developers can use Section’s Adaptive Edge Engine to optimize dynamic edge networks across the attributes of their choosing. For example, customers can craft unique strategies for obtaining performance and cost targets, meeting security/regulatory requirements, or even preferring low-carbon/energy-efficient infrastructure (helping bolster a more sustainable energy model).

The AEE: Four Primary Goals

At the outset, our four primary goals for the AEE were to:

1. Minimize end-user latency by shortening network paths and determining the best possible path based on signals from the entire workload pool.
2. Provide broad runtime system support, enabling easy deployment of complex application logic written in a variety of languages.
3. Ensure that edge workloads efficiently reserve and use compute capacity so that applications are only deployed in locations while they add value and are withdrawn otherwise.
4. Enable low friction workload adoption, consistent with our collective passion to deliver a better Edge for DevOps Engineers with minimal hassle.

To achieve these goals and overcome the limitations of content delivery solutions on the market, Section’s AEE makes extensive use of the strategy pattern - in order to support the broadest set of use cases without altering the core technology.

How Does the AEE Work?

The AEE is a system built of several components. These include:

Location Optimizer

Determines where a workload should be deployed.

• The Location Optimizer is a general purpose system for implementing optimization “strategies”.
• The customer defines their strategy, a combination of objective(s) and constraints and the Location Optimizer will solve for the optimum.
• Optimality is conditional. No solution is optimal for all systems and under all circumstances. Each customer needs the flexibility to drive results toward their business needs at the time. Some businesses may need to maximize performance within a fixed budget; others may seek to minimize cost for an accepted level of performance; still others may have geographic or security concerns that out-weigh both cost and performance. The flexibility of the AEE Location Optimizer allows any such combinations to be optimized.

Health Checker

Once the workload has been deployed, the health checker ensures it is where it should be.

• The Health Checker enables health check strategies that will be specific to each different environment.
• A specific workload might have a unique test or health check that it requires.
• The Health Checker also accepts strategies.

Both these components interface with Section customers. Customer and workload-specific requirements drive how the Location Optimizer and Health Checker work for any given environment.

The other components in the system are working to make sure that Section is doing what it needs to, in terms of deploying the workload, routing traffic to it, etc. These components are:

Traffic Director

Responsible for routing the traffic to wherever the workload is deployed using DNS and BGP.

• Controls traffic flows to the optimal endpoints when application traffic is desired;
• Removes traffic flows from endpoints when application flows are not desired;
• Allows traffic to flow to endpoints based on the outcomes of health checks performed on an application per endpoint;
• Controlling the rate of traffic flow to an endpoint.

Workload Controller

Deploys customer environments to endpoints.

• Scales the environment horizontally and vertically as needed.

Endpoint Controller

Manages the endpoints.

• Manages the number of nodes running;
• Determines where the endpoints are located;
• Spins them up when we need them and down when we don’t;
• Controls the size of the application resource allocation within the endpoint.

These three components are always running in the background, making sure that there is constant alignment between customer preferences and what the AEE discovers and implements, so that the requests get to where they need to as quickly and efficiently as possible.

The Intelligence Backbone

Any Location Optimizer strategy that optimizes relative to traffic volumes needs a traffic data feed. The basic choices are: past data or forecasts. We currently have a “forecastRPS” (requests per second) strategy which uses the user-traffic-forecast as a data feed.

ML/AI is a core Section capability that allows better data feeds that will:

1. Enable more sophisticated strategies and;
2. Increase the effectiveness of the AEE components, e.g. making HPA (Horizontal Pod Autoscaler) and HCA (Horizontal Cluster Autoscaler) work better.

The actions of the AEE are passively coordinated through a central data store recording the current states of all environments on Section crossed with all available endpoints. We call this the state store and the data it retains is amazingly simple. The complexity of the myriad use cases is retained in the strategies the customer helps define. The AEE is robust and powerful in direct relation to its simplicity.

As the different components do their jobs: optimizing location strategies, creating and destroying endpoints, health-checking every workload, etc., they are reading from and writing to the state store. The components safely ignore each other and still develop coordinated, efficient outcomes.

The AEE is not a set of activities or rules that are executed as an ordered series of jobs. The components are separate, each running and executing its own process with or without extra logic provided in customer strategies. By basing those processes on information in the state store, each component becomes connected, leading to a robust, continuous, coordinated system with functions spanning concerns such as “Where are the necessary endpoints?”, “How many copies of the workload are needed?”, and “How much does this customer favor latency over cost?”

Conclusion

Ultimately, Section’s Adaptive Edge Engine serves five key objectives for our end-users:

1. Enable system builders to define their intent. Using the AEE, system builders can direct the AEE to find an optimal solution to latency management based on their specific performance needs and budgetary constraints.
2. Provide support for system builder defined restrictions. System builders can tag their workload with properties that let them control which nodes their workload can execute on.
3. Customize the edge network. The AEE can expand and contract the total available edge network by automating the purchasing of commodity server infrastructure informed by signals from the entire workload pool.
4. Enable decision frequency. The AEE can evaluate and adjust the deployment and routing of each workload independently, adjust the network accordingly, and converge end-user traffic to that determination within a ten minute window.
5. Create zero server management. The AEE allows system builders to place their focus on business imperatives as opposed to operational server and network management responsibilities.

As we continue to develop the AEE over the coming months and years, we are excited to see how we can expand its use as a framework while delivering the greatest value. We will be looking at how we can help customers define their optimal strategy within it and hopefully arrive at patterns that allow people to build their own strategies for the AEE to easily run for them.