Virtual waiting rooms, or online queuing systems, are a good, simple way to control traffic surges on web applications. While they aren’t intended to be a replacement for properly scaling infrastructure, they can help protect a website’s overall user experience by remaining online for a subset of users, while excess traffic is queued.
At Section, we recently underwent a re-evaluation of the underlying logic behind our Virtual Waiting Room edge module. Previously, we were using a random selection model, which started to raise questions of fairness around user experience. So, we began evaluating FIFO and True Queuing models, each of which has its own benefits and drawbacks.
Online Queuing Models
There are several different models when it comes to virtual waiting room technology and these nuances can be broken down into logic around ‘queuing’. At a high level, queuing methods can be broken down into three different models.
Random Selection
In a random queuing model, application administrators set a maximum threshold for the number of concurrent visitors allowed on the website. Once this threshold is reached, subsequent users trying to access the website are placed into a holding area. As users exit and capacity becomes available for new users to enter the site, the technology will serve the next request, either from the queue (i.e. if a user in the queue refreshes their page), or from a new user attempting to access the site.
Random queuing logic is solely based on the next request hitting the website and does not discern between users in the queue and new users, meaning that new users can essentially ‘jump the line’ ahead of users who have been waiting in the holding area.
First In First Out (FIFO)
As the name suggests, a true First In First Out (FIFO) queuing model is a linear model that lines up and admits users based on the order in which their requests were received. On the surface, this can seem like the ‘fairest’ model; however, if you’re only ever waiting on the first person in the queue, that user is potentially (and likely) holding up the rest of the line until they refresh or trigger a new page request, which can lead to a degraded user experience as a whole. There are instances where this model is appropriate, such as with online ticketing applications.
Modified FIFO
A modified FIFO model (like the one used for Section’s Virtual Waiting Room) applies more logic to queuing decisions by creating progressive layers of the queue. Application administrators still set a maximum threshold for concurrent users on the website. However, there is an additional layer that provides a mechanism to only grant access to people at the head of the queue. The decision to admit new users from the head of the queue is still based on the first request hitting the website, but this eliminates the possibility of users bypassing the queue.
Building Section’s FIFO Virtual Waiting Room
As we looked to improve upon our random selection Virtual Waiting Room model, our aim was to strike an optimal balance of good throughput and fairness. We started by documenting the desired flow.
We were then able to translate this flow to technology decisions that include:
- A script to detect the amount of concurrent users on the website
- The ability for Section admins to set thresholds for visitor access allowances and size of the head of the queue waiting area.
- Session id objects that include a time to live (ttl) that determines active vs. inactive users in the head of the queue
Additional Considerations
There are several other factors to consider when evaluating a FIFO model, including handling of bots and exempt users.
Bots
You don’t want bot activity, such as Google or Bing bots, holding up spots in your queue. For this scenario, we’ve included logic that checks for a valid session id before placing users in the queue.
Exempt Users
Exempt users, such as site administrators, need to be able to bypass the queue. For these cases, you can configure a custom url that allows you to obtain an access cookie immediately.
Summary
Online queuing solutions are very effective at preventing websites from crashing during high traffic events. Virtual waiting room technology, and more specifically, the underlying logic driving it, should be evaluated based on the specific use case.
Section’s Virtual Waiting Room offers flexible configuration options to meet a wide variety of requirements. Contact our team of solutions engineers to chat through your specific use case.
