Memory Limits and Eviction Policies

Thiicket requires you to set a memory limit and by extension an eviction policy for each API key. Memory limits are designed to prevent bill shock, improve system stability and can be changed arbitrarily

Memory Limits

Memory limits are best thought of as safety quotas rather than a "memory allocation", insofar as if you have an application which uses only 10 MiB on average but bursts to 80 MiB it is reasonable to set your limit to 100 MiB or higher. You are only billed for what you use. You can assign an API key any limit from 0 to your account limit, as long as the sum of all limits set is less than the account limit

By default each account has a cumulative assignable limit of 100 MiB, reach out to support to have this increased

Evictions and eviction policies

Thiicket doesn't attempt to keep your memory usage exactly below the limits set, but rather tries to ensure in general the limit isn't exceeded. For more information about the internal mechanics of evictions, please see "Evictions: Deep Dive" below

When an eviction is triggered, Thiicket will attempt to evict memory as according to the eviction policy you have set

• Random

• Least Frequently Used

• Least Recently Used

• Biggest first

• Smallest first

• Everything

For all intents and purposes the act of evicting a key can be modelled as a user submitting a DEL command as a typical Redis client

Memory usage and evictions is evaluated on a regional base, a limit of 100 MiB with 80 MiB in region A and 70 MiB in region B will not trigger an eviction

It is worth nothing, there is no account level evictions, all memory usage and evictions are tracked and managed per API key. The account memory limit is only for the purposes of allowable limits to be set on each API key

Evictions: Deep Dive

The following may change at any time arbitrarily and without notice, it is only intended to provide a feel for what occurs 'under the hood'

Each time a key is created, deleted or otherwise modified an eviction can occur. Evictions will not occur in the following situations:

• An eviction has occurred in the past second (generally)

• Calculated memory usage hasn't exceeded the limit by at least 10%

• The underlying infrastructure keeping track of memory usage has reason to believe it isn't 'real-time' to within a certain tolerance

When an eviction occurs keys will be randomly sampled for eviction until enough keys are ready to evict to bring memory usage below the limit

In all instances except for the "Random" and "Everything" policy, two keys are sampled at a time and compared using the logic dictated by the key's eviction policy. The key out of the two chosen to be the best candidate is added to a queue for pending eviction. The "Random" policy does no such comparison, and the "Everything" policy queues all keys for eviction

Furthermore, every minute memory usage is also evaluated for eviction. In the case of no events relating to changes in memory usage while the limit is sufficiently breached, Thiicket will perform an eviction

The eviction policy caches memory limits and eviction policies, hence user changes may not be reflected instantly and could take up to a minute to manifest themselves