Lua scripting in Redis for performance gain

Redis, renowned for its speed and flexibility, is a powerful in-memory data store. However, as operations become more complex, optimizing queries becomes crucial. Lua scripting within Redis provides a solution, enabling the execution of custom logic directly on the server. This article delves into the performance benefits of Lua scripting, transitioning from traditional calls to transactions, and finally embracing Lua scripting for optimal efficiency.

Redis Database

The Bottleneck: Node.js / Deno and Multiple Commands

Traditionally, we use single function calls for each actions. For example,

redis.set(key1, val1);
redis.set(key2, val2);
redis.set(key3, val3);
redis.set(key4, val4);

In this code we are setting some values. But you can notice one thing… Ahhmm.. yes! We are kind of performing same type of actions. So if we have N number of calls, we use N number of network request to the redis server. So that will slow down you application a bit because the next line won’t execute until the previous one finishes.

So we have a network round-trip problem here.

Primary solution: Transactions

Redis transactions offer a powerful mechanism for optimizing the execution of multiple commands within a single network request. What I mean is we can send multiple commands using one single network request.

const tx = redis.tx();
tx.set(key1, val1);
tx.set(key2, val2);
tx.set(key3, val3);
tx.set(key4, val4);
// Additional set operations…
const res = await tx.flush();

Now all set command will be sent via one single network request and redis will execute them one by one.

Looks like we have solved our problem. Yes, but not really. We have a little issue here also. Not an issue actually, it depends on your application requirement. This operations are not executed at once by the redis application itself. And what if we needed logical operation. Then we have no way to improve by performing transaction. Look at this code,

// Perform multiple operations based on different conditions
const count = redis.get('key1:users:1');
const isAdmin = redis.get('user:isAdmin');

const tx = redis.tx();

if (count > 0) {
  if (isAdmin) {
    tx.del(key10); // Delete key10 if the user is an admin and count is positive
    tx.set(key1, val1); // Set key1 to val1
    tx.set(key2, val2); // Set key2 to val2
  } else {
    tx.set(key1, val1); // Set key1 to val1
    tx.set(key2, val2); // Set key2 to val2
    tx.set('log:user:action', 'non-admin-count-positive'); // Log action for non-admin user
  }
} else if (count === 0) {
  if (isAdmin) {
    tx.set(key1, val1); // Set key1 to val1
    tx.set(key2, val2); // Set key2 to val2
  } else {
    tx.set(key1, 'default_val'); // Set key1 to default value for non-admin user
    tx.set(key2, 'default_val'); // Set key2 to default value for non-admin user
    tx.set('log:user:action', 'non-admin-count-zero'); // Log action for non-admin user
  }
} else {
  // Handle negative count scenario
  tx.set('log:error', 'negative_count_encountered'); // Log error for negative count
}

const res = await tx.flush(); // Execute the transaction

Even though we used transactions we still need 2 redis calls before transaction to perform logical operations. So….. now what can be improved?

Lua Scripting

Lua scripting allows encapsulation of complex logic within a single script, executed on the Redis server. Initially, Lua code is sent to the server each time it’s needed. This code will be executed by redis itself so we eleminate the network round-trips.

local count = tonumber(redis.call('GET', ARGV[1])) // Get the arguments from ARGV
local isAdmin = redis.call('GET', ARGV[2])

if count and isAdmin then
    if count > 0 then
        if isAdmin == "true" then
            redis.call('DEL', 'key10') -- Delete key10 if the user is an admin and count is positive
            redis.call('SET', 'key1', 'val1') -- Set key1 to val1
            redis.call('SET', 'key2', 'val2') -- Set key2 to val2
        else
            redis.call('SET', 'key1', 'val1') -- Set key1 to val1
            redis.call('SET', 'key2', 'val2') -- Set key2 to val2
            redis.call('SET', 'log:user:action', 'non-admin-count-positive') -- Log action for non-admin user
        end
    elseif count == 0 then
        if isAdmin == "true" then
            redis.call('SET', 'key1', 'val1') -- Set key1 to val1
            redis.call('SET', 'key2', 'val2') -- Set key2 to val2
        else
            redis.call('SET', 'key1', 'default_val') -- Set key1 to default value for non-admin user
            redis.call('SET', 'key2', 'default_val') -- Set key2 to default value for non-admin user
            redis.call('SET', 'log:user:action', 'non-admin-count-zero') -- Log action for non-admin user
        end
    else
        redis.call('SET', 'log:error', 'negative_count_encountered') -- Log error for negative count
    end
else
    return "Error: Unable to retrieve count or isAdmin value from Redis"
end

So this the lua code which we will send to redis server via `redis.eval()` function like this,

const luaCode = "..." // Lua code as string

const res = redis.eval(luaCode, [], ['key1:users:1', 'user:isAdmin']);

//handle the result

Looks good. But….. it has even more serious problem. Look at the lua code size. You would send that whole code to redis over the network every time you call the script. So that eats a lot of bandwidth. That’s why we use `loadScript()` instead of `eval()`. They both work the same but `loadScript` loads the script into the redis server once and retuns a SHA hash that you use to call the lua code later. Subsequent calls reference the script by its SHA hash, eliminating the need to send the code repeatedly. Benefits include:

• Reduced Network Traffic: Only the SHA hash, a compact identifier, travels over the network for subsequent calls.
• Atomic Execution: The script runs on the server-side, ensuring all operations within it are treated as a single unit. This improves consistency and avoids race conditions.

const SHA = await redis.scriptLoad(getAllUsersDataScript);

//later call the script
const result = await redis.evalsha(SHA, [], [...agrs]) as string;

How cool is that!

Performance Gains and Beyond

While the exact performance improvement depends on your specific use case, Lua scripting can significantly reduce network traffic and improve response times, especially for complex operations involving multiple keys or conditional logic. Additionally, Lua allows for more intricate data manipulation and control flow within Redis, unlocking further optimization possibilities.

Here is a average benchmark. This result may vary based on the server you are using.

Perfomance graph (smaller is better)

So, for the low amount of operations performance difference is not that huge. For simple read-write, lua scripting may be overkill. But we can clearly see the difference here.

Conclusion

Lua scripting empowers developers to leverage Redis’s processing power for complex tasks. By reducing network communication and enabling atomic operations, Lua scripts can significantly enhance the performance of your Redis interactions. Consider exploring Lua scripting in your Deno or Node.js applications to unlock the full potential of Redis for data manipulation and caching.