The Joy of Hex

Drunken Monkey Coding Style with a hint of Carlin humor

Sep 28, 2016 - 2 minute read - Redis Lua PHP

Deleting keys with wildcards in Redis

So, for you who don’t know Redis is an

open source (BSD licensed), in-memory data structure store, used as database, cache and message broker

Or to put it simply, it is a Key/Value Store and so much more

You have different data structures, that are efficient and great and all, that are at the end of the day identified by their keys, and this is where my problem begins.

I have some 100k+ hashes in redis with keys in the form of yer:key:harry:xxxyyzz. Which means that when I need to purge them from the application, I could just use yer:key:harry:* and be sure that only those hashes would be purged and not anything else of value.

So I looked at the del keyword in redis, and found out it can’t do wildcards. Isn’t that fun…

But, I can run LUA scripts by using eval command. So I google a bit and put together this piece of erm… code.

1
2
3
4
5
local keys = redis.call('keys', ARGV[1])
 for i=1,#keys,5000 do
  redis.call('del', unpack(keys, i, math.min(i+4999, #keys)))
 end
return keys

What the above does is call the keys with arguments passed ARGV[1] and then calls the del with nice 5000 item slices, so as to avoid Redis choking

So when you put it together with eval the command looks like

1
EVAL "local keys = redis.call('keys', ARGV[1]) \n for i=1,#keys,5000 do \n redis.call('del', unpack(keys, i, math.min(i+4999, #keys))) \n end \n return keys" 0 yer:key:harry:*

Second argument is the number of arguments that follows the script (starting from the third argument) that represent Redis key names, in this case it is 0.

The third argument is actually your wildcarded key name.

And since I am using PHP and phpredis, the full command is

1
$client->eval("local keys = redis.call('keys', ARGV[1]) \n for i=1,#keys,5000 do \n redis.call('del', unpack(keys, i, math.min(i+4999, #keys))) \n end \n return keys", ["yer:key:harry:*"], 0);

Notice that the second argument is wildcaded key name THAT MUST BE PASSED AS ARRAY, and that the third argument is second argument in the Redis lua example

With this in place, I can delete my keys in bulk, and go back to being productive.