Redis CPU Pinning

Or, How to Run Multiple Instances of Redis on One Machine.

Why would you even want to do something like this? Well Redis is a single threaded application. So if you have a server with 8 cores and it runs Redis, only 1 of those cores will ever be used by Redis.

Yes, there are some cases, such as bgsave, where this is not true.

By running multiple instances on the same machine and pinning each instance to specific CPU core you can better utilize the Cores to more quickly serve data.

To accomplish this I use :

taskset -c N

Here is an example from my init.d file that I use to run Redis on my Ubuntu 16.04 machine:

EXEC=/usr/bin/taskset
CLIEXEC=/usr/local/bin/redis-cli
PIDFILE=/var/run/redis_6380.pid
CONF="-c 1 /usr/local/bin/redis-server /etc/redis/6380.conf"
REDISPORT="6380"

Its pretty simple, instead of calling redis-server directly you first call /usr/bin/tasket and then pass in the proper arguments. If you were to type the full command out it would look like this:

taskset -c 0 redis-server /etc/redis/redis.conf
#this will use taskset to launch an instance of redis server and pin it #to core 0 on the server

The full file is below

willd@myserver.local@ORDRedis1:~$ cat /etc/init.d/redis_6380
#!/bin/sh
#Configurations injected by install_server below....

EXEC=/usr/bin/taskset
CLIEXEC=/usr/local/bin/redis-cli
PIDFILE=/var/run/redis_6380.pid
CONF="-c 1 /usr/local/bin/redis-server /etc/redis/6380.conf"
REDISPORT="6380"
###############
# SysV Init Information
# chkconfig: - 58 74
# description: redis_6380 is the redis daemon.
### BEGIN INIT INFO
# Provides: redis_6380
# Required-Start: $network $local_fs $remote_fs
# Required-Stop: $network $local_fs $remote_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Should-Start: $syslog $named
# Should-Stop: $syslog $named
# Short-Description: start and stop redis_6380
# Description: Redis daemon
### END INIT INFO


case "$1" in
    start)
        if [ -f $PIDFILE ]
        then
            echo "$PIDFILE exists, process is already running or crashed"
        else
            echo "Starting Redis server..."
            $EXEC $CONF
        fi
        ;;
    stop)
        if [ ! -f $PIDFILE ]
        then
            echo "$PIDFILE does not exist, process is not running"
        else
            PID=$(cat $PIDFILE)
            echo "Stopping ..."
            $CLIEXEC -p $REDISPORT shutdown
            while [ -x /proc/${PID} ]
            do
                echo "Waiting for Redis to shutdown ..."
                sleep 1
            done
            echo "Redis stopped"
        fi
        ;;
    status)
        PID=$(cat $PIDFILE)
        if [ ! -x /proc/${PID} ]
        then
            echo 'Redis is not running'
        else
            echo "Redis is running ($PID)"
        fi
        ;;
    restart)
        $0 stop
        $0 start
        ;;
    *)
        echo "Please use start, stop, restart or status as first argument"
        ;;
esac
Advertisement

2 thoughts on “Redis CPU Pinning

  1. Do you start another process in different port? e.g 6379, 6380, 6381 and so forth? Or they’ll use the same port?

    Runing multiple redis proceses(8) on different port will also allow using all of the cores. Whats the advantages of taskset in that case?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s