Data Science in Drilling - Episode 13

Manipulating Redis with Python - Part I

written by Zeyu Yan, Ph.D., Head of Data Science from Nvicta AI

Data Science in Drilling is a multi-episode series written by the technical team members in Nvicta AI. Nvicta AI is a startup company who helps drilling service companies increase their value offering by providing them with advanced AI and automation technologies and services. The goal of this Data Science in Drilling series is to provide both data engineers and drilling engineers an insight of the state-of-art techniques combining both drilling engineering and data science.

This episode is about both Redis and Python. Enjoy! :)

Enjoying great knowledge is just like enjoying delicious popcorn chicken.

Introduction

Redis is an open source (BSD licensed), in-memory data structure store used as a database, cache, message broker, and streaming engine. It is really popular among the developers all over the world. In today’s blog post, we will cover how to manipulate Redis through Python.

What We'll Cover Today

1. How to run Redis using docker-compose.

2. How to manipulate Redis using Python.

3. How to use Redis with Python's asyncio.

Run Redis using Docker-compose

The first step is to run a Redis instance on the local machine for testing. The easiest way is through docker-compose. Our docker-compose.yml file is as follows:

version: "3.3"

services:
  redis:
    image: redis
    container_name: redis
    ports:
      - "6380:6379"
    command: ["redis-server", "--requirepass", "123456"]

Then use the following command to start the Redis container:

docker compose up --build

The Redis container will be up running and can be accessed through port 6380 on the local machine. The password is 123456. We can use the following command to check the status of the containers running on the local machine:

docker ps

It is seen that the Redis container is up running:

Basic Operations

To manipulate Redis through Python, the corresponding Python package needs to be installed:

pip install redis

A Redis client can be created as follows:

r = redis.StrictRedis(
    host='localhost', 
    port=6380, 
    db=0, 
    password='123456', 
    decode_responses=True
)

Alternatively, a Redis client can also be created from a connection pool:

pool = redis.ConnectionPool(
    host='localhost', 
    port=6380, 
    db=0, 
    password='123456', 
    decode_responses=True
)
r = redis.StrictRedis(connection_pool=pool)

Set a variable in Redis:

r.set('a', 0)

Retrieve the value of the variable:

r.get('a')

The result is:

'0'

Check if a variable exists in the current db:

r.exists('a')

The result is:

1

There are several useful parameters for the set method in Redis, for example:

import time

# ex stands for expire
r.set('b', 'apple', ex=3)
print(r.get('b'))
time.sleep(4)
print(r.get('b'))

In the above code, the ex parameter sets the expiration time of the variable. The result of the above code snippet is:

apple
None

It is seen that the variable expires after the expiration time.

The nx parameter sets a variable if it doesn't exist:

# nx stands for set if not exist
r.set('c', 'banana', nx=True)  # Returns True
r.get('c')

The result is:

'banana'

Try to set the value of an existing variable with the nx parameter equals to True:

r.set('c', 'melon', nx=True)  # Returns None
r.get('c')

The result is:

'banana'

The xx parameter sets a variable if it already exists:

# xx stands for set if already exist
r.set('c', 'peach', xx=True)  # Returns True
r.get('c')

The result is:

'peach'

Try to set the value of an non-existing variable with the xx parameter equals to True:

r.set('d', 'tomato', xx=True)  # Returns None
print(r.get('d'))

The result is:

None

The get parameter returns the previous value of the variable:

r.set('c', 'orange', get=True)

The result is:

'peach'

The mset and mget methods can be used to set and get the values of multiple variables:

r.mset({
    'k1': 'v1',
    'k2': 'v2'
})

r.mget(['k1', 'k2'])

The result is:

['v1', 'v2']

The getrange method can be used to retrieve a substring of the value of a variable:

# getrange
r.set('player_name', 'Kobe Bryant')
r.getrange('player_name', 0, 2)

The above code retrieves a substring starting from the 1st index to the 3rd index. The result is:

'Kob'

-1 represents the index of the last character:

r.getrange('player_name', 0, -1)

The result is:

'Kobe Bryant'

The setrange method can be used to replace part of the string starting from an index:

# setrange
r.setrange('player_name', 1, 'ccc')
r.get('player_name')

The result is:

'Kccc Bryant'

The strlen method can be used to retrieve the length of a string:

# strlen
r.strlen('player_name')

The result is:

11

The incr method can be used to increase the value of a variable if it can be converted into an integer:

# incr
r.set('num', 123)
result = r.incr('num', amount=1)
result

The result is:

124

Which is an integer.

If we get the value of num again:

r.get('num')

The result is:

'124'

Which is a string.

The incrbyfloat method can be used to increase the value of a variable if it can be converted into a float:

# incrbyfloat
r.set('float_num', '123.0')
result = r.incrbyfloat('float_num', amount=2.5)
result

The result is:

125.5

Which is a float.

If we get the value of float_num again:

r.get('float_num')

The result is:

'125.5'

Which is still a string.

The decr method is opposite to the incr method:

# decr
result = r.decr('num', amount=2)
result

The result is:

122

The decr method is equivalent to passing a negative value to the incr method:

result = r.incr('num', amount=-5)
result

The result is:

117

The append method can be used to append two strings together:

# append
r.set('name', 'kobe')
r.append('name', 'haha')
r.get('name')

The result is:

'kobehaha'

Async Redis

To use Redis with Python in an async way, the aioredis package needs to be installed:

pip install aioredis

The following code shows how to use aioredis:

import asyncio
import aioredis

async def main():
    try:
        r = aioredis.from_url(
            "redis://:123456@localhost:6380/0",
            encoding="utf-8",
            decode_responses=True
        )
        await r.set("key", "value")
        result = await r.get("key")
        print(f"result: {result}")
    except Exception as e:
        print(f"Operation failed due to: {e}.")

asyncio.run(main())

Here instead of using parameters, a URI is used for creating an async Redis client. The result of the above code snippet is:

result: value

Conclusions

In this article, we went through how to use Python to manipulate Redis and some basic operations. Hope you enjoy it! More contents like hash and list related operations in Redis with Python will be covered in the Part II. Stay tuned!

Get in Touch

Thank you for reading! Please let us know if you like this series or if you have critiques. If this series was helpful to you, please follow us and share this series to your friends.

If you or your company needs any help on projects related to drilling automation and optimization, AI, and data science, please get in touch with us Nvicta AI. We are here to help. Cheers!