Data Science in Drilling - Episode 15

Manipulating Redis with Python - Part II

written by Zeyu Yan, Ph.D., Head of Data Science from Nvicta AI

Data Science in Drilling is a multi-episode series written by the technical team members in Nvicta AI. Nvicta AI is a startup company who helps drilling service companies increase their value offering by providing them with advanced AI and automation technologies and services. The goal of this Data Science in Drilling series is to provide both data engineers and drilling engineers an insight of the state-of-art techniques combining both drilling engineering and data science.

This is the second episode of the Python-Redis series. Please refer to the first episode for more details. :)

Enjoying great knowledge is just like enjoying a delicious piece of pie.

Introduction

In the first episode of the Python-Redis series, we covered how to use Python to manipulate Redis and some basic operations. In this episode, we will continue to cover how to manipulate Hash and List in Redis using Python.

What We'll Cover Today

1. How to manipulate Hash in Redis using Python.

2. How to manipulate List in Redis using Python.

Hash in Redis

The development environment is exactly the same as the first episode of this series. Please refer to the first episode for more details on the setup of the environment. Let’s try to create a hash in Redis and insert two key-value pairs:

r.hset('hash1', 'k1', 'v1')
r.hset('hash1', 'k2', 'v2')

Retrieve the value of k1 in hash1:

r.hget('hash1', 'k1')

The result is:

'v1'

Check if hash1 exists:

r.exists('hash1')

The result is:

1

Get all the keys in hash1:

r.hkeys('hash1')

The results are:

['k1', 'k2']

Get all the values in hash1:

r.hvals('hash1')

The results are:

['v1', 'v2']

Get multiple values by keys at once:

r.hmget('hash1', 'k1', 'k2')

The results are:

['v1', 'v2']

Instead of passing each key separately, a list of keys can also be passed:

r.hmget('hash1', ['k1', 'k2'])

The results remain the same.

The hsetnx method sets the value of a key if its value doesn't exist:

# set if not exist
r.hsetnx('hash1', 'k2', 'v2_new')

The result is 0 since the value of k2 already exists. Try to retrieve the value of k2 again:

r.hget('hash1', 'k2')

The result is:

'v2'

A hash can also be created using the mapping parameter to pass a Python Dictionary:

r.hset('hash2', mapping={'k2': 'v2', 'k3': 'v3'})

Retrieve the values of keys k2 and k3 in hash2:

r.hmget('hash2', ['k2', 'k3'])

The results are:

['v2', 'v3']

Change the values of k2 and k3 in hash2:

r.hset('hash2', mapping={'k2': 'a', 'k3': 'b'})

Check if their values have changed:

r.hmget('hash2', ['k2', 'k3'])

The results are:

['a', 'b']

Get all the keys and values from hash2:

r.hgetall('hash2')

The result is a Python Dictionary:

{'k2': 'a', 'k3': 'b'}

Delete key k2 from hash2:

r.hdel('hash2', 'k2')

Check if key k2 still exists in hash2:

r.hexists('hash2', 'k2')

The result is:

False

Add a new key k3 in hash1:

r.hset('hash1', 'k3', 123)
r.hgetall('hash1')

The result is:

{'k1': 'v1', 'k2': 'v2', 'k3': '123'}

Decrease the value of key k3 by 1:

result = r.hincrby('hash1', 'k3', amount=-1)
result

The result is:

122

Increasing/decreasing the value of a non-existing key will create the key and set its value:

result = r.hincrby('hash1', 'k4', amount=1)
print(f'result: {result}')
print(r.hgetall('hash1'))

The results are:

result: 1
{'k1': 'v1', 'k2': 'v2', 'k3': '122', 'k4': '1'}

The hincrbyfloat method can be used to increase or decrease the value of a key whose value is a float:

r.hset('hash1', 'k5', '10.0')
result = r.hincrbyfloat('hash1', 'k5', amount=-1.4)
print(f'result: {result}')
print(r.hgetall('hash1'))

The results are:

result: 8.6
{'k1': 'v1', 'k2': 'v2', 'k3': '122', 'k4': '1', 'k5': '8.6'}

Increasing/decreasing the value of a non-existing key using the hincrbyfloat method will also create the key and set its value:

result = r.hincrbyfloat('hash1', 'k6', amount=-2.6)
print(f'result: {result}')
print(r.hgetall('hash1'))

The results are:

result: -2.6
{'k1': 'v1', 'k2': 'v2', 'k3': '122', 'k4': '1', 'k5': '8.6', 'k6': '-2.6'}

Loop through a hash and print all its key-value pairs:

for item in r.hscan_iter('hash1'):
    print(item)

The results are:

('k1', 'v1')
('k2', 'v2')
('k3', '122')
('k4', '1')
('k5', '8.6')
('k6', '-2.6')

Lastly, a hash can be deleted as follows:

# delete whole hash
r.delete('hash2')

List in Redis

Create a list list1 in Redis and push 3 elements into the list from left:

# push from the left
r.lpush('list1', 1, 2, 3)

Check the length of the list:

r.llen('list1')

The result is:

3

Loop through the first and the last element of the list:

r.lrange('list1', 0, -1)

The result is a Python List:

['3', '2', '1']

Create a new list list2 and push 3 elements into the list from right:

# push from the right
r.rpush('list2', 1, 2, 3)

Loop through the first and the last element of list2:

r.lrange('list2', 0, -1)

The result is:

['1', '2', '3']

Create a new list list3 and push multiple elements into it:

r.rpush('list3', 1, 2, 3, 4, 5, 6)

Loop through the first and the fourth element of list3:

r.lrange('list3', 0, 3)

The result is:

['1', '2', '3', '4']

If the second index passed is greater than the largest index of the list, then all the elements in the list will be included:

r.lrange('list3', 0, 10)

The result is:

['1', '2', '3', '4', '5', '6']

The lpushx and rpushx methods can be used to push elements to the list if the list exists:

# push if the list exists
r.lpushx('list4', 10)

The result is 0 since list4 does not exist.

Get the length of a non-existing list:

r.llen('list4')

The result is:

0

Retrieve a non-existing list:

r.lrange('list4', 0, -1)

The result is an empty list:

[]

Push a new element to an existing list list2:

r.lpushx('list2', 77)
r.lrange('list2', 0, -1)

The result is:

['77', '1', '2', '3']

Insert a 0 to list2 before 1:

r.linsert('list2', 'before', '1', '0')
r.lrange('list2', 0, -1)

The result is:

['77', '0', '1', '2', '3']

Change the value of the first element in list2 to 100:

# set the value of an element
r.lset('list2', 0, -100)
r.lrange('list2', 0, -1)

The result is:

['-100', '0', '1', '2', '3']

Remove all the '1' element in list2:

r.lrem('list2', 0, '1')
r.lrange('list2', 0, -1)

The result is:

['-100', '0', '2', '3']

Get more details about the parameters in the lrem method from Redis' offical documents.

Delete an element from list2 from the left and get its value:

result = r.lpop('list2')
print(f'result: {result}')
r.lrange('list2', 0, -1)

The results are:

result: '-100'
['0', '2', '3']

Delete an element from list2 from the right and get its value:

result = r.rpop('list2')
print(f'result: {result}')
r.lrange('list2', 0, -1)

The results are:

result: '3'
['0', '2']

Trim list3 to a specific range (element 1 through element 3, inclusively):

# trim to a specific range
r.ltrim('list3', 1, 3)
r.lrange('list3', 0, -1)

The result is:

['2', '3', '4']

Retrieve element by index:

r.lindex('list3', 0)

The result is:

'2'

Lastly, to iterate through a list in Redis, define the following helper function:

def list_iter(name: str):
    list_count = r.llen(name)
    for index in range(list_count):
        yield r.lindex(name, index)

Then iterate through a list using the helper function:

for item in list_iter('list3'):
    print(item)

The results are:

2
3
4

Conclusions

In this article, we went through how to use Python to manipulate Redis' Hash and List. Hope you enjoy it! More interesting contents will be covered in the future episodes. Stay tuned!

Get in Touch

Thank you for reading! Please let us know if you like this series or if you have critiques. If this series was helpful to you, please follow us and share this series to your friends.

If you or your company needs any help on projects related to drilling automation and optimization, AI, and data science, please get in touch with us Nvicta AI. We are here to help. Cheers!