Python Shelve Module

We all have come across so many instances in our daily life where we have to use a database management system to store all the required information and data. It is often very simple to use a database management system, but sometimes it becomes a bit difficult when we have to store some complex structured data. Database Management Systems are various types, but the most common type we usually use in our daily lives is Relational Database Management System (RDBMS). RDBMS is an effective tool and management system where we can store all our necessary information & data and manage them very easily. There are many instances when we don’t have to use Relational Database Management System, or it is not possible to use RDBMS to store data. In such cases, some questions come to our mind: are there any other alternatives in this situation, are there other methods or tools to store data, etc.

The answer to these questions is Yes! We have other tools & methods through which we can store data when we don’t have to use the RDBMS method. We have other tools, and many of these tools are associated with the library of a programming language. It means that, in the programming world, there are libraries that provide effective tools to use when using Relational Database Management System is not an option. This thing is also applicable to Python, and we have libraries in Python through which we can store all our important data by creating a DBMS-like interface. One of such libraries or modules in Python is Shelve Module, an effective tool for storing or persistent data storage. We will learn about this Shelve Module of Python in this tutorial, where we store data inside using Shelve Module inside a Python program.

Shelve Module in Python

The Shelve Module of Python is a very popular module of Python which works like an effective tool for persistent data storage inside files using a Python program. As the name of this module suggests, i.e., Shelve, we can easily interpret that it will work as a shelf object to keep all our data inside a file and save all the necessary information. In the Shelve Module, a shelf object is defined, which acts like a dictionary-type object, and it is persistently stored in the disk file of our computer. That’s how we can save all the data and information through Python Shelve Module in our system and keep it till whenever we want.

Shelve Module: Introduction

Shelve Module of Python is an effective tool through which we can store all the data and information inside a file and keep it in our computer’s storage until we change our mind to remove it. Shelve Module is not only helpful in storing information inside a file, but it is also very effective in modifying the already present information and adding some new information in the same file. We can perform all these operations (creating, reading, writing, and deleting from a file) from the Shelve Module by using this module and its functions inside a Python program. The Shelve Module creates a very similar file to the DBMS database present on systems such as UNIX. We can only use string data type as the keys in the special dictionary object present in the file, whereas the pickable objects from the shelve files can serve as the value for the keys.

Shelve Module: Classes

Like any other Python module, there are some classes present in the Shelve Module of Python, too, and we are going to discuss these classes in this section. The following three classes are defined inside Python Shelve Module:

Now, here we have described all the classes and sub-classes present inside the Shelve Module, and we have also learned which object type is taken as a parameter inside these classes.

Working with the Shelve Module

Shelve Module is an in-built library of Python, and one can find this module pre-installed in all versions of Python 2 and 3. Therefore, for working with Shelve Module, we don’t have to use any installer or pip command first to install the Shelve Module. We can directly use the ‘import shelve’ statement in a Python program to import and use all functions of the Shelve module in that program. To understand the working of the Shelve Module, we will divide the implementation part of it into three parts where first we create a shelve file and store data in it, secondly, we open the file and retrieve values from it, and in last, we update data inside the shelve file. We will learn the working and implementation of the Shelve Module through the following three steps:

• Creating a shelve file to store data
• Retrieving Data from the Shelve file
• Updating data in the Shelve file

We will learn about each of the three parts as mentioned above by using an example program where we perform the respective tasks using the functions of the Shelve Module. Now, let’s learn the implementation of each part of the working of the Shelve Module by first creating a Sample Shelve file.

Creating a Shelve File to Store Data:

If we want to store some data inside the shelve file, we have first to create one, and only then can we add the desirable data. We can create a shelve file by simply using its open() function, which works similarly to the open() function we use during the file handling process. With the open() function of Shelve Module, we have to follow the given below syntax to open or create a new shelve file:

It will return a shelf object from the DbfilenameShelf sub-class, where the object will collect the data from the user to be stored in the file. Following is the description of the parameters used inside the open() function’s syntax given above:

• NameOfFile: It is the name of the file which we want to or have assigned to the database (shelve file) we are creating/opening using the open()
• flag: The flag parameters for the open() function’s syntax is c, w, n, and r, in which ‘c’ is the default one. The ‘c’ keyword in the flag parameter represents we gave read and write access for the file, ‘w’ represents we gave write-only access, ‘r’ represents we gave only read access, and ‘n’ represents that we are creating a new file with NameOfFile name.
• Writeback: Writeback parameter is false by default, but if we set this parameter as true, the entries we give in the shelve file will be cached in the program’s cache memory.
• Protocol: The protocol parameter in the function denotes the pickle protocol for the entries.

As now we have learned the parameters and syntax of the Shelve Module’s open() function, it is time we should use this function in an example program to understand its implementation.

Example 1:

Look at the following Python program where we created a shelve file and stored data inside that file:

Output:

A Shelve file with SampleFile name is successfully created
All entries are successfully made inside the sample shelve file

Explanation:

We have first imported the Shelve Module in the example program so that we can use the open() function of it to create a sample shelve file. Then, we used the open() function inside the initialized shelveVariable to use it further for entries and created a sample shelve file with the ‘SampleFile’ name. We then used initialized shelveVariable to make entries inside the sample shelve file we created, and we used the dictionary method to make multiple entries in it. We made multiple entries in the separate columns using the initialized variable, and last, we used the close() function to save these entries in the file.

When we run the example program given above, a shelve file with ‘SampleFile.dir’ will be created in our device, and entries will be made in it.

Note: Every time we access the shelve file present in our system, sync() and close() operations are performed along with it, which may slow down the overall process.

Methods in Shelve Module:

In the above example, we used the open() method to create a database shelve file, but this isn’t the only method present in the shelve module. Shelve Module provides us with many different methods, and each one of these methods has different functionality, which is very useful for us.

Following various types of methods are present in the shelve module:

We can use all the methods of shelve module to perform various functions on the database shelve file we created, and we are going to use some of these methods in later examples.

Retrieving Data from the Shelve File:

After creating a shelve file and making data entries into this file, the first thing which we want to do is to check if all the entries are successfully made into the file or not. The best way to check is to print all these entries in the output and see whether all entries are printed. This method is also useful when we don’t know what data is present in a shelve file (cases someone worked on our shelve file or we are working on someone else’s shelve file). We can access data from the Shelve by many methods like get(), values(), items(), and keys(). These all methods work very differently and return the same data but in different forms, and therefore we will use each of them to return data from the sample shelve file we created in example 1.

(i) Getting single value by get() method:

We can directly print the value from the particular key by providing a key inside the get() method, and to understand this method, we will use this in an example program.

Example 2:

Output:

Name of Employee in the file:  Jackie Chan
Age of Employee:  27
Department of Employee:  Management
Score of Employee:  9.2

As we can see, the data value with the respective key given in the get() method is printed in the output.

(ii) Getting all items at once from shelve file:

If we want to retrieve all items from the shelve file at once, we can use the items() method with the shelve object, and it will return all the items present inside the sample shelve file in the output. We will use the following example program to understand the implementation of this method:

Example 3:

Output:

Items in the sample shelve file:  [('Employee Name', 'Jackie Chan'), ('Employee Age', 27), ('Employee department', 'Management'), ('Employee Performance', 'Excellent'), ('Employee Score', 9.2)]

As we can see in the output, all the items present in the sample file were printed when we used the items() function.

(iii) Retrieving only keys from shelve file:

If we want to print only keys present in the sample shelve file, we can use the keys() method inside the program after opening the shelve file.

Example 4:

Output:

Keys in the sample shelve file:  ['Employee Name', 'Employee Age', 'Employee department', 'Employee Performance', 'Employee Score']

All the keys present in the sample shelve file are printed in the output with the keys() method.

(iv) Retrieving only values from shelve file:

This method and its implementation work exactly like the previous method where we used the keys() function, but here we have to use the values() function to print all values from the shelve file.

Example 5:

Output:

Values in the sample shelve file:  ['Jackie Chan', 27, 'Management', 'Excellent', 9.2]

All the values present in the sample shelve file are printed in the output with the keys() method.

Updating Data in the Shelve File:

We can even update the data already present in the shelve file, and in this method, we can perform three tasks, but here we will perform only two. First, we can update the value of a key already present in the file; second, we can add new entries; and last, we can remove an entry from the file. If we want to update the current value associated with a key to a new value, we have to write the new value with the key object, and the program will assign the new value in that key.

(i) Adding new entries in the shelve file:

If we want to add new entries, we first have to define a dictionary variable in the program, and then we can write new entries there with key-value pair. After that, we can use the update() method by providing the name of the dictionary variable as an argument in the method, and it will add new entries in the file.

Example 6:

Output:

Items in the sample shelve file:  [('Employee Name', 'Jackie Chan'), ('Employee Age', 27), ('Employee department', 'Management'), ('Employee Performance', 'Excellent'), ('Employee Score', 9.2), ('Employee Salary', 50000), ('Employee Sub-department', 'Field Work')]

As we can see, when we printed all items from the sample shelve file, new items were also present, which means new entries were made in the file.

(ii) Removing an entry from the shelve file:

If we want to remove an already existing entry in the file, we can use the pop() method. We have to write the key name in the function, and it will remove the key and value associated with it from the shelve file. We can verify this through by printing all the items from the shelve file.

Example 7:

Output:

Items in the sample shelve file:  [('Employee Name', 'Jackie Chan'), ('Employee Age', 27), ('Employee department', 'Management'), ('Employee Score', 9.2), ('Employee Salary', 50000), ('Employee Sub-department', 'Field Work')]

As we can see, the employee performance key-value entry is not present in the output as we have removed it through the pop() method.