Snake_Byte #24: Working with MongoDB replica sets and PyMongo

In this post we'll discuss how using PyMongo can simplify working with a MongoDB replica set.

First, start up a MongoDB replica set. This article demonstrates how to run the replica set using Docker.

You can also run a Python Docker container on the same network as the replica set by doing:

Now that we have a replica set configured, we'll use PyMongo to connect to it

Let's create a class to interact with a MongoDB collection.

Now that we have a class for our MongoDB collection, we can save and retrieve docs

Check your MongoDB output and you can see that the bread document has been copied to the secondary nodes. So far, our application has only needed URIs and a replicaSet config to interact with the replica set.

In some scenarios, like during failover, we'll need to handle connection errors. If the MongoDB primary node fails and another node is promoted, the client may have trouble reconnecting. When this happens, PyMongo raises an AutoReconnect error.

Since every method in our BreadsCollection class can raise this exception, we'll need to decorate
each one with an exception handler.

We can write a method that catches AutoReconnect exceptions and retries the database calls. We'll also limit the handler to 3 retry attempts and give the cluster a few seconds to finish recovering between each retry. If the handler fails we'll raise the last exception.

We can add the error handling to our MongoClient class methods by using the decorator like so:

Now the application is a little more resilient even during MongoDB failover.

Our generic database transaction class paired with PyMongo allows us to deal with more complex connections in a simple way. It also lets us focus solely on application logic when developing new features.