A Simple Fix for Django "unable to read data" with pgpool2

I recently noticed occasional errors in a Django project that uses PostgreSQL together with pgpool2. The log looked like this:

psycopg2.OperationalError: unable to read data
DETAIL:  child connection forced to terminate due to client_idle_limit:10 is reached
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.

From the error message, it was clear that pgpool2 had client_idle_limit set too aggressively. Idle connections were being closed after 10 seconds, but Django did not realize the connection had been terminated and kept trying to use it.

In our project, this usually showed up in asynchronous jobs such as Celery tasks. A task would connect to the database, fetch some information, then perform a long-running operation that might take more than 10 seconds, and finally use the ORM again. Since Django manages the connection internally, the stale connection caused exactly the error above.

Because of company policy, we could not change pgpool2’s client_idle_limit setting. Django’s CONN_MAX_AGE setting also did not help in this case, because the real issue was that Django could not detect that pgpool2 had already dropped the connection.

After some searching, I found a simple workaround: manually close Django’s database connections before using the database again, so Django reconnects cleanly.

1
2
3

from django import db

db.connections.close_all()

Here is a small example to show how that helps.

First, create a model for testing:

from django.db import models


class TestPgpool(models.Model):
    secs = models.IntegerField(help_text='sleep seconds')
    status = models.CharField(max_length=16, help_text='set start when test starts and set end when test ends')
    create_time = models.DateTimeField(auto_now_add=True)
    update_time = models.DateTimeField(auto_now=True)

Then create a simple view:

import time
from django import db
from django.http import JsonResponse
from .models import TestPgpool


def test_pgpool(request):
    """ test write data after a sleep time """
    secs = int(request.GET.get('secs'))

    obj = TestPgpool(secs=secs, status='start')
    obj.save()

    db.connections.close_all()

    time.sleep(secs)

    obj.status = 'end'
    obj.save()

    return JsonResponse({'status_code': 0, 'sleep_secs': secs, 'id': obj.id})

This view accepts a secs parameter that controls how long it waits between two database operations. If you do not manually close the connection, requests that sleep longer than client_idle_limit will fail. If you close the connection first, Django reconnects and the error disappears.

This is only a simple workaround. A more complete solution would require improving how Django handles dropped database connections, or adding connection retry logic.