Assumptions are sneaky

This blog post is a weekly update of my journey in Djangonaut Space , an 8-week mentoring program for future open source developers in the Django ecosystem.

The number 1 thing I always talk and write about in regards to debugging is to leave assumptions at the door and validate everything before following your assumptions. Yesterday, I failed miserably to follow my own advice.

I fell into the classic trap: I thought I knew how a thing works.

While I was helping a team mate to set up her development environment, we discovered a broken test. I had seen it before and had done a workaround for my own setup but I felt it was a good idea now to fix it so others wouldn’t have to. I hate telling people “just do this trick and ignore it” while ignoring the issue. These things have a tendency to pile up and become problematic down the line.

Here’s what we had:

import pytest
from django.test import override_settings
from django.urls import reverse


STRIPE_PUBLIC_KEY = os.environ.get("STRIPE_PUBLIC_KEY", "test_public")
STRIPE_SECRET_KEY = os.environ.get("STRIPE_SECRET_KEY", "test_public")


def test_donate(client):
    with override_settings(STRIPE_PUBLIC_KEY=STRIPE_PUBLIC_KEY, STRIPE_SECRET_KEY=STRIPE_SECRET_KEY):
        resp = client.get(reverse("donations:donate"))
        assert resp.status_code == 200

        # Check if it returns a list of past and future events
        assert "form" and "STRIPE_PUBLIC_KEY" in resp.context

And the test was failing. The function at donations:donate was:

def donate(request):
    context = {
        "patreon_stats": FundraisingStatus.objects.first(),  # TODO: This isn't used
    }
    if settings.STRIPE_PUBLIC_KEY:
        context.update({"form": StripeForm(), "STRIPE_PUBLIC_KEY": settings.STRIPE_PUBLIC_KEY})
    return render(request, "donations/donate.html", context)

Hence, if the STRIPE_PUBLIC_KEY was not set, we’d skip setting up the form and public key in context.

Reading through this code, here’s what goes through in my mind:

1. In the test, if a key is missing, we set it to a default test value
2. With django.test.override_settings , we change the settings.STRIPE_PUBLIC_KEY value to the test value

Here’s where I made my mistake by assuming: I thought I knew how os.environ.get works and only ever tested the values at settings object. They never changed.

I even went down the rabbit hole of finding that instead of using override_settings, with pytest-django you can use settings fixture and change that. I thought that was the solution because a test temporarily passed but when I started setting up a pull request for the fix, they started failing again.

That’s when I realised I should check what STRIPE_PUBLIC_KEY was and found out, it was always empty string. It was never set to “test_public”.

The thing is, os.environ.get doesn’t fall to default value on falsy values as I imagined but only if the value is None which makes so much sense now but never crossed my mind while debugging this.

I changed it to check for falsy value with

STRIPE_PUBLIC_KEY = os.environ.get("STRIPE_PUBLIC_KEY") or "test_public"
STRIPE_SECRET_KEY = os.environ.get("STRIPE_SECRET_KEY") or "test_public"

which now properly uses those values when testing if the key is missing.

Remember kids: always validate your assumptions! Never go chase a wild goose before covering all your bases at home. You can save so much time and effort.