• 2023/08/11

A Python Logging Gotcha

While running a containerized web app, I wanted most logs to go to stdout (for docker logs) but some cases to write to files. Libraries handled request logging nicely, so I assumed I could configure both destinations easily.

Setup

logging.basicConfig with stream=sys.stdout sends logs to stdout. Likewise, filename=... sends them to a file. I tried to use both, expecting one logger per destination. Instead, everything went to the file—stdout was silent. Libraries that relied on basicConfig adopted the file handler as well, adding to the confusion.

Root cause

basicConfig is global. You cannot call it twice to set up different handlers; the first call wins. Re-calling it inside logger configuration does nothing, and libraries that respect the global config inherit the file-only behavior.

Fix

Skip basicConfig and configure handlers explicitly:

import logging
import sys

# stdout logger
stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.setFormatter(logging.Formatter("%(message)s"))
stdout_logger = logging.getLogger("stdout-logger")
stdout_logger.setLevel(logging.INFO)
stdout_logger.addHandler(stdout_handler)

# file logger
file_handler = logging.FileHandler("/var/log/xxx/xxx.log", mode="a")
file_handler.setFormatter(logging.Formatter("%(message)s"))
file_logger = logging.getLogger("file-logger")
file_logger.setLevel(logging.INFO)
file_logger.addHandler(file_handler)

Once I did that, stdout and file logging behaved as intended. Maybe there is a clever way to juggle multiple configurations with basicConfig, but explicit handlers are clearer and only need to be set once.