DevOps Zone is brought to you in partnership with:

I have been programming in Python since 2006 and writing about Python almost as long on my blog. I also enjoy apologetics, reading, and photography. Mike is a DZone MVB and is not an employee of DZone and has posted 70 posts at DZone. You can read more from them at their website. View Full User Profile

Python Logging: How to Log to Multiple Locations

07.19.2013
| 5625 views |
  • submit to reddit

Today I decided to figure out how to make Python log to a file and the console simultaneously. Most of the time, I just want to log to a file, but occasionally I want to be able to see stuff on the console too to help with debugging. I found this ancient example in the Python documentation and ended up using it to mock up the following script:

import logging
 
#----------------------------------------------------------------------
def log(path, multipleLocs=False):
    """
    Log to multiple locations if multipleLocs is True
    """
    fmt_str = '%(asctime)s - %(name)s - %(message)s'
    formatter = logging.Formatter(fmt_str)
 
    logging.basicConfig(filename=path, level=logging.INFO,
                        format=fmt_str)
 
    if multipleLocs:
        console = logging.StreamHandler()
        console.setLevel(logging.INFO)
        console.setFormatter(formatter)
 
        logging.getLogger("").addHandler(console)
 
    logging.info("This is an informational message")
    try:
        1 / 0
    except ZeroDivisionError:
        logging.exception("You can't do that!")
 
    logging.critical("THIS IS A SHOW STOPPER!!!")
 
if __name__ == "__main__":
    log("sample.log") # log only to file
    log("sample2.log", multipleLocs=True) # log to file AND console!

As you can see, when you pass True to the second argument, the script will create an instance ofStreamHandler() which you can then configure and add to the current logger via the following call:

logging.getLogger("").addHandler(console)

This works great on Linux, but on Windows 7 the sample2.log wasn’t getting created, so I had to modify the if statement as follows:

if multipleLocs:
    console = logging.StreamHandler()
    console.setLevel(logging.INFO)
    console.setFormatter(formatter)
 
    fhandler = logging.FileHandler(path)
    fhandler.setFormatter(formatter)
 
    logging.getLogger("").addHandler(console)
    logging.getLogger("").addHandler(fhandler)

Now I should note that this causes a rather odd bug in that Python is somehow keeping track of the file names I write to across calls to my log function such that when I tell it to write to sample2.log, it write to it PLUS the original sample.log. To get around this, we have to create a logger instance with a unique name each time we call the script. Here’s an updated example that works correctly:

import logging
import os
 
#----------------------------------------------------------------------
def log(path, multipleLocs=False):
    """
    Log to multiple locations if multipleLocs is True
    """
    fname = os.path.splitext(path)[0]
    logger = logging.getLogger("Test_logger_%s" % fname)
    logger.setLevel(logging.INFO)
    fh = logging.FileHandler(path)
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(message)s')
    fh.setFormatter(formatter)
    logger.addHandler(fh)
 
    if multipleLocs:
        console = logging.StreamHandler()
        console.setLevel(logging.INFO)
        console.setFormatter(formatter)
        logger.addHandler(console)
 
    logger.info("This is an informational message")
    try:
        1 / 0
    except ZeroDivisionError:
        logger.exception("You can't do that!")
 
    logger.critical("THIS IS A SHOW STOPPER!!!")
 
if __name__ == "__main__":
    log("sample.log") # log only to file
    log("sample2.log", multipleLocs=True) # log to file AND console!

You will note that this time we base the logger name on the file name of the log. The logging module is pretty slick and lots of fun to play around with. I hope you found that as interesting as I did.

Published at DZone with permission of Mike Driscoll, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)