Better way writing logs using logger

As a Data Scientist Programmer, you might have to build a project based on machine learning, deep learning, and other related projects. It’s easy to compile and see the output of a project are according to your requirement.

During development phase most of the time we print the output and verify. But in the production phase, it’s very important to keep track of logs in a file while the program is running. Otherwise, you will not be able to know which modules or lines of code have got errors and problems. This leads to high time consuming to fix the errors if the program crashes. In this post, you will know how to use logging in python to keep records of the occurred events within a program.

We created solution works both environments development and production. Basic idea was to maintain configuration file. Read all logging related information from this config. So, it is easy to modify any point of time by developer. In development environment we can keep different config value as per our requirement.

In our solution we have 2 main files

1.     Logging.conf 

2.     Loghelper.py

Logging.conf

Main important part is to define loggers, handlers and formatters

[loggers]  # unique name

keys=root,simpleExample,filelogExample,timedRotatingFileHandler

[handlers] # for to define different logs type (like console or file)

keys=consoleHandler,fileHandler,timedRotatingFileHandler

[formatters] # message formatter

keys=simpleFormatter

In my case, I defined 3 types

1.     simpleExample to log errors on console

2.     filelogExample to log errors on specific log file

3.     timedRotatingFileHandler to log errors on to specific file but every midnight creates new file.

Loghelper.py

This is simple file, we are reading conf file here.

import logging

import logging.config

logging.config.fileConfig('logging.conf')

employee.py

Import loghelper, create logger object using getLogger and specify the name of the logger you want to use.

import loghelper

import logging

import traceback

import time

logger = logging.getLogger('filelogExample')

based on the config values all logs goes to python.log file.

logger = logging.getLogger('timedRotatingFileHandler')

exception logging

import loghelper

import logging

import traceback

logger.error("uncaught exception: %s", traceback.format_exc())

or

logger.exception("exception")

Uploaded files to github, check url here