2589 lines
294 KiB
HTML
2589 lines
294 KiB
HTML
|
|
|||
|
|
<!DOCTYPE html>
|
|||
|
|
|
|||
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|||
|
|
<head>
|
|||
|
|
<meta charset="utf-8" />
|
|||
|
|
<title>Logging Cookbook — Python 3.7.4 documentation</title>
|
|||
|
|
<link rel="stylesheet" href="../_static/pydoctheme.css" type="text/css" />
|
|||
|
|
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
|
|||
|
|
|
|||
|
|
<script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
|
|||
|
|
<script type="text/javascript" src="../_static/jquery.js"></script>
|
|||
|
|
<script type="text/javascript" src="../_static/underscore.js"></script>
|
|||
|
|
<script type="text/javascript" src="../_static/doctools.js"></script>
|
|||
|
|
<script type="text/javascript" src="../_static/language_data.js"></script>
|
|||
|
|
|
|||
|
|
<script type="text/javascript" src="../_static/sidebar.js"></script>
|
|||
|
|
|
|||
|
|
<link rel="search" type="application/opensearchdescription+xml"
|
|||
|
|
title="Search within Python 3.7.4 documentation"
|
|||
|
|
href="../_static/opensearch.xml"/>
|
|||
|
|
<link rel="author" title="About these documents" href="../about.html" />
|
|||
|
|
<link rel="index" title="Index" href="../genindex.html" />
|
|||
|
|
<link rel="search" title="Search" href="../search.html" />
|
|||
|
|
<link rel="copyright" title="Copyright" href="../copyright.html" />
|
|||
|
|
<link rel="next" title="Regular Expression HOWTO" href="regex.html" />
|
|||
|
|
<link rel="prev" title="Logging HOWTO" href="logging.html" />
|
|||
|
|
<link rel="shortcut icon" type="image/png" href="../_static/py.png" />
|
|||
|
|
<link rel="canonical" href="https://docs.python.org/3/howto/logging-cookbook.html" />
|
|||
|
|
|
|||
|
|
<script type="text/javascript" src="../_static/copybutton.js"></script>
|
|||
|
|
<script type="text/javascript" src="../_static/switchers.js"></script>
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
<style>
|
|||
|
|
@media only screen {
|
|||
|
|
table.full-width-table {
|
|||
|
|
width: 100%;
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
</style>
|
|||
|
|
|
|||
|
|
|
|||
|
|
</head><body>
|
|||
|
|
|
|||
|
|
<div class="related" role="navigation" aria-label="related navigation">
|
|||
|
|
<h3>Navigation</h3>
|
|||
|
|
<ul>
|
|||
|
|
<li class="right" style="margin-right: 10px">
|
|||
|
|
<a href="../genindex.html" title="General Index"
|
|||
|
|
accesskey="I">index</a></li>
|
|||
|
|
<li class="right" >
|
|||
|
|
<a href="../py-modindex.html" title="Python Module Index"
|
|||
|
|
>modules</a> |</li>
|
|||
|
|
<li class="right" >
|
|||
|
|
<a href="regex.html" title="Regular Expression HOWTO"
|
|||
|
|
accesskey="N">next</a> |</li>
|
|||
|
|
<li class="right" >
|
|||
|
|
<a href="logging.html" title="Logging HOWTO"
|
|||
|
|
accesskey="P">previous</a> |</li>
|
|||
|
|
<li><img src="../_static/py.png" alt=""
|
|||
|
|
style="vertical-align: middle; margin-top: -1px"/></li>
|
|||
|
|
<li><a href="https://www.python.org/">Python</a> »</li>
|
|||
|
|
<li>
|
|||
|
|
<span class="language_switcher_placeholder">en</span>
|
|||
|
|
<span class="version_switcher_placeholder">3.7.4</span>
|
|||
|
|
<a href="../index.html">Documentation </a> »
|
|||
|
|
</li>
|
|||
|
|
|
|||
|
|
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Python HOWTOs</a> »</li>
|
|||
|
|
<li class="right">
|
|||
|
|
|
|||
|
|
|
|||
|
|
<div class="inline-search" style="display: none" role="search">
|
|||
|
|
<form class="inline-search" action="../search.html" method="get">
|
|||
|
|
<input placeholder="Quick search" type="text" name="q" />
|
|||
|
|
<input type="submit" value="Go" />
|
|||
|
|
<input type="hidden" name="check_keywords" value="yes" />
|
|||
|
|
<input type="hidden" name="area" value="default" />
|
|||
|
|
</form>
|
|||
|
|
</div>
|
|||
|
|
<script type="text/javascript">$('.inline-search').show(0);</script>
|
|||
|
|
|
|
|||
|
|
</li>
|
|||
|
|
|
|||
|
|
</ul>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<div class="document">
|
|||
|
|
<div class="documentwrapper">
|
|||
|
|
<div class="bodywrapper">
|
|||
|
|
<div class="body" role="main">
|
|||
|
|
|
|||
|
|
<div class="section" id="logging-cookbook">
|
|||
|
|
<span id="id1"></span><h1>Logging Cookbook<a class="headerlink" href="#logging-cookbook" title="Permalink to this headline">¶</a></h1>
|
|||
|
|
<dl class="field-list simple">
|
|||
|
|
<dt class="field-odd">Author</dt>
|
|||
|
|
<dd class="field-odd"><p>Vinay Sajip <vinay_sajip at red-dove dot com></p>
|
|||
|
|
</dd>
|
|||
|
|
</dl>
|
|||
|
|
<p>This page contains a number of recipes related to logging, which have been found
|
|||
|
|
useful in the past.</p>
|
|||
|
|
<div class="section" id="using-logging-in-multiple-modules">
|
|||
|
|
<h2>Using logging in multiple modules<a class="headerlink" href="#using-logging-in-multiple-modules" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Multiple calls to <code class="docutils literal notranslate"><span class="pre">logging.getLogger('someLogger')</span></code> return a reference to the
|
|||
|
|
same logger object. This is true not only within the same module, but also
|
|||
|
|
across modules as long as it is in the same Python interpreter process. It is
|
|||
|
|
true for references to the same object; additionally, application code can
|
|||
|
|
define and configure a parent logger in one module and create (but not
|
|||
|
|
configure) a child logger in a separate module, and all logger calls to the
|
|||
|
|
child will pass up to the parent. Here is a main module:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">auxiliary_module</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># create logger with 'spam_application'</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'spam_application'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># create file handler which logs even debug messages</span>
|
|||
|
|
<span class="n">fh</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">FileHandler</span><span class="p">(</span><span class="s1">'spam.log'</span><span class="p">)</span>
|
|||
|
|
<span class="n">fh</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># create console handler with a higher log level</span>
|
|||
|
|
<span class="n">ch</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">StreamHandler</span><span class="p">()</span>
|
|||
|
|
<span class="n">ch</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">ERROR</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># create formatter and add it to the handlers</span>
|
|||
|
|
<span class="n">formatter</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span><span class="s1">'</span><span class="si">%(asctime)s</span><span class="s1"> - </span><span class="si">%(name)s</span><span class="s1"> - </span><span class="si">%(levelname)s</span><span class="s1"> - </span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">)</span>
|
|||
|
|
<span class="n">fh</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">formatter</span><span class="p">)</span>
|
|||
|
|
<span class="n">ch</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">formatter</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># add the handlers to the logger</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">fh</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">ch</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'creating an instance of auxiliary_module.Auxiliary'</span><span class="p">)</span>
|
|||
|
|
<span class="n">a</span> <span class="o">=</span> <span class="n">auxiliary_module</span><span class="o">.</span><span class="n">Auxiliary</span><span class="p">()</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'created an instance of auxiliary_module.Auxiliary'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'calling auxiliary_module.Auxiliary.do_something'</span><span class="p">)</span>
|
|||
|
|
<span class="n">a</span><span class="o">.</span><span class="n">do_something</span><span class="p">()</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'finished auxiliary_module.Auxiliary.do_something'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'calling auxiliary_module.some_function()'</span><span class="p">)</span>
|
|||
|
|
<span class="n">auxiliary_module</span><span class="o">.</span><span class="n">some_function</span><span class="p">()</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'done with auxiliary_module.some_function()'</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Here is the auxiliary module:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># create logger</span>
|
|||
|
|
<span class="n">module_logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'spam_application.auxiliary'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">Auxiliary</span><span class="p">:</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'spam_application.auxiliary.Auxiliary'</span><span class="p">)</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'creating an instance of Auxiliary'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">do_something</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'doing something'</span><span class="p">)</span>
|
|||
|
|
<span class="n">a</span> <span class="o">=</span> <span class="mi">1</span> <span class="o">+</span> <span class="mi">1</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'done doing something'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">some_function</span><span class="p">():</span>
|
|||
|
|
<span class="n">module_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'received a call to "some_function"'</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>The output looks like this:</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>2005-03-23 23:47:11,663 - spam_application - INFO -
|
|||
|
|
creating an instance of auxiliary_module.Auxiliary
|
|||
|
|
2005-03-23 23:47:11,665 - spam_application.auxiliary.Auxiliary - INFO -
|
|||
|
|
creating an instance of Auxiliary
|
|||
|
|
2005-03-23 23:47:11,665 - spam_application - INFO -
|
|||
|
|
created an instance of auxiliary_module.Auxiliary
|
|||
|
|
2005-03-23 23:47:11,668 - spam_application - INFO -
|
|||
|
|
calling auxiliary_module.Auxiliary.do_something
|
|||
|
|
2005-03-23 23:47:11,668 - spam_application.auxiliary.Auxiliary - INFO -
|
|||
|
|
doing something
|
|||
|
|
2005-03-23 23:47:11,669 - spam_application.auxiliary.Auxiliary - INFO -
|
|||
|
|
done doing something
|
|||
|
|
2005-03-23 23:47:11,670 - spam_application - INFO -
|
|||
|
|
finished auxiliary_module.Auxiliary.do_something
|
|||
|
|
2005-03-23 23:47:11,671 - spam_application - INFO -
|
|||
|
|
calling auxiliary_module.some_function()
|
|||
|
|
2005-03-23 23:47:11,672 - spam_application.auxiliary - INFO -
|
|||
|
|
received a call to 'some_function'
|
|||
|
|
2005-03-23 23:47:11,673 - spam_application - INFO -
|
|||
|
|
done with auxiliary_module.some_function()
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="logging-from-multiple-threads">
|
|||
|
|
<h2>Logging from multiple threads<a class="headerlink" href="#logging-from-multiple-threads" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Logging from multiple threads requires no special effort. The following example
|
|||
|
|
shows logging from the main (initial) thread and another thread:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">threading</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">time</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">worker</span><span class="p">(</span><span class="n">arg</span><span class="p">):</span>
|
|||
|
|
<span class="k">while</span> <span class="ow">not</span> <span class="n">arg</span><span class="p">[</span><span class="s1">'stop'</span><span class="p">]:</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'Hi from myfunc'</span><span class="p">)</span>
|
|||
|
|
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">0.5</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">basicConfig</span><span class="p">(</span><span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">,</span> <span class="nb">format</span><span class="o">=</span><span class="s1">'</span><span class="si">%(relativeCreated)6d</span><span class="s1"> </span><span class="si">%(threadName)s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">)</span>
|
|||
|
|
<span class="n">info</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'stop'</span><span class="p">:</span> <span class="kc">False</span><span class="p">}</span>
|
|||
|
|
<span class="n">thread</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">worker</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">info</span><span class="p">,))</span>
|
|||
|
|
<span class="n">thread</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
|
|||
|
|
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
|
|||
|
|
<span class="k">try</span><span class="p">:</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'Hello from main'</span><span class="p">)</span>
|
|||
|
|
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">0.75</span><span class="p">)</span>
|
|||
|
|
<span class="k">except</span> <span class="ne">KeyboardInterrupt</span><span class="p">:</span>
|
|||
|
|
<span class="n">info</span><span class="p">[</span><span class="s1">'stop'</span><span class="p">]</span> <span class="o">=</span> <span class="kc">True</span>
|
|||
|
|
<span class="k">break</span>
|
|||
|
|
<span class="n">thread</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">main</span><span class="p">()</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>When run, the script should print something like the following:</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span> 0 Thread-1 Hi from myfunc
|
|||
|
|
3 MainThread Hello from main
|
|||
|
|
505 Thread-1 Hi from myfunc
|
|||
|
|
755 MainThread Hello from main
|
|||
|
|
1007 Thread-1 Hi from myfunc
|
|||
|
|
1507 MainThread Hello from main
|
|||
|
|
1508 Thread-1 Hi from myfunc
|
|||
|
|
2010 Thread-1 Hi from myfunc
|
|||
|
|
2258 MainThread Hello from main
|
|||
|
|
2512 Thread-1 Hi from myfunc
|
|||
|
|
3009 MainThread Hello from main
|
|||
|
|
3013 Thread-1 Hi from myfunc
|
|||
|
|
3515 Thread-1 Hi from myfunc
|
|||
|
|
3761 MainThread Hello from main
|
|||
|
|
4017 Thread-1 Hi from myfunc
|
|||
|
|
4513 MainThread Hello from main
|
|||
|
|
4518 Thread-1 Hi from myfunc
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>This shows the logging output interspersed as one might expect. This approach
|
|||
|
|
works for more threads than shown here, of course.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="multiple-handlers-and-formatters">
|
|||
|
|
<h2>Multiple handlers and formatters<a class="headerlink" href="#multiple-handlers-and-formatters" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Loggers are plain Python objects. The <a class="reference internal" href="../library/logging.html#logging.Logger.addHandler" title="logging.Logger.addHandler"><code class="xref py py-meth docutils literal notranslate"><span class="pre">addHandler()</span></code></a> method has no
|
|||
|
|
minimum or maximum quota for the number of handlers you may add. Sometimes it
|
|||
|
|
will be beneficial for an application to log all messages of all severities to a
|
|||
|
|
text file while simultaneously logging errors or above to the console. To set
|
|||
|
|
this up, simply configure the appropriate handlers. The logging calls in the
|
|||
|
|
application code will remain unchanged. Here is a slight modification to the
|
|||
|
|
previous simple module-based configuration example:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'simple_example'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># create file handler which logs even debug messages</span>
|
|||
|
|
<span class="n">fh</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">FileHandler</span><span class="p">(</span><span class="s1">'spam.log'</span><span class="p">)</span>
|
|||
|
|
<span class="n">fh</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># create console handler with a higher log level</span>
|
|||
|
|
<span class="n">ch</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">StreamHandler</span><span class="p">()</span>
|
|||
|
|
<span class="n">ch</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">ERROR</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># create formatter and add it to the handlers</span>
|
|||
|
|
<span class="n">formatter</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span><span class="s1">'</span><span class="si">%(asctime)s</span><span class="s1"> - </span><span class="si">%(name)s</span><span class="s1"> - </span><span class="si">%(levelname)s</span><span class="s1"> - </span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">)</span>
|
|||
|
|
<span class="n">ch</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">formatter</span><span class="p">)</span>
|
|||
|
|
<span class="n">fh</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">formatter</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># add the handlers to logger</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">ch</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">fh</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># 'application' code</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'debug message'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'info message'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="s1">'warn message'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s1">'error message'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">critical</span><span class="p">(</span><span class="s1">'critical message'</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Notice that the ‘application’ code does not care about multiple handlers. All
|
|||
|
|
that changed was the addition and configuration of a new handler named <em>fh</em>.</p>
|
|||
|
|
<p>The ability to create new handlers with higher- or lower-severity filters can be
|
|||
|
|
very helpful when writing and testing an application. Instead of using many
|
|||
|
|
<code class="docutils literal notranslate"><span class="pre">print</span></code> statements for debugging, use <code class="docutils literal notranslate"><span class="pre">logger.debug</span></code>: Unlike the print
|
|||
|
|
statements, which you will have to delete or comment out later, the logger.debug
|
|||
|
|
statements can remain intact in the source code and remain dormant until you
|
|||
|
|
need them again. At that time, the only change that needs to happen is to
|
|||
|
|
modify the severity level of the logger and/or handler to debug.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="logging-to-multiple-destinations">
|
|||
|
|
<span id="multiple-destinations"></span><h2>Logging to multiple destinations<a class="headerlink" href="#logging-to-multiple-destinations" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Let’s say you want to log to console and file with different message formats and
|
|||
|
|
in differing circumstances. Say you want to log messages with levels of DEBUG
|
|||
|
|
and higher to file, and those messages at level INFO and higher to the console.
|
|||
|
|
Let’s also assume that the file should contain timestamps, but the console
|
|||
|
|
messages should not. Here’s how you can achieve this:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># set up logging to file - see previous section for more details</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">basicConfig</span><span class="p">(</span><span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">,</span>
|
|||
|
|
<span class="nb">format</span><span class="o">=</span><span class="s1">'</span><span class="si">%(asctime)s</span><span class="s1"> </span><span class="si">%(name)-12s</span><span class="s1"> </span><span class="si">%(levelname)-8s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">,</span>
|
|||
|
|
<span class="n">datefmt</span><span class="o">=</span><span class="s1">'%m-</span><span class="si">%d</span><span class="s1"> %H:%M'</span><span class="p">,</span>
|
|||
|
|
<span class="n">filename</span><span class="o">=</span><span class="s1">'/temp/myapp.log'</span><span class="p">,</span>
|
|||
|
|
<span class="n">filemode</span><span class="o">=</span><span class="s1">'w'</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># define a Handler which writes INFO messages or higher to the sys.stderr</span>
|
|||
|
|
<span class="n">console</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">StreamHandler</span><span class="p">()</span>
|
|||
|
|
<span class="n">console</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># set a format which is simpler for console use</span>
|
|||
|
|
<span class="n">formatter</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span><span class="s1">'</span><span class="si">%(name)-12s</span><span class="s1">: </span><span class="si">%(levelname)-8s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># tell the handler to use this format</span>
|
|||
|
|
<span class="n">console</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">formatter</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># add the handler to the root logger</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">''</span><span class="p">)</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">console</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># Now, we can log to the root logger, or any other logger. First the root...</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'Jackdaws love my big sphinx of quartz.'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># Now, define a couple of other loggers which might represent areas in your</span>
|
|||
|
|
<span class="c1"># application:</span>
|
|||
|
|
|
|||
|
|
<span class="n">logger1</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'myapp.area1'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger2</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'myapp.area2'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="n">logger1</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'Quick zephyrs blow, vexing daft Jim.'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger1</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'How quickly daft jumping zebras vex.'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger2</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="s1">'Jail zesty vixen who grabbed pay from quack.'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger2</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s1">'The five boxing wizards jump quickly.'</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>When you run this, on the console you will see</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>root : INFO Jackdaws love my big sphinx of quartz.
|
|||
|
|
myapp.area1 : INFO How quickly daft jumping zebras vex.
|
|||
|
|
myapp.area2 : WARNING Jail zesty vixen who grabbed pay from quack.
|
|||
|
|
myapp.area2 : ERROR The five boxing wizards jump quickly.
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>and in the file you will see something like</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>10-22 22:19 root INFO Jackdaws love my big sphinx of quartz.
|
|||
|
|
10-22 22:19 myapp.area1 DEBUG Quick zephyrs blow, vexing daft Jim.
|
|||
|
|
10-22 22:19 myapp.area1 INFO How quickly daft jumping zebras vex.
|
|||
|
|
10-22 22:19 myapp.area2 WARNING Jail zesty vixen who grabbed pay from quack.
|
|||
|
|
10-22 22:19 myapp.area2 ERROR The five boxing wizards jump quickly.
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>As you can see, the DEBUG message only shows up in the file. The other messages
|
|||
|
|
are sent to both destinations.</p>
|
|||
|
|
<p>This example uses console and file handlers, but you can use any number and
|
|||
|
|
combination of handlers you choose.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="configuration-server-example">
|
|||
|
|
<h2>Configuration server example<a class="headerlink" href="#configuration-server-example" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Here is an example of a module using the logging configuration server:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging.config</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">time</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">os</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># read initial config file</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">fileConfig</span><span class="p">(</span><span class="s1">'logging.conf'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># create and start listener on port 9999</span>
|
|||
|
|
<span class="n">t</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">listen</span><span class="p">(</span><span class="mi">9999</span><span class="p">)</span>
|
|||
|
|
<span class="n">t</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
|
|||
|
|
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'simpleExample'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">try</span><span class="p">:</span>
|
|||
|
|
<span class="c1"># loop through logging calls to see the difference</span>
|
|||
|
|
<span class="c1"># new configurations make, until Ctrl+C is pressed</span>
|
|||
|
|
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'debug message'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'info message'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="s1">'warn message'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s1">'error message'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">critical</span><span class="p">(</span><span class="s1">'critical message'</span><span class="p">)</span>
|
|||
|
|
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
|
|||
|
|
<span class="k">except</span> <span class="ne">KeyboardInterrupt</span><span class="p">:</span>
|
|||
|
|
<span class="c1"># cleanup</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">stopListening</span><span class="p">()</span>
|
|||
|
|
<span class="n">t</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>And here is a script that takes a filename and sends that file to the server,
|
|||
|
|
properly preceded with the binary-encoded length, as the new logging
|
|||
|
|
configuration:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="ch">#!/usr/bin/env python</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">socket</span><span class="o">,</span> <span class="nn">sys</span><span class="o">,</span> <span class="nn">struct</span>
|
|||
|
|
|
|||
|
|
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="s1">'rb'</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
|
|||
|
|
<span class="n">data_to_send</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
|
|||
|
|
|
|||
|
|
<span class="n">HOST</span> <span class="o">=</span> <span class="s1">'localhost'</span>
|
|||
|
|
<span class="n">PORT</span> <span class="o">=</span> <span class="mi">9999</span>
|
|||
|
|
<span class="n">s</span> <span class="o">=</span> <span class="n">socket</span><span class="o">.</span><span class="n">socket</span><span class="p">(</span><span class="n">socket</span><span class="o">.</span><span class="n">AF_INET</span><span class="p">,</span> <span class="n">socket</span><span class="o">.</span><span class="n">SOCK_STREAM</span><span class="p">)</span>
|
|||
|
|
<span class="nb">print</span><span class="p">(</span><span class="s1">'connecting...'</span><span class="p">)</span>
|
|||
|
|
<span class="n">s</span><span class="o">.</span><span class="n">connect</span><span class="p">((</span><span class="n">HOST</span><span class="p">,</span> <span class="n">PORT</span><span class="p">))</span>
|
|||
|
|
<span class="nb">print</span><span class="p">(</span><span class="s1">'sending config...'</span><span class="p">)</span>
|
|||
|
|
<span class="n">s</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="n">struct</span><span class="o">.</span><span class="n">pack</span><span class="p">(</span><span class="s1">'>L'</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">data_to_send</span><span class="p">)))</span>
|
|||
|
|
<span class="n">s</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="n">data_to_send</span><span class="p">)</span>
|
|||
|
|
<span class="n">s</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
|
|||
|
|
<span class="nb">print</span><span class="p">(</span><span class="s1">'complete'</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="dealing-with-handlers-that-block">
|
|||
|
|
<h2>Dealing with handlers that block<a class="headerlink" href="#dealing-with-handlers-that-block" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Sometimes you have to get your logging handlers to do their work without
|
|||
|
|
blocking the thread you’re logging from. This is common in Web applications,
|
|||
|
|
though of course it also occurs in other scenarios.</p>
|
|||
|
|
<p>A common culprit which demonstrates sluggish behaviour is the
|
|||
|
|
<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.SMTPHandler" title="logging.handlers.SMTPHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">SMTPHandler</span></code></a>: sending emails can take a long time, for a
|
|||
|
|
number of reasons outside the developer’s control (for example, a poorly
|
|||
|
|
performing mail or network infrastructure). But almost any network-based
|
|||
|
|
handler can block: Even a <a class="reference internal" href="../library/logging.handlers.html#logging.handlers.SocketHandler" title="logging.handlers.SocketHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">SocketHandler</span></code></a> operation may do a
|
|||
|
|
DNS query under the hood which is too slow (and this query can be deep in the
|
|||
|
|
socket library code, below the Python layer, and outside your control).</p>
|
|||
|
|
<p>One solution is to use a two-part approach. For the first part, attach only a
|
|||
|
|
<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.QueueHandler" title="logging.handlers.QueueHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">QueueHandler</span></code></a> to those loggers which are accessed from
|
|||
|
|
performance-critical threads. They simply write to their queue, which can be
|
|||
|
|
sized to a large enough capacity or initialized with no upper bound to their
|
|||
|
|
size. The write to the queue will typically be accepted quickly, though you
|
|||
|
|
will probably need to catch the <a class="reference internal" href="../library/queue.html#queue.Full" title="queue.Full"><code class="xref py py-exc docutils literal notranslate"><span class="pre">queue.Full</span></code></a> exception as a precaution
|
|||
|
|
in your code. If you are a library developer who has performance-critical
|
|||
|
|
threads in their code, be sure to document this (together with a suggestion to
|
|||
|
|
attach only <code class="docutils literal notranslate"><span class="pre">QueueHandlers</span></code> to your loggers) for the benefit of other
|
|||
|
|
developers who will use your code.</p>
|
|||
|
|
<p>The second part of the solution is <a class="reference internal" href="../library/logging.handlers.html#logging.handlers.QueueListener" title="logging.handlers.QueueListener"><code class="xref py py-class docutils literal notranslate"><span class="pre">QueueListener</span></code></a>, which has been
|
|||
|
|
designed as the counterpart to <a class="reference internal" href="../library/logging.handlers.html#logging.handlers.QueueHandler" title="logging.handlers.QueueHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">QueueHandler</span></code></a>. A
|
|||
|
|
<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.QueueListener" title="logging.handlers.QueueListener"><code class="xref py py-class docutils literal notranslate"><span class="pre">QueueListener</span></code></a> is very simple: it’s passed a queue and some handlers,
|
|||
|
|
and it fires up an internal thread which listens to its queue for LogRecords
|
|||
|
|
sent from <code class="docutils literal notranslate"><span class="pre">QueueHandlers</span></code> (or any other source of <code class="docutils literal notranslate"><span class="pre">LogRecords</span></code>, for that
|
|||
|
|
matter). The <code class="docutils literal notranslate"><span class="pre">LogRecords</span></code> are removed from the queue and passed to the
|
|||
|
|
handlers for processing.</p>
|
|||
|
|
<p>The advantage of having a separate <a class="reference internal" href="../library/logging.handlers.html#logging.handlers.QueueListener" title="logging.handlers.QueueListener"><code class="xref py py-class docutils literal notranslate"><span class="pre">QueueListener</span></code></a> class is that you
|
|||
|
|
can use the same instance to service multiple <code class="docutils literal notranslate"><span class="pre">QueueHandlers</span></code>. This is more
|
|||
|
|
resource-friendly than, say, having threaded versions of the existing handler
|
|||
|
|
classes, which would eat up one thread per handler for no particular benefit.</p>
|
|||
|
|
<p>An example of using these two classes follows (imports omitted):</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">que</span> <span class="o">=</span> <span class="n">queue</span><span class="o">.</span><span class="n">Queue</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="c1"># no limit on size</span>
|
|||
|
|
<span class="n">queue_handler</span> <span class="o">=</span> <span class="n">QueueHandler</span><span class="p">(</span><span class="n">que</span><span class="p">)</span>
|
|||
|
|
<span class="n">handler</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">StreamHandler</span><span class="p">()</span>
|
|||
|
|
<span class="n">listener</span> <span class="o">=</span> <span class="n">QueueListener</span><span class="p">(</span><span class="n">que</span><span class="p">,</span> <span class="n">handler</span><span class="p">)</span>
|
|||
|
|
<span class="n">root</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">()</span>
|
|||
|
|
<span class="n">root</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">queue_handler</span><span class="p">)</span>
|
|||
|
|
<span class="n">formatter</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span><span class="s1">'</span><span class="si">%(threadName)s</span><span class="s1">: </span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">)</span>
|
|||
|
|
<span class="n">handler</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">formatter</span><span class="p">)</span>
|
|||
|
|
<span class="n">listener</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
|
|||
|
|
<span class="c1"># The log output will display the thread which generated</span>
|
|||
|
|
<span class="c1"># the event (the main thread) rather than the internal</span>
|
|||
|
|
<span class="c1"># thread which monitors the internal queue. This is what</span>
|
|||
|
|
<span class="c1"># you want to happen.</span>
|
|||
|
|
<span class="n">root</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="s1">'Look out!'</span><span class="p">)</span>
|
|||
|
|
<span class="n">listener</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>which, when run, will produce:</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>MainThread: Look out!
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<div class="versionchanged">
|
|||
|
|
<p><span class="versionmodified changed">Changed in version 3.5: </span>Prior to Python 3.5, the <a class="reference internal" href="../library/logging.handlers.html#logging.handlers.QueueListener" title="logging.handlers.QueueListener"><code class="xref py py-class docutils literal notranslate"><span class="pre">QueueListener</span></code></a> always passed every message
|
|||
|
|
received from the queue to every handler it was initialized with. (This was
|
|||
|
|
because it was assumed that level filtering was all done on the other side,
|
|||
|
|
where the queue is filled.) From 3.5 onwards, this behaviour can be changed
|
|||
|
|
by passing a keyword argument <code class="docutils literal notranslate"><span class="pre">respect_handler_level=True</span></code> to the
|
|||
|
|
listener’s constructor. When this is done, the listener compares the level
|
|||
|
|
of each message with the handler’s level, and only passes a message to a
|
|||
|
|
handler if it’s appropriate to do so.</p>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="sending-and-receiving-logging-events-across-a-network">
|
|||
|
|
<span id="network-logging"></span><h2>Sending and receiving logging events across a network<a class="headerlink" href="#sending-and-receiving-logging-events-across-a-network" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Let’s say you want to send logging events across a network, and handle them at
|
|||
|
|
the receiving end. A simple way of doing this is attaching a
|
|||
|
|
<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.SocketHandler" title="logging.handlers.SocketHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">SocketHandler</span></code></a> instance to the root logger at the sending end:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span><span class="o">,</span> <span class="nn">logging.handlers</span>
|
|||
|
|
|
|||
|
|
<span class="n">rootLogger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">''</span><span class="p">)</span>
|
|||
|
|
<span class="n">rootLogger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
<span class="n">socketHandler</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">handlers</span><span class="o">.</span><span class="n">SocketHandler</span><span class="p">(</span><span class="s1">'localhost'</span><span class="p">,</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">handlers</span><span class="o">.</span><span class="n">DEFAULT_TCP_LOGGING_PORT</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># don't bother with a formatter, since a socket handler sends the event as</span>
|
|||
|
|
<span class="c1"># an unformatted pickle</span>
|
|||
|
|
<span class="n">rootLogger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">socketHandler</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># Now, we can log to the root logger, or any other logger. First the root...</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'Jackdaws love my big sphinx of quartz.'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># Now, define a couple of other loggers which might represent areas in your</span>
|
|||
|
|
<span class="c1"># application:</span>
|
|||
|
|
|
|||
|
|
<span class="n">logger1</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'myapp.area1'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger2</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'myapp.area2'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="n">logger1</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'Quick zephyrs blow, vexing daft Jim.'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger1</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'How quickly daft jumping zebras vex.'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger2</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="s1">'Jail zesty vixen who grabbed pay from quack.'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger2</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s1">'The five boxing wizards jump quickly.'</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>At the receiving end, you can set up a receiver using the <a class="reference internal" href="../library/socketserver.html#module-socketserver" title="socketserver: A framework for network servers."><code class="xref py py-mod docutils literal notranslate"><span class="pre">socketserver</span></code></a>
|
|||
|
|
module. Here is a basic working example:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pickle</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging.handlers</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">socketserver</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">struct</span>
|
|||
|
|
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">LogRecordStreamHandler</span><span class="p">(</span><span class="n">socketserver</span><span class="o">.</span><span class="n">StreamRequestHandler</span><span class="p">):</span>
|
|||
|
|
<span class="sd">"""Handler for a streaming logging request.</span>
|
|||
|
|
|
|||
|
|
<span class="sd"> This basically logs the record using whatever logging policy is</span>
|
|||
|
|
<span class="sd"> configured locally.</span>
|
|||
|
|
<span class="sd"> """</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">handle</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="sd">"""</span>
|
|||
|
|
<span class="sd"> Handle multiple requests - each expected to be a 4-byte length,</span>
|
|||
|
|
<span class="sd"> followed by the LogRecord in pickle format. Logs the record</span>
|
|||
|
|
<span class="sd"> according to whatever policy is configured locally.</span>
|
|||
|
|
<span class="sd"> """</span>
|
|||
|
|
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
|
|||
|
|
<span class="n">chunk</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">connection</span><span class="o">.</span><span class="n">recv</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span>
|
|||
|
|
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">chunk</span><span class="p">)</span> <span class="o"><</span> <span class="mi">4</span><span class="p">:</span>
|
|||
|
|
<span class="k">break</span>
|
|||
|
|
<span class="n">slen</span> <span class="o">=</span> <span class="n">struct</span><span class="o">.</span><span class="n">unpack</span><span class="p">(</span><span class="s1">'>L'</span><span class="p">,</span> <span class="n">chunk</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
|
|||
|
|
<span class="n">chunk</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">connection</span><span class="o">.</span><span class="n">recv</span><span class="p">(</span><span class="n">slen</span><span class="p">)</span>
|
|||
|
|
<span class="k">while</span> <span class="nb">len</span><span class="p">(</span><span class="n">chunk</span><span class="p">)</span> <span class="o"><</span> <span class="n">slen</span><span class="p">:</span>
|
|||
|
|
<span class="n">chunk</span> <span class="o">=</span> <span class="n">chunk</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">connection</span><span class="o">.</span><span class="n">recv</span><span class="p">(</span><span class="n">slen</span> <span class="o">-</span> <span class="nb">len</span><span class="p">(</span><span class="n">chunk</span><span class="p">))</span>
|
|||
|
|
<span class="n">obj</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">unPickle</span><span class="p">(</span><span class="n">chunk</span><span class="p">)</span>
|
|||
|
|
<span class="n">record</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">makeLogRecord</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">handleLogRecord</span><span class="p">(</span><span class="n">record</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">unPickle</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">data</span><span class="p">):</span>
|
|||
|
|
<span class="k">return</span> <span class="n">pickle</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">handleLogRecord</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">record</span><span class="p">):</span>
|
|||
|
|
<span class="c1"># if a name is specified, we use the named logger rather than the one</span>
|
|||
|
|
<span class="c1"># implied by the record.</span>
|
|||
|
|
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">server</span><span class="o">.</span><span class="n">logname</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
|
|||
|
|
<span class="n">name</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">server</span><span class="o">.</span><span class="n">logname</span>
|
|||
|
|
<span class="k">else</span><span class="p">:</span>
|
|||
|
|
<span class="n">name</span> <span class="o">=</span> <span class="n">record</span><span class="o">.</span><span class="n">name</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># N.B. EVERY record gets logged. This is because Logger.handle</span>
|
|||
|
|
<span class="c1"># is normally called AFTER logger-level filtering. If you want</span>
|
|||
|
|
<span class="c1"># to do filtering, do it at the client end to save wasting</span>
|
|||
|
|
<span class="c1"># cycles and network bandwidth!</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">handle</span><span class="p">(</span><span class="n">record</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">LogRecordSocketReceiver</span><span class="p">(</span><span class="n">socketserver</span><span class="o">.</span><span class="n">ThreadingTCPServer</span><span class="p">):</span>
|
|||
|
|
<span class="sd">"""</span>
|
|||
|
|
<span class="sd"> Simple TCP socket-based logging receiver suitable for testing.</span>
|
|||
|
|
<span class="sd"> """</span>
|
|||
|
|
|
|||
|
|
<span class="n">allow_reuse_address</span> <span class="o">=</span> <span class="kc">True</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">host</span><span class="o">=</span><span class="s1">'localhost'</span><span class="p">,</span>
|
|||
|
|
<span class="n">port</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">handlers</span><span class="o">.</span><span class="n">DEFAULT_TCP_LOGGING_PORT</span><span class="p">,</span>
|
|||
|
|
<span class="n">handler</span><span class="o">=</span><span class="n">LogRecordStreamHandler</span><span class="p">):</span>
|
|||
|
|
<span class="n">socketserver</span><span class="o">.</span><span class="n">ThreadingTCPServer</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="p">(</span><span class="n">host</span><span class="p">,</span> <span class="n">port</span><span class="p">),</span> <span class="n">handler</span><span class="p">)</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">abort</span> <span class="o">=</span> <span class="mi">0</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">timeout</span> <span class="o">=</span> <span class="mi">1</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">logname</span> <span class="o">=</span> <span class="kc">None</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">serve_until_stopped</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">select</span>
|
|||
|
|
<span class="n">abort</span> <span class="o">=</span> <span class="mi">0</span>
|
|||
|
|
<span class="k">while</span> <span class="ow">not</span> <span class="n">abort</span><span class="p">:</span>
|
|||
|
|
<span class="n">rd</span><span class="p">,</span> <span class="n">wr</span><span class="p">,</span> <span class="n">ex</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">select</span><span class="p">([</span><span class="bp">self</span><span class="o">.</span><span class="n">socket</span><span class="o">.</span><span class="n">fileno</span><span class="p">()],</span>
|
|||
|
|
<span class="p">[],</span> <span class="p">[],</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">timeout</span><span class="p">)</span>
|
|||
|
|
<span class="k">if</span> <span class="n">rd</span><span class="p">:</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">handle_request</span><span class="p">()</span>
|
|||
|
|
<span class="n">abort</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">abort</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">basicConfig</span><span class="p">(</span>
|
|||
|
|
<span class="nb">format</span><span class="o">=</span><span class="s1">'</span><span class="si">%(relativeCreated)5d</span><span class="s1"> </span><span class="si">%(name)-15s</span><span class="s1"> </span><span class="si">%(levelname)-8s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">)</span>
|
|||
|
|
<span class="n">tcpserver</span> <span class="o">=</span> <span class="n">LogRecordSocketReceiver</span><span class="p">()</span>
|
|||
|
|
<span class="nb">print</span><span class="p">(</span><span class="s1">'About to start TCP server...'</span><span class="p">)</span>
|
|||
|
|
<span class="n">tcpserver</span><span class="o">.</span><span class="n">serve_until_stopped</span><span class="p">()</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">main</span><span class="p">()</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>First run the server, and then the client. On the client side, nothing is
|
|||
|
|
printed on the console; on the server side, you should see something like:</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>About to start TCP server...
|
|||
|
|
59 root INFO Jackdaws love my big sphinx of quartz.
|
|||
|
|
59 myapp.area1 DEBUG Quick zephyrs blow, vexing daft Jim.
|
|||
|
|
69 myapp.area1 INFO How quickly daft jumping zebras vex.
|
|||
|
|
69 myapp.area2 WARNING Jail zesty vixen who grabbed pay from quack.
|
|||
|
|
69 myapp.area2 ERROR The five boxing wizards jump quickly.
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Note that there are some security issues with pickle in some scenarios. If
|
|||
|
|
these affect you, you can use an alternative serialization scheme by overriding
|
|||
|
|
the <code class="xref py py-meth docutils literal notranslate"><span class="pre">makePickle()</span></code> method and implementing your
|
|||
|
|
alternative there, as well as adapting the above script to use your alternative
|
|||
|
|
serialization.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="adding-contextual-information-to-your-logging-output">
|
|||
|
|
<span id="context-info"></span><h2>Adding contextual information to your logging output<a class="headerlink" href="#adding-contextual-information-to-your-logging-output" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Sometimes you want logging output to contain contextual information in
|
|||
|
|
addition to the parameters passed to the logging call. For example, in a
|
|||
|
|
networked application, it may be desirable to log client-specific information
|
|||
|
|
in the log (e.g. remote client’s username, or IP address). Although you could
|
|||
|
|
use the <em>extra</em> parameter to achieve this, it’s not always convenient to pass
|
|||
|
|
the information in this way. While it might be tempting to create
|
|||
|
|
<code class="xref py py-class docutils literal notranslate"><span class="pre">Logger</span></code> instances on a per-connection basis, this is not a good idea
|
|||
|
|
because these instances are not garbage collected. While this is not a problem
|
|||
|
|
in practice, when the number of <code class="xref py py-class docutils literal notranslate"><span class="pre">Logger</span></code> instances is dependent on the
|
|||
|
|
level of granularity you want to use in logging an application, it could
|
|||
|
|
be hard to manage if the number of <code class="xref py py-class docutils literal notranslate"><span class="pre">Logger</span></code> instances becomes
|
|||
|
|
effectively unbounded.</p>
|
|||
|
|
<div class="section" id="using-loggeradapters-to-impart-contextual-information">
|
|||
|
|
<h3>Using LoggerAdapters to impart contextual information<a class="headerlink" href="#using-loggeradapters-to-impart-contextual-information" title="Permalink to this headline">¶</a></h3>
|
|||
|
|
<p>An easy way in which you can pass contextual information to be output along
|
|||
|
|
with logging event information is to use the <code class="xref py py-class docutils literal notranslate"><span class="pre">LoggerAdapter</span></code> class.
|
|||
|
|
This class is designed to look like a <code class="xref py py-class docutils literal notranslate"><span class="pre">Logger</span></code>, so that you can call
|
|||
|
|
<code class="xref py py-meth docutils literal notranslate"><span class="pre">debug()</span></code>, <code class="xref py py-meth docutils literal notranslate"><span class="pre">info()</span></code>, <code class="xref py py-meth docutils literal notranslate"><span class="pre">warning()</span></code>, <code class="xref py py-meth docutils literal notranslate"><span class="pre">error()</span></code>,
|
|||
|
|
<code class="xref py py-meth docutils literal notranslate"><span class="pre">exception()</span></code>, <code class="xref py py-meth docutils literal notranslate"><span class="pre">critical()</span></code> and <code class="xref py py-meth docutils literal notranslate"><span class="pre">log()</span></code>. These methods have the
|
|||
|
|
same signatures as their counterparts in <code class="xref py py-class docutils literal notranslate"><span class="pre">Logger</span></code>, so you can use the
|
|||
|
|
two types of instances interchangeably.</p>
|
|||
|
|
<p>When you create an instance of <code class="xref py py-class docutils literal notranslate"><span class="pre">LoggerAdapter</span></code>, you pass it a
|
|||
|
|
<code class="xref py py-class docutils literal notranslate"><span class="pre">Logger</span></code> instance and a dict-like object which contains your contextual
|
|||
|
|
information. When you call one of the logging methods on an instance of
|
|||
|
|
<code class="xref py py-class docutils literal notranslate"><span class="pre">LoggerAdapter</span></code>, it delegates the call to the underlying instance of
|
|||
|
|
<code class="xref py py-class docutils literal notranslate"><span class="pre">Logger</span></code> passed to its constructor, and arranges to pass the contextual
|
|||
|
|
information in the delegated call. Here’s a snippet from the code of
|
|||
|
|
<code class="xref py py-class docutils literal notranslate"><span class="pre">LoggerAdapter</span></code>:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">debug</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">msg</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|||
|
|
<span class="sd">"""</span>
|
|||
|
|
<span class="sd"> Delegate a debug call to the underlying logger, after adding</span>
|
|||
|
|
<span class="sd"> contextual information from this adapter instance.</span>
|
|||
|
|
<span class="sd"> """</span>
|
|||
|
|
<span class="n">msg</span><span class="p">,</span> <span class="n">kwargs</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">process</span><span class="p">(</span><span class="n">msg</span><span class="p">,</span> <span class="n">kwargs</span><span class="p">)</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="n">msg</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>The <code class="xref py py-meth docutils literal notranslate"><span class="pre">process()</span></code> method of <code class="xref py py-class docutils literal notranslate"><span class="pre">LoggerAdapter</span></code> is where the
|
|||
|
|
contextual information is added to the logging output. It’s passed the message
|
|||
|
|
and keyword arguments of the logging call, and it passes back (potentially)
|
|||
|
|
modified versions of these to use in the call to the underlying logger. The
|
|||
|
|
default implementation of this method leaves the message alone, but inserts
|
|||
|
|
an ‘extra’ key in the keyword argument whose value is the dict-like object
|
|||
|
|
passed to the constructor. Of course, if you had passed an ‘extra’ keyword
|
|||
|
|
argument in the call to the adapter, it will be silently overwritten.</p>
|
|||
|
|
<p>The advantage of using ‘extra’ is that the values in the dict-like object are
|
|||
|
|
merged into the <code class="xref py py-class docutils literal notranslate"><span class="pre">LogRecord</span></code> instance’s __dict__, allowing you to use
|
|||
|
|
customized strings with your <code class="xref py py-class docutils literal notranslate"><span class="pre">Formatter</span></code> instances which know about
|
|||
|
|
the keys of the dict-like object. If you need a different method, e.g. if you
|
|||
|
|
want to prepend or append the contextual information to the message string,
|
|||
|
|
you just need to subclass <code class="xref py py-class docutils literal notranslate"><span class="pre">LoggerAdapter</span></code> and override
|
|||
|
|
<code class="xref py py-meth docutils literal notranslate"><span class="pre">process()</span></code> to do what you need. Here is a simple example:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">CustomAdapter</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">LoggerAdapter</span><span class="p">):</span>
|
|||
|
|
<span class="sd">"""</span>
|
|||
|
|
<span class="sd"> This example adapter expects the passed in dict-like object to have a</span>
|
|||
|
|
<span class="sd"> 'connid' key, whose value in brackets is prepended to the log message.</span>
|
|||
|
|
<span class="sd"> """</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">process</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">msg</span><span class="p">,</span> <span class="n">kwargs</span><span class="p">):</span>
|
|||
|
|
<span class="k">return</span> <span class="s1">'[</span><span class="si">%s</span><span class="s1">] </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">extra</span><span class="p">[</span><span class="s1">'connid'</span><span class="p">],</span> <span class="n">msg</span><span class="p">),</span> <span class="n">kwargs</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>which you can use like this:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
|
|||
|
|
<span class="n">adapter</span> <span class="o">=</span> <span class="n">CustomAdapter</span><span class="p">(</span><span class="n">logger</span><span class="p">,</span> <span class="p">{</span><span class="s1">'connid'</span><span class="p">:</span> <span class="n">some_conn_id</span><span class="p">})</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Then any events that you log to the adapter will have the value of
|
|||
|
|
<code class="docutils literal notranslate"><span class="pre">some_conn_id</span></code> prepended to the log messages.</p>
|
|||
|
|
<div class="section" id="using-objects-other-than-dicts-to-pass-contextual-information">
|
|||
|
|
<h4>Using objects other than dicts to pass contextual information<a class="headerlink" href="#using-objects-other-than-dicts-to-pass-contextual-information" title="Permalink to this headline">¶</a></h4>
|
|||
|
|
<p>You don’t need to pass an actual dict to a <code class="xref py py-class docutils literal notranslate"><span class="pre">LoggerAdapter</span></code> - you could
|
|||
|
|
pass an instance of a class which implements <code class="docutils literal notranslate"><span class="pre">__getitem__</span></code> and <code class="docutils literal notranslate"><span class="pre">__iter__</span></code> so
|
|||
|
|
that it looks like a dict to logging. This would be useful if you want to
|
|||
|
|
generate values dynamically (whereas the values in a dict would be constant).</p>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="using-filters-to-impart-contextual-information">
|
|||
|
|
<span id="filters-contextual"></span><h3>Using Filters to impart contextual information<a class="headerlink" href="#using-filters-to-impart-contextual-information" title="Permalink to this headline">¶</a></h3>
|
|||
|
|
<p>You can also add contextual information to log output using a user-defined
|
|||
|
|
<code class="xref py py-class docutils literal notranslate"><span class="pre">Filter</span></code>. <code class="docutils literal notranslate"><span class="pre">Filter</span></code> instances are allowed to modify the <code class="docutils literal notranslate"><span class="pre">LogRecords</span></code>
|
|||
|
|
passed to them, including adding additional attributes which can then be output
|
|||
|
|
using a suitable format string, or if needed a custom <code class="xref py py-class docutils literal notranslate"><span class="pre">Formatter</span></code>.</p>
|
|||
|
|
<p>For example in a web application, the request being processed (or at least,
|
|||
|
|
the interesting parts of it) can be stored in a threadlocal
|
|||
|
|
(<a class="reference internal" href="../library/threading.html#threading.local" title="threading.local"><code class="xref py py-class docutils literal notranslate"><span class="pre">threading.local</span></code></a>) variable, and then accessed from a <code class="docutils literal notranslate"><span class="pre">Filter</span></code> to
|
|||
|
|
add, say, information from the request - say, the remote IP address and remote
|
|||
|
|
user’s username - to the <code class="docutils literal notranslate"><span class="pre">LogRecord</span></code>, using the attribute names ‘ip’ and
|
|||
|
|
‘user’ as in the <code class="docutils literal notranslate"><span class="pre">LoggerAdapter</span></code> example above. In that case, the same format
|
|||
|
|
string can be used to get similar output to that shown above. Here’s an example
|
|||
|
|
script:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">from</span> <span class="nn">random</span> <span class="k">import</span> <span class="n">choice</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">ContextFilter</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">Filter</span><span class="p">):</span>
|
|||
|
|
<span class="sd">"""</span>
|
|||
|
|
<span class="sd"> This is a filter which injects contextual information into the log.</span>
|
|||
|
|
|
|||
|
|
<span class="sd"> Rather than use actual contextual information, we just use random</span>
|
|||
|
|
<span class="sd"> data in this demo.</span>
|
|||
|
|
<span class="sd"> """</span>
|
|||
|
|
|
|||
|
|
<span class="n">USERS</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'jim'</span><span class="p">,</span> <span class="s1">'fred'</span><span class="p">,</span> <span class="s1">'sheila'</span><span class="p">]</span>
|
|||
|
|
<span class="n">IPS</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'123.231.231.123'</span><span class="p">,</span> <span class="s1">'127.0.0.1'</span><span class="p">,</span> <span class="s1">'192.168.0.1'</span><span class="p">]</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">filter</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">record</span><span class="p">):</span>
|
|||
|
|
|
|||
|
|
<span class="n">record</span><span class="o">.</span><span class="n">ip</span> <span class="o">=</span> <span class="n">choice</span><span class="p">(</span><span class="n">ContextFilter</span><span class="o">.</span><span class="n">IPS</span><span class="p">)</span>
|
|||
|
|
<span class="n">record</span><span class="o">.</span><span class="n">user</span> <span class="o">=</span> <span class="n">choice</span><span class="p">(</span><span class="n">ContextFilter</span><span class="o">.</span><span class="n">USERS</span><span class="p">)</span>
|
|||
|
|
<span class="k">return</span> <span class="kc">True</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">levels</span> <span class="o">=</span> <span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">WARNING</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">ERROR</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">CRITICAL</span><span class="p">)</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">basicConfig</span><span class="p">(</span><span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">,</span>
|
|||
|
|
<span class="nb">format</span><span class="o">=</span><span class="s1">'</span><span class="si">%(asctime)-15s</span><span class="s1"> </span><span class="si">%(name)-5s</span><span class="s1"> </span><span class="si">%(levelname)-8s</span><span class="s1"> IP: </span><span class="si">%(ip)-15s</span><span class="s1"> User: </span><span class="si">%(user)-8s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">)</span>
|
|||
|
|
<span class="n">a1</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'a.b.c'</span><span class="p">)</span>
|
|||
|
|
<span class="n">a2</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'d.e.f'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="n">f</span> <span class="o">=</span> <span class="n">ContextFilter</span><span class="p">()</span>
|
|||
|
|
<span class="n">a1</span><span class="o">.</span><span class="n">addFilter</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
|
|||
|
|
<span class="n">a2</span><span class="o">.</span><span class="n">addFilter</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
|
|||
|
|
<span class="n">a1</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'A debug message'</span><span class="p">)</span>
|
|||
|
|
<span class="n">a1</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'An info message with </span><span class="si">%s</span><span class="s1">'</span><span class="p">,</span> <span class="s1">'some parameters'</span><span class="p">)</span>
|
|||
|
|
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
|
|||
|
|
<span class="n">lvl</span> <span class="o">=</span> <span class="n">choice</span><span class="p">(</span><span class="n">levels</span><span class="p">)</span>
|
|||
|
|
<span class="n">lvlname</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLevelName</span><span class="p">(</span><span class="n">lvl</span><span class="p">)</span>
|
|||
|
|
<span class="n">a2</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">lvl</span><span class="p">,</span> <span class="s1">'A message at </span><span class="si">%s</span><span class="s1"> level with </span><span class="si">%d</span><span class="s1"> </span><span class="si">%s</span><span class="s1">'</span><span class="p">,</span> <span class="n">lvlname</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="s1">'parameters'</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>which, when run, produces something like:</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>2010-09-06 22:38:15,292 a.b.c DEBUG IP: 123.231.231.123 User: fred A debug message
|
|||
|
|
2010-09-06 22:38:15,300 a.b.c INFO IP: 192.168.0.1 User: sheila An info message with some parameters
|
|||
|
|
2010-09-06 22:38:15,300 d.e.f CRITICAL IP: 127.0.0.1 User: sheila A message at CRITICAL level with 2 parameters
|
|||
|
|
2010-09-06 22:38:15,300 d.e.f ERROR IP: 127.0.0.1 User: jim A message at ERROR level with 2 parameters
|
|||
|
|
2010-09-06 22:38:15,300 d.e.f DEBUG IP: 127.0.0.1 User: sheila A message at DEBUG level with 2 parameters
|
|||
|
|
2010-09-06 22:38:15,300 d.e.f ERROR IP: 123.231.231.123 User: fred A message at ERROR level with 2 parameters
|
|||
|
|
2010-09-06 22:38:15,300 d.e.f CRITICAL IP: 192.168.0.1 User: jim A message at CRITICAL level with 2 parameters
|
|||
|
|
2010-09-06 22:38:15,300 d.e.f CRITICAL IP: 127.0.0.1 User: sheila A message at CRITICAL level with 2 parameters
|
|||
|
|
2010-09-06 22:38:15,300 d.e.f DEBUG IP: 192.168.0.1 User: jim A message at DEBUG level with 2 parameters
|
|||
|
|
2010-09-06 22:38:15,301 d.e.f ERROR IP: 127.0.0.1 User: sheila A message at ERROR level with 2 parameters
|
|||
|
|
2010-09-06 22:38:15,301 d.e.f DEBUG IP: 123.231.231.123 User: fred A message at DEBUG level with 2 parameters
|
|||
|
|
2010-09-06 22:38:15,301 d.e.f INFO IP: 123.231.231.123 User: fred A message at INFO level with 2 parameters
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="logging-to-a-single-file-from-multiple-processes">
|
|||
|
|
<span id="multiple-processes"></span><h2>Logging to a single file from multiple processes<a class="headerlink" href="#logging-to-a-single-file-from-multiple-processes" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Although logging is thread-safe, and logging to a single file from multiple
|
|||
|
|
threads in a single process <em>is</em> supported, logging to a single file from
|
|||
|
|
<em>multiple processes</em> is <em>not</em> supported, because there is no standard way to
|
|||
|
|
serialize access to a single file across multiple processes in Python. If you
|
|||
|
|
need to log to a single file from multiple processes, one way of doing this is
|
|||
|
|
to have all the processes log to a <code class="xref py py-class docutils literal notranslate"><span class="pre">SocketHandler</span></code>, and have a
|
|||
|
|
separate process which implements a socket server which reads from the socket
|
|||
|
|
and logs to file. (If you prefer, you can dedicate one thread in one of the
|
|||
|
|
existing processes to perform this function.)
|
|||
|
|
<a class="reference internal" href="#network-logging"><span class="std std-ref">This section</span></a> documents this approach in more detail and
|
|||
|
|
includes a working socket receiver which can be used as a starting point for you
|
|||
|
|
to adapt in your own applications.</p>
|
|||
|
|
<p>If you are using a recent version of Python which includes the
|
|||
|
|
<a class="reference internal" href="../library/multiprocessing.html#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal notranslate"><span class="pre">multiprocessing</span></code></a> module, you could write your own handler which uses the
|
|||
|
|
<a class="reference internal" href="../library/multiprocessing.html#multiprocessing.Lock" title="multiprocessing.Lock"><code class="xref py py-class docutils literal notranslate"><span class="pre">Lock</span></code></a> class from this module to serialize access to the
|
|||
|
|
file from your processes. The existing <code class="xref py py-class docutils literal notranslate"><span class="pre">FileHandler</span></code> and subclasses do
|
|||
|
|
not make use of <a class="reference internal" href="../library/multiprocessing.html#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal notranslate"><span class="pre">multiprocessing</span></code></a> at present, though they may do so in the
|
|||
|
|
future. Note that at present, the <a class="reference internal" href="../library/multiprocessing.html#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal notranslate"><span class="pre">multiprocessing</span></code></a> module does not provide
|
|||
|
|
working lock functionality on all platforms (see
|
|||
|
|
<a class="reference external" href="https://bugs.python.org/issue3770">https://bugs.python.org/issue3770</a>).</p>
|
|||
|
|
<p>Alternatively, you can use a <code class="docutils literal notranslate"><span class="pre">Queue</span></code> and a <a class="reference internal" href="../library/logging.handlers.html#logging.handlers.QueueHandler" title="logging.handlers.QueueHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">QueueHandler</span></code></a> to send
|
|||
|
|
all logging events to one of the processes in your multi-process application.
|
|||
|
|
The following example script demonstrates how you can do this; in the example
|
|||
|
|
a separate listener process listens for events sent by other processes and logs
|
|||
|
|
them according to its own logging configuration. Although the example only
|
|||
|
|
demonstrates one way of doing it (for example, you may want to use a listener
|
|||
|
|
thread rather than a separate listener process – the implementation would be
|
|||
|
|
analogous) it does allow for completely different logging configurations for
|
|||
|
|
the listener and the other processes in your application, and can be used as
|
|||
|
|
the basis for code meeting your own specific requirements:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="c1"># You'll need these imports in your own code</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging.handlers</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">multiprocessing</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># Next two import lines for this demo only</span>
|
|||
|
|
<span class="kn">from</span> <span class="nn">random</span> <span class="k">import</span> <span class="n">choice</span><span class="p">,</span> <span class="n">random</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">time</span>
|
|||
|
|
|
|||
|
|
<span class="c1">#</span>
|
|||
|
|
<span class="c1"># Because you'll want to define the logging configurations for listener and workers, the</span>
|
|||
|
|
<span class="c1"># listener and worker process functions take a configurer parameter which is a callable</span>
|
|||
|
|
<span class="c1"># for configuring logging for that process. These functions are also passed the queue,</span>
|
|||
|
|
<span class="c1"># which they use for communication.</span>
|
|||
|
|
<span class="c1">#</span>
|
|||
|
|
<span class="c1"># In practice, you can configure the listener however you want, but note that in this</span>
|
|||
|
|
<span class="c1"># simple example, the listener does not apply level or filter logic to received records.</span>
|
|||
|
|
<span class="c1"># In practice, you would probably want to do this logic in the worker processes, to avoid</span>
|
|||
|
|
<span class="c1"># sending events which would be filtered out between processes.</span>
|
|||
|
|
<span class="c1">#</span>
|
|||
|
|
<span class="c1"># The size of the rotated files is made small so you can see the results easily.</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">listener_configurer</span><span class="p">():</span>
|
|||
|
|
<span class="n">root</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">()</span>
|
|||
|
|
<span class="n">h</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">handlers</span><span class="o">.</span><span class="n">RotatingFileHandler</span><span class="p">(</span><span class="s1">'mptest.log'</span><span class="p">,</span> <span class="s1">'a'</span><span class="p">,</span> <span class="mi">300</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>
|
|||
|
|
<span class="n">f</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span><span class="s1">'</span><span class="si">%(asctime)s</span><span class="s1"> </span><span class="si">%(processName)-10s</span><span class="s1"> </span><span class="si">%(name)s</span><span class="s1"> </span><span class="si">%(levelname)-8s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">)</span>
|
|||
|
|
<span class="n">h</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
|
|||
|
|
<span class="n">root</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">h</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># This is the listener process top-level loop: wait for logging events</span>
|
|||
|
|
<span class="c1"># (LogRecords)on the queue and handle them, quit when you get a None for a</span>
|
|||
|
|
<span class="c1"># LogRecord.</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">listener_process</span><span class="p">(</span><span class="n">queue</span><span class="p">,</span> <span class="n">configurer</span><span class="p">):</span>
|
|||
|
|
<span class="n">configurer</span><span class="p">()</span>
|
|||
|
|
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
|
|||
|
|
<span class="k">try</span><span class="p">:</span>
|
|||
|
|
<span class="n">record</span> <span class="o">=</span> <span class="n">queue</span><span class="o">.</span><span class="n">get</span><span class="p">()</span>
|
|||
|
|
<span class="k">if</span> <span class="n">record</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span> <span class="c1"># We send this as a sentinel to tell the listener to quit.</span>
|
|||
|
|
<span class="k">break</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="n">record</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">handle</span><span class="p">(</span><span class="n">record</span><span class="p">)</span> <span class="c1"># No level or filter logic applied - just do it!</span>
|
|||
|
|
<span class="k">except</span> <span class="ne">Exception</span><span class="p">:</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">sys</span><span class="o">,</span> <span class="nn">traceback</span>
|
|||
|
|
<span class="nb">print</span><span class="p">(</span><span class="s1">'Whoops! Problem:'</span><span class="p">,</span> <span class="n">file</span><span class="o">=</span><span class="n">sys</span><span class="o">.</span><span class="n">stderr</span><span class="p">)</span>
|
|||
|
|
<span class="n">traceback</span><span class="o">.</span><span class="n">print_exc</span><span class="p">(</span><span class="n">file</span><span class="o">=</span><span class="n">sys</span><span class="o">.</span><span class="n">stderr</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># Arrays used for random selections in this demo</span>
|
|||
|
|
|
|||
|
|
<span class="n">LEVELS</span> <span class="o">=</span> <span class="p">[</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">WARNING</span><span class="p">,</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">ERROR</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">CRITICAL</span><span class="p">]</span>
|
|||
|
|
|
|||
|
|
<span class="n">LOGGERS</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'a.b.c'</span><span class="p">,</span> <span class="s1">'d.e.f'</span><span class="p">]</span>
|
|||
|
|
|
|||
|
|
<span class="n">MESSAGES</span> <span class="o">=</span> <span class="p">[</span>
|
|||
|
|
<span class="s1">'Random message #1'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'Random message #2'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'Random message #3'</span><span class="p">,</span>
|
|||
|
|
<span class="p">]</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># The worker configuration is done at the start of the worker process run.</span>
|
|||
|
|
<span class="c1"># Note that on Windows you can't rely on fork semantics, so each process</span>
|
|||
|
|
<span class="c1"># will run the logging configuration code when it starts.</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">worker_configurer</span><span class="p">(</span><span class="n">queue</span><span class="p">):</span>
|
|||
|
|
<span class="n">h</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">handlers</span><span class="o">.</span><span class="n">QueueHandler</span><span class="p">(</span><span class="n">queue</span><span class="p">)</span> <span class="c1"># Just the one handler needed</span>
|
|||
|
|
<span class="n">root</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">()</span>
|
|||
|
|
<span class="n">root</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">h</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># send all messages, for demo; no other level or filter logic applied.</span>
|
|||
|
|
<span class="n">root</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># This is the worker process top-level loop, which just logs ten events with</span>
|
|||
|
|
<span class="c1"># random intervening delays before terminating.</span>
|
|||
|
|
<span class="c1"># The print messages are just so you know it's doing something!</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">worker_process</span><span class="p">(</span><span class="n">queue</span><span class="p">,</span> <span class="n">configurer</span><span class="p">):</span>
|
|||
|
|
<span class="n">configurer</span><span class="p">(</span><span class="n">queue</span><span class="p">)</span>
|
|||
|
|
<span class="n">name</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="o">.</span><span class="n">current_process</span><span class="p">()</span><span class="o">.</span><span class="n">name</span>
|
|||
|
|
<span class="nb">print</span><span class="p">(</span><span class="s1">'Worker started: </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="n">name</span><span class="p">)</span>
|
|||
|
|
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
|
|||
|
|
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="n">random</span><span class="p">())</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="n">choice</span><span class="p">(</span><span class="n">LOGGERS</span><span class="p">))</span>
|
|||
|
|
<span class="n">level</span> <span class="o">=</span> <span class="n">choice</span><span class="p">(</span><span class="n">LEVELS</span><span class="p">)</span>
|
|||
|
|
<span class="n">message</span> <span class="o">=</span> <span class="n">choice</span><span class="p">(</span><span class="n">MESSAGES</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">level</span><span class="p">,</span> <span class="n">message</span><span class="p">)</span>
|
|||
|
|
<span class="nb">print</span><span class="p">(</span><span class="s1">'Worker finished: </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="n">name</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># Here's where the demo gets orchestrated. Create the queue, create and start</span>
|
|||
|
|
<span class="c1"># the listener, create ten workers and start them, wait for them to finish,</span>
|
|||
|
|
<span class="c1"># then send a None to the queue to tell the listener to finish.</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
|
|||
|
|
<span class="n">queue</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="o">.</span><span class="n">Queue</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
|
|||
|
|
<span class="n">listener</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="o">.</span><span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">listener_process</span><span class="p">,</span>
|
|||
|
|
<span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">queue</span><span class="p">,</span> <span class="n">listener_configurer</span><span class="p">))</span>
|
|||
|
|
<span class="n">listener</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
|
|||
|
|
<span class="n">workers</span> <span class="o">=</span> <span class="p">[]</span>
|
|||
|
|
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
|
|||
|
|
<span class="n">worker</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="o">.</span><span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">worker_process</span><span class="p">,</span>
|
|||
|
|
<span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">queue</span><span class="p">,</span> <span class="n">worker_configurer</span><span class="p">))</span>
|
|||
|
|
<span class="n">workers</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">worker</span><span class="p">)</span>
|
|||
|
|
<span class="n">worker</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
|
|||
|
|
<span class="k">for</span> <span class="n">w</span> <span class="ow">in</span> <span class="n">workers</span><span class="p">:</span>
|
|||
|
|
<span class="n">w</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
|
|||
|
|
<span class="n">queue</span><span class="o">.</span><span class="n">put_nowait</span><span class="p">(</span><span class="kc">None</span><span class="p">)</span>
|
|||
|
|
<span class="n">listener</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">main</span><span class="p">()</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>A variant of the above script keeps the logging in the main process, in a
|
|||
|
|
separate thread:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging.config</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging.handlers</span>
|
|||
|
|
<span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Queue</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">random</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">threading</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">time</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">logger_thread</span><span class="p">(</span><span class="n">q</span><span class="p">):</span>
|
|||
|
|
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
|
|||
|
|
<span class="n">record</span> <span class="o">=</span> <span class="n">q</span><span class="o">.</span><span class="n">get</span><span class="p">()</span>
|
|||
|
|
<span class="k">if</span> <span class="n">record</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
|||
|
|
<span class="k">break</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="n">record</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">handle</span><span class="p">(</span><span class="n">record</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">worker_process</span><span class="p">(</span><span class="n">q</span><span class="p">):</span>
|
|||
|
|
<span class="n">qh</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">handlers</span><span class="o">.</span><span class="n">QueueHandler</span><span class="p">(</span><span class="n">q</span><span class="p">)</span>
|
|||
|
|
<span class="n">root</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">()</span>
|
|||
|
|
<span class="n">root</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
<span class="n">root</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">qh</span><span class="p">)</span>
|
|||
|
|
<span class="n">levels</span> <span class="o">=</span> <span class="p">[</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">WARNING</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">ERROR</span><span class="p">,</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">CRITICAL</span><span class="p">]</span>
|
|||
|
|
<span class="n">loggers</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'foo'</span><span class="p">,</span> <span class="s1">'foo.bar'</span><span class="p">,</span> <span class="s1">'foo.bar.baz'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'spam'</span><span class="p">,</span> <span class="s1">'spam.ham'</span><span class="p">,</span> <span class="s1">'spam.ham.eggs'</span><span class="p">]</span>
|
|||
|
|
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">100</span><span class="p">):</span>
|
|||
|
|
<span class="n">lvl</span> <span class="o">=</span> <span class="n">random</span><span class="o">.</span><span class="n">choice</span><span class="p">(</span><span class="n">levels</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="n">random</span><span class="o">.</span><span class="n">choice</span><span class="p">(</span><span class="n">loggers</span><span class="p">))</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">lvl</span><span class="p">,</span> <span class="s1">'Message no. </span><span class="si">%d</span><span class="s1">'</span><span class="p">,</span> <span class="n">i</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">q</span> <span class="o">=</span> <span class="n">Queue</span><span class="p">()</span>
|
|||
|
|
<span class="n">d</span> <span class="o">=</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'version'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatters'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'detailed'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.Formatter'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'format'</span><span class="p">:</span> <span class="s1">'</span><span class="si">%(asctime)s</span><span class="s1"> </span><span class="si">%(name)-15s</span><span class="s1"> </span><span class="si">%(levelname)-8s</span><span class="s1"> </span><span class="si">%(processName)-10s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'console'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.StreamHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'INFO'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'file'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.FileHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'filename'</span><span class="p">:</span> <span class="s1">'mplog.log'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'mode'</span><span class="p">:</span> <span class="s1">'w'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatter'</span><span class="p">:</span> <span class="s1">'detailed'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'foofile'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.FileHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'filename'</span><span class="p">:</span> <span class="s1">'mplog-foo.log'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'mode'</span><span class="p">:</span> <span class="s1">'w'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatter'</span><span class="p">:</span> <span class="s1">'detailed'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'errors'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.FileHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'filename'</span><span class="p">:</span> <span class="s1">'mplog-errors.log'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'mode'</span><span class="p">:</span> <span class="s1">'w'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'ERROR'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatter'</span><span class="p">:</span> <span class="s1">'detailed'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'loggers'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'foo'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'foofile'</span><span class="p">]</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'root'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'DEBUG'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'console'</span><span class="p">,</span> <span class="s1">'file'</span><span class="p">,</span> <span class="s1">'errors'</span><span class="p">]</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="n">workers</span> <span class="o">=</span> <span class="p">[]</span>
|
|||
|
|
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">):</span>
|
|||
|
|
<span class="n">wp</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">worker_process</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s1">'worker </span><span class="si">%d</span><span class="s1">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">),</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">q</span><span class="p">,))</span>
|
|||
|
|
<span class="n">workers</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">wp</span><span class="p">)</span>
|
|||
|
|
<span class="n">wp</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">dictConfig</span><span class="p">(</span><span class="n">d</span><span class="p">)</span>
|
|||
|
|
<span class="n">lp</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">logger_thread</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">q</span><span class="p">,))</span>
|
|||
|
|
<span class="n">lp</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
|
|||
|
|
<span class="c1"># At this point, the main process could do some useful work of its own</span>
|
|||
|
|
<span class="c1"># Once it's done that, it can wait for the workers to terminate...</span>
|
|||
|
|
<span class="k">for</span> <span class="n">wp</span> <span class="ow">in</span> <span class="n">workers</span><span class="p">:</span>
|
|||
|
|
<span class="n">wp</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
|
|||
|
|
<span class="c1"># And now tell the logging thread to finish up, too</span>
|
|||
|
|
<span class="n">q</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="kc">None</span><span class="p">)</span>
|
|||
|
|
<span class="n">lp</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>This variant shows how you can e.g. apply configuration for particular loggers
|
|||
|
|
- e.g. the <code class="docutils literal notranslate"><span class="pre">foo</span></code> logger has a special handler which stores all events in the
|
|||
|
|
<code class="docutils literal notranslate"><span class="pre">foo</span></code> subsystem in a file <code class="docutils literal notranslate"><span class="pre">mplog-foo.log</span></code>. This will be used by the logging
|
|||
|
|
machinery in the main process (even though the logging events are generated in
|
|||
|
|
the worker processes) to direct the messages to the appropriate destinations.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="using-file-rotation">
|
|||
|
|
<h2>Using file rotation<a class="headerlink" href="#using-file-rotation" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Sometimes you want to let a log file grow to a certain size, then open a new
|
|||
|
|
file and log to that. You may want to keep a certain number of these files, and
|
|||
|
|
when that many files have been created, rotate the files so that the number of
|
|||
|
|
files and the size of the files both remain bounded. For this usage pattern, the
|
|||
|
|
logging package provides a <code class="xref py py-class docutils literal notranslate"><span class="pre">RotatingFileHandler</span></code>:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">glob</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging.handlers</span>
|
|||
|
|
|
|||
|
|
<span class="n">LOG_FILENAME</span> <span class="o">=</span> <span class="s1">'logging_rotatingfile_example.out'</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># Set up a specific logger with our desired output level</span>
|
|||
|
|
<span class="n">my_logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'MyLogger'</span><span class="p">)</span>
|
|||
|
|
<span class="n">my_logger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># Add the log message handler to the logger</span>
|
|||
|
|
<span class="n">handler</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">handlers</span><span class="o">.</span><span class="n">RotatingFileHandler</span><span class="p">(</span>
|
|||
|
|
<span class="n">LOG_FILENAME</span><span class="p">,</span> <span class="n">maxBytes</span><span class="o">=</span><span class="mi">20</span><span class="p">,</span> <span class="n">backupCount</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="n">my_logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">handler</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># Log some messages</span>
|
|||
|
|
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">20</span><span class="p">):</span>
|
|||
|
|
<span class="n">my_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'i = </span><span class="si">%d</span><span class="s1">'</span> <span class="o">%</span> <span class="n">i</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># See what files are created</span>
|
|||
|
|
<span class="n">logfiles</span> <span class="o">=</span> <span class="n">glob</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s1">'</span><span class="si">%s</span><span class="s1">*'</span> <span class="o">%</span> <span class="n">LOG_FILENAME</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">for</span> <span class="n">filename</span> <span class="ow">in</span> <span class="n">logfiles</span><span class="p">:</span>
|
|||
|
|
<span class="nb">print</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>The result should be 6 separate files, each with part of the log history for the
|
|||
|
|
application:</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>logging_rotatingfile_example.out
|
|||
|
|
logging_rotatingfile_example.out.1
|
|||
|
|
logging_rotatingfile_example.out.2
|
|||
|
|
logging_rotatingfile_example.out.3
|
|||
|
|
logging_rotatingfile_example.out.4
|
|||
|
|
logging_rotatingfile_example.out.5
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>The most current file is always <code class="file docutils literal notranslate"><span class="pre">logging_rotatingfile_example.out</span></code>,
|
|||
|
|
and each time it reaches the size limit it is renamed with the suffix
|
|||
|
|
<code class="docutils literal notranslate"><span class="pre">.1</span></code>. Each of the existing backup files is renamed to increment the suffix
|
|||
|
|
(<code class="docutils literal notranslate"><span class="pre">.1</span></code> becomes <code class="docutils literal notranslate"><span class="pre">.2</span></code>, etc.) and the <code class="docutils literal notranslate"><span class="pre">.6</span></code> file is erased.</p>
|
|||
|
|
<p>Obviously this example sets the log length much too small as an extreme
|
|||
|
|
example. You would want to set <em>maxBytes</em> to an appropriate value.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="use-of-alternative-formatting-styles">
|
|||
|
|
<span id="format-styles"></span><h2>Use of alternative formatting styles<a class="headerlink" href="#use-of-alternative-formatting-styles" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>When logging was added to the Python standard library, the only way of
|
|||
|
|
formatting messages with variable content was to use the %-formatting
|
|||
|
|
method. Since then, Python has gained two new formatting approaches:
|
|||
|
|
<a class="reference internal" href="../library/string.html#string.Template" title="string.Template"><code class="xref py py-class docutils literal notranslate"><span class="pre">string.Template</span></code></a> (added in Python 2.4) and <a class="reference internal" href="../library/stdtypes.html#str.format" title="str.format"><code class="xref py py-meth docutils literal notranslate"><span class="pre">str.format()</span></code></a>
|
|||
|
|
(added in Python 2.6).</p>
|
|||
|
|
<p>Logging (as of 3.2) provides improved support for these two additional
|
|||
|
|
formatting styles. The <code class="xref py py-class docutils literal notranslate"><span class="pre">Formatter</span></code> class been enhanced to take an
|
|||
|
|
additional, optional keyword parameter named <code class="docutils literal notranslate"><span class="pre">style</span></code>. This defaults to
|
|||
|
|
<code class="docutils literal notranslate"><span class="pre">'%'</span></code>, but other possible values are <code class="docutils literal notranslate"><span class="pre">'{'</span></code> and <code class="docutils literal notranslate"><span class="pre">'$'</span></code>, which correspond
|
|||
|
|
to the other two formatting styles. Backwards compatibility is maintained by
|
|||
|
|
default (as you would expect), but by explicitly specifying a style parameter,
|
|||
|
|
you get the ability to specify format strings which work with
|
|||
|
|
<a class="reference internal" href="../library/stdtypes.html#str.format" title="str.format"><code class="xref py py-meth docutils literal notranslate"><span class="pre">str.format()</span></code></a> or <a class="reference internal" href="../library/string.html#string.Template" title="string.Template"><code class="xref py py-class docutils literal notranslate"><span class="pre">string.Template</span></code></a>. Here’s an example console
|
|||
|
|
session to show the possibilities:</p>
|
|||
|
|
<div class="highlight-pycon notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">root</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">()</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">root</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">handler</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">StreamHandler</span><span class="p">()</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">bf</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span><span class="s1">'{asctime} {name} {levelname:8s} {message}'</span><span class="p">,</span>
|
|||
|
|
<span class="gp">... </span> <span class="n">style</span><span class="o">=</span><span class="s1">'{'</span><span class="p">)</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">handler</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">bf</span><span class="p">)</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">root</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">handler</span><span class="p">)</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'foo.bar'</span><span class="p">)</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'This is a DEBUG message'</span><span class="p">)</span>
|
|||
|
|
<span class="go">2010-10-28 15:11:55,341 foo.bar DEBUG This is a DEBUG message</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">logger</span><span class="o">.</span><span class="n">critical</span><span class="p">(</span><span class="s1">'This is a CRITICAL message'</span><span class="p">)</span>
|
|||
|
|
<span class="go">2010-10-28 15:12:11,526 foo.bar CRITICAL This is a CRITICAL message</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span><span class="s1">'$asctime $name ${levelname} $message'</span><span class="p">,</span>
|
|||
|
|
<span class="gp">... </span> <span class="n">style</span><span class="o">=</span><span class="s1">'$'</span><span class="p">)</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">handler</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">df</span><span class="p">)</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'This is a DEBUG message'</span><span class="p">)</span>
|
|||
|
|
<span class="go">2010-10-28 15:13:06,924 foo.bar DEBUG This is a DEBUG message</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">logger</span><span class="o">.</span><span class="n">critical</span><span class="p">(</span><span class="s1">'This is a CRITICAL message'</span><span class="p">)</span>
|
|||
|
|
<span class="go">2010-10-28 15:13:11,494 foo.bar CRITICAL This is a CRITICAL message</span>
|
|||
|
|
<span class="go">>>></span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Note that the formatting of logging messages for final output to logs is
|
|||
|
|
completely independent of how an individual logging message is constructed.
|
|||
|
|
That can still use %-formatting, as shown here:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s1">'This is an</span><span class="si">%s</span><span class="s1"> </span><span class="si">%s</span><span class="s1"> </span><span class="si">%s</span><span class="s1">'</span><span class="p">,</span> <span class="s1">'other,'</span><span class="p">,</span> <span class="s1">'ERROR,'</span><span class="p">,</span> <span class="s1">'message'</span><span class="p">)</span>
|
|||
|
|
<span class="go">2010-10-28 15:19:29,833 foo.bar ERROR This is another, ERROR, message</span>
|
|||
|
|
<span class="go">>>></span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Logging calls (<code class="docutils literal notranslate"><span class="pre">logger.debug()</span></code>, <code class="docutils literal notranslate"><span class="pre">logger.info()</span></code> etc.) only take
|
|||
|
|
positional parameters for the actual logging message itself, with keyword
|
|||
|
|
parameters used only for determining options for how to handle the actual
|
|||
|
|
logging call (e.g. the <code class="docutils literal notranslate"><span class="pre">exc_info</span></code> keyword parameter to indicate that
|
|||
|
|
traceback information should be logged, or the <code class="docutils literal notranslate"><span class="pre">extra</span></code> keyword parameter
|
|||
|
|
to indicate additional contextual information to be added to the log). So
|
|||
|
|
you cannot directly make logging calls using <a class="reference internal" href="../library/stdtypes.html#str.format" title="str.format"><code class="xref py py-meth docutils literal notranslate"><span class="pre">str.format()</span></code></a> or
|
|||
|
|
<a class="reference internal" href="../library/string.html#string.Template" title="string.Template"><code class="xref py py-class docutils literal notranslate"><span class="pre">string.Template</span></code></a> syntax, because internally the logging package
|
|||
|
|
uses %-formatting to merge the format string and the variable arguments.
|
|||
|
|
There would be no changing this while preserving backward compatibility, since
|
|||
|
|
all logging calls which are out there in existing code will be using %-format
|
|||
|
|
strings.</p>
|
|||
|
|
<p>There is, however, a way that you can use {}- and $- formatting to construct
|
|||
|
|
your individual log messages. Recall that for a message you can use an
|
|||
|
|
arbitrary object as a message format string, and that the logging package will
|
|||
|
|
call <code class="docutils literal notranslate"><span class="pre">str()</span></code> on that object to get the actual format string. Consider the
|
|||
|
|
following two classes:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">BraceMessage</span><span class="p">:</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">fmt</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">fmt</span> <span class="o">=</span> <span class="n">fmt</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">args</span> <span class="o">=</span> <span class="n">args</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">kwargs</span> <span class="o">=</span> <span class="n">kwargs</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">fmt</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="o">*</span><span class="bp">self</span><span class="o">.</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="bp">self</span><span class="o">.</span><span class="n">kwargs</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">DollarMessage</span><span class="p">:</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">fmt</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">fmt</span> <span class="o">=</span> <span class="n">fmt</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">kwargs</span> <span class="o">=</span> <span class="n">kwargs</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="kn">from</span> <span class="nn">string</span> <span class="k">import</span> <span class="n">Template</span>
|
|||
|
|
<span class="k">return</span> <span class="n">Template</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">fmt</span><span class="p">)</span><span class="o">.</span><span class="n">substitute</span><span class="p">(</span><span class="o">**</span><span class="bp">self</span><span class="o">.</span><span class="n">kwargs</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Either of these can be used in place of a format string, to allow {}- or
|
|||
|
|
$-formatting to be used to build the actual “message” part which appears in the
|
|||
|
|
formatted log output in place of “%(message)s” or “{message}” or “$message”.
|
|||
|
|
It’s a little unwieldy to use the class names whenever you want to log
|
|||
|
|
something, but it’s quite palatable if you use an alias such as __ (double
|
|||
|
|
underscore — not to be confused with _, the single underscore used as a
|
|||
|
|
synonym/alias for <a class="reference internal" href="../library/gettext.html#gettext.gettext" title="gettext.gettext"><code class="xref py py-func docutils literal notranslate"><span class="pre">gettext.gettext()</span></code></a> or its brethren).</p>
|
|||
|
|
<p>The above classes are not included in Python, though they’re easy enough to
|
|||
|
|
copy and paste into your own code. They can be used as follows (assuming that
|
|||
|
|
they’re declared in a module called <code class="docutils literal notranslate"><span class="pre">wherever</span></code>):</p>
|
|||
|
|
<div class="highlight-pycon notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">wherever</span> <span class="kn">import</span> <span class="n">BraceMessage</span> <span class="k">as</span> <span class="n">__</span>
|
|||
|
|
<span class="gp">>>> </span><span class="k">print</span><span class="p">(</span><span class="n">__</span><span class="p">(</span><span class="s1">'Message with {0} {name}'</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s1">'placeholders'</span><span class="p">))</span>
|
|||
|
|
<span class="go">Message with 2 placeholders</span>
|
|||
|
|
<span class="gp">>>> </span><span class="k">class</span> <span class="nc">Point</span><span class="p">:</span> <span class="k">pass</span>
|
|||
|
|
<span class="gp">...</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">p</span> <span class="o">=</span> <span class="n">Point</span><span class="p">()</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">p</span><span class="o">.</span><span class="n">x</span> <span class="o">=</span> <span class="mf">0.5</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">p</span><span class="o">.</span><span class="n">y</span> <span class="o">=</span> <span class="mf">0.5</span>
|
|||
|
|
<span class="gp">>>> </span><span class="k">print</span><span class="p">(</span><span class="n">__</span><span class="p">(</span><span class="s1">'Message with coordinates: ({point.x:.2f}, {point.y:.2f})'</span><span class="p">,</span>
|
|||
|
|
<span class="gp">... </span> <span class="n">point</span><span class="o">=</span><span class="n">p</span><span class="p">))</span>
|
|||
|
|
<span class="go">Message with coordinates: (0.50, 0.50)</span>
|
|||
|
|
<span class="gp">>>> </span><span class="kn">from</span> <span class="nn">wherever</span> <span class="kn">import</span> <span class="n">DollarMessage</span> <span class="k">as</span> <span class="n">__</span>
|
|||
|
|
<span class="gp">>>> </span><span class="k">print</span><span class="p">(</span><span class="n">__</span><span class="p">(</span><span class="s1">'Message with $num $what'</span><span class="p">,</span> <span class="n">num</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">what</span><span class="o">=</span><span class="s1">'placeholders'</span><span class="p">))</span>
|
|||
|
|
<span class="go">Message with 2 placeholders</span>
|
|||
|
|
<span class="go">>>></span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>While the above examples use <code class="docutils literal notranslate"><span class="pre">print()</span></code> to show how the formatting works, you
|
|||
|
|
would of course use <code class="docutils literal notranslate"><span class="pre">logger.debug()</span></code> or similar to actually log using this
|
|||
|
|
approach.</p>
|
|||
|
|
<p>One thing to note is that you pay no significant performance penalty with this
|
|||
|
|
approach: the actual formatting happens not when you make the logging call, but
|
|||
|
|
when (and if) the logged message is actually about to be output to a log by a
|
|||
|
|
handler. So the only slightly unusual thing which might trip you up is that the
|
|||
|
|
parentheses go around the format string and the arguments, not just the format
|
|||
|
|
string. That’s because the __ notation is just syntax sugar for a constructor
|
|||
|
|
call to one of the XXXMessage classes.</p>
|
|||
|
|
<p>If you prefer, you can use a <code class="xref py py-class docutils literal notranslate"><span class="pre">LoggerAdapter</span></code> to achieve a similar effect
|
|||
|
|
to the above, as in the following example:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">Message</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">fmt</span><span class="p">,</span> <span class="n">args</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">fmt</span> <span class="o">=</span> <span class="n">fmt</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">args</span> <span class="o">=</span> <span class="n">args</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">fmt</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="o">*</span><span class="bp">self</span><span class="o">.</span><span class="n">args</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">StyleAdapter</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">LoggerAdapter</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">logger</span><span class="p">,</span> <span class="n">extra</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
|||
|
|
<span class="nb">super</span><span class="p">(</span><span class="n">StyleAdapter</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">logger</span><span class="p">,</span> <span class="n">extra</span> <span class="ow">or</span> <span class="p">{})</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">log</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">level</span><span class="p">,</span> <span class="n">msg</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|||
|
|
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">isEnabledFor</span><span class="p">(</span><span class="n">level</span><span class="p">):</span>
|
|||
|
|
<span class="n">msg</span><span class="p">,</span> <span class="n">kwargs</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">process</span><span class="p">(</span><span class="n">msg</span><span class="p">,</span> <span class="n">kwargs</span><span class="p">)</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">_log</span><span class="p">(</span><span class="n">level</span><span class="p">,</span> <span class="n">Message</span><span class="p">(</span><span class="n">msg</span><span class="p">,</span> <span class="n">args</span><span class="p">),</span> <span class="p">(),</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">StyleAdapter</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="vm">__name__</span><span class="p">))</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'Hello, </span><span class="si">{}</span><span class="s1">'</span><span class="p">,</span> <span class="s1">'world!'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">basicConfig</span><span class="p">(</span><span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
<span class="n">main</span><span class="p">()</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>The above script should log the message <code class="docutils literal notranslate"><span class="pre">Hello,</span> <span class="pre">world!</span></code> when run with
|
|||
|
|
Python 3.2 or later.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="customizing-logrecord">
|
|||
|
|
<span id="custom-logrecord"></span><h2>Customizing <code class="docutils literal notranslate"><span class="pre">LogRecord</span></code><a class="headerlink" href="#customizing-logrecord" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Every logging event is represented by a <a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal notranslate"><span class="pre">LogRecord</span></code></a> instance.
|
|||
|
|
When an event is logged and not filtered out by a logger’s level, a
|
|||
|
|
<a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal notranslate"><span class="pre">LogRecord</span></code></a> is created, populated with information about the event and
|
|||
|
|
then passed to the handlers for that logger (and its ancestors, up to and
|
|||
|
|
including the logger where further propagation up the hierarchy is disabled).
|
|||
|
|
Before Python 3.2, there were only two places where this creation was done:</p>
|
|||
|
|
<ul class="simple">
|
|||
|
|
<li><p><a class="reference internal" href="../library/logging.html#logging.Logger.makeRecord" title="logging.Logger.makeRecord"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Logger.makeRecord()</span></code></a>, which is called in the normal process of
|
|||
|
|
logging an event. This invoked <a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal notranslate"><span class="pre">LogRecord</span></code></a> directly to create an
|
|||
|
|
instance.</p></li>
|
|||
|
|
<li><p><a class="reference internal" href="../library/logging.html#logging.makeLogRecord" title="logging.makeLogRecord"><code class="xref py py-func docutils literal notranslate"><span class="pre">makeLogRecord()</span></code></a>, which is called with a dictionary containing
|
|||
|
|
attributes to be added to the LogRecord. This is typically invoked when a
|
|||
|
|
suitable dictionary has been received over the network (e.g. in pickle form
|
|||
|
|
via a <a class="reference internal" href="../library/logging.handlers.html#logging.handlers.SocketHandler" title="logging.handlers.SocketHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">SocketHandler</span></code></a>, or in JSON form via an
|
|||
|
|
<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.HTTPHandler" title="logging.handlers.HTTPHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">HTTPHandler</span></code></a>).</p></li>
|
|||
|
|
</ul>
|
|||
|
|
<p>This has usually meant that if you need to do anything special with a
|
|||
|
|
<a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal notranslate"><span class="pre">LogRecord</span></code></a>, you’ve had to do one of the following.</p>
|
|||
|
|
<ul class="simple">
|
|||
|
|
<li><p>Create your own <a class="reference internal" href="../library/logging.html#logging.Logger" title="logging.Logger"><code class="xref py py-class docutils literal notranslate"><span class="pre">Logger</span></code></a> subclass, which overrides
|
|||
|
|
<a class="reference internal" href="../library/logging.html#logging.Logger.makeRecord" title="logging.Logger.makeRecord"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Logger.makeRecord()</span></code></a>, and set it using <a class="reference internal" href="../library/logging.html#logging.setLoggerClass" title="logging.setLoggerClass"><code class="xref py py-func docutils literal notranslate"><span class="pre">setLoggerClass()</span></code></a>
|
|||
|
|
before any loggers that you care about are instantiated.</p></li>
|
|||
|
|
<li><p>Add a <a class="reference internal" href="../library/logging.html#logging.Filter" title="logging.Filter"><code class="xref py py-class docutils literal notranslate"><span class="pre">Filter</span></code></a> to a logger or handler, which does the
|
|||
|
|
necessary special manipulation you need when its
|
|||
|
|
<a class="reference internal" href="../library/logging.html#logging.Filter.filter" title="logging.Filter.filter"><code class="xref py py-meth docutils literal notranslate"><span class="pre">filter()</span></code></a> method is called.</p></li>
|
|||
|
|
</ul>
|
|||
|
|
<p>The first approach would be a little unwieldy in the scenario where (say)
|
|||
|
|
several different libraries wanted to do different things. Each would attempt
|
|||
|
|
to set its own <a class="reference internal" href="../library/logging.html#logging.Logger" title="logging.Logger"><code class="xref py py-class docutils literal notranslate"><span class="pre">Logger</span></code></a> subclass, and the one which did this last would
|
|||
|
|
win.</p>
|
|||
|
|
<p>The second approach works reasonably well for many cases, but does not allow
|
|||
|
|
you to e.g. use a specialized subclass of <a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal notranslate"><span class="pre">LogRecord</span></code></a>. Library
|
|||
|
|
developers can set a suitable filter on their loggers, but they would have to
|
|||
|
|
remember to do this every time they introduced a new logger (which they would
|
|||
|
|
do simply by adding new packages or modules and doing</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>at module level). It’s probably one too many things to think about. Developers
|
|||
|
|
could also add the filter to a <a class="reference internal" href="../library/logging.handlers.html#logging.NullHandler" title="logging.NullHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">NullHandler</span></code></a> attached to their
|
|||
|
|
top-level logger, but this would not be invoked if an application developer
|
|||
|
|
attached a handler to a lower-level library logger — so output from that
|
|||
|
|
handler would not reflect the intentions of the library developer.</p>
|
|||
|
|
<p>In Python 3.2 and later, <a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal notranslate"><span class="pre">LogRecord</span></code></a> creation is done through a
|
|||
|
|
factory, which you can specify. The factory is just a callable you can set with
|
|||
|
|
<a class="reference internal" href="../library/logging.html#logging.setLogRecordFactory" title="logging.setLogRecordFactory"><code class="xref py py-func docutils literal notranslate"><span class="pre">setLogRecordFactory()</span></code></a>, and interrogate with
|
|||
|
|
<a class="reference internal" href="../library/logging.html#logging.getLogRecordFactory" title="logging.getLogRecordFactory"><code class="xref py py-func docutils literal notranslate"><span class="pre">getLogRecordFactory()</span></code></a>. The factory is invoked with the same
|
|||
|
|
signature as the <a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal notranslate"><span class="pre">LogRecord</span></code></a> constructor, as <a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal notranslate"><span class="pre">LogRecord</span></code></a>
|
|||
|
|
is the default setting for the factory.</p>
|
|||
|
|
<p>This approach allows a custom factory to control all aspects of LogRecord
|
|||
|
|
creation. For example, you could return a subclass, or just add some additional
|
|||
|
|
attributes to the record once created, using a pattern similar to this:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">old_factory</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogRecordFactory</span><span class="p">()</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">record_factory</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|||
|
|
<span class="n">record</span> <span class="o">=</span> <span class="n">old_factory</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
|||
|
|
<span class="n">record</span><span class="o">.</span><span class="n">custom_attribute</span> <span class="o">=</span> <span class="mh">0xdecafbad</span>
|
|||
|
|
<span class="k">return</span> <span class="n">record</span>
|
|||
|
|
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">setLogRecordFactory</span><span class="p">(</span><span class="n">record_factory</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>This pattern allows different libraries to chain factories together, and as
|
|||
|
|
long as they don’t overwrite each other’s attributes or unintentionally
|
|||
|
|
overwrite the attributes provided as standard, there should be no surprises.
|
|||
|
|
However, it should be borne in mind that each link in the chain adds run-time
|
|||
|
|
overhead to all logging operations, and the technique should only be used when
|
|||
|
|
the use of a <a class="reference internal" href="../library/logging.html#logging.Filter" title="logging.Filter"><code class="xref py py-class docutils literal notranslate"><span class="pre">Filter</span></code></a> does not provide the desired result.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="subclassing-queuehandler-a-zeromq-example">
|
|||
|
|
<span id="zeromq-handlers"></span><h2>Subclassing QueueHandler - a ZeroMQ example<a class="headerlink" href="#subclassing-queuehandler-a-zeromq-example" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>You can use a <code class="xref py py-class docutils literal notranslate"><span class="pre">QueueHandler</span></code> subclass to send messages to other kinds
|
|||
|
|
of queues, for example a ZeroMQ ‘publish’ socket. In the example below,the
|
|||
|
|
socket is created separately and passed to the handler (as its ‘queue’):</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">zmq</span> <span class="c1"># using pyzmq, the Python binding for ZeroMQ</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">json</span> <span class="c1"># for serializing records portably</span>
|
|||
|
|
|
|||
|
|
<span class="n">ctx</span> <span class="o">=</span> <span class="n">zmq</span><span class="o">.</span><span class="n">Context</span><span class="p">()</span>
|
|||
|
|
<span class="n">sock</span> <span class="o">=</span> <span class="n">zmq</span><span class="o">.</span><span class="n">Socket</span><span class="p">(</span><span class="n">ctx</span><span class="p">,</span> <span class="n">zmq</span><span class="o">.</span><span class="n">PUB</span><span class="p">)</span> <span class="c1"># or zmq.PUSH, or other suitable value</span>
|
|||
|
|
<span class="n">sock</span><span class="o">.</span><span class="n">bind</span><span class="p">(</span><span class="s1">'tcp://*:5556'</span><span class="p">)</span> <span class="c1"># or wherever</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">ZeroMQSocketHandler</span><span class="p">(</span><span class="n">QueueHandler</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">enqueue</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">record</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">queue</span><span class="o">.</span><span class="n">send_json</span><span class="p">(</span><span class="n">record</span><span class="o">.</span><span class="vm">__dict__</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
|
|||
|
|
<span class="n">handler</span> <span class="o">=</span> <span class="n">ZeroMQSocketHandler</span><span class="p">(</span><span class="n">sock</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Of course there are other ways of organizing this, for example passing in the
|
|||
|
|
data needed by the handler to create the socket:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">ZeroMQSocketHandler</span><span class="p">(</span><span class="n">QueueHandler</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">uri</span><span class="p">,</span> <span class="n">socktype</span><span class="o">=</span><span class="n">zmq</span><span class="o">.</span><span class="n">PUB</span><span class="p">,</span> <span class="n">ctx</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">ctx</span> <span class="o">=</span> <span class="n">ctx</span> <span class="ow">or</span> <span class="n">zmq</span><span class="o">.</span><span class="n">Context</span><span class="p">()</span>
|
|||
|
|
<span class="n">socket</span> <span class="o">=</span> <span class="n">zmq</span><span class="o">.</span><span class="n">Socket</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">ctx</span><span class="p">,</span> <span class="n">socktype</span><span class="p">)</span>
|
|||
|
|
<span class="n">socket</span><span class="o">.</span><span class="n">bind</span><span class="p">(</span><span class="n">uri</span><span class="p">)</span>
|
|||
|
|
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">socket</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">enqueue</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">record</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">queue</span><span class="o">.</span><span class="n">send_json</span><span class="p">(</span><span class="n">record</span><span class="o">.</span><span class="vm">__dict__</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">close</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">queue</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="subclassing-queuelistener-a-zeromq-example">
|
|||
|
|
<h2>Subclassing QueueListener - a ZeroMQ example<a class="headerlink" href="#subclassing-queuelistener-a-zeromq-example" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>You can also subclass <code class="xref py py-class docutils literal notranslate"><span class="pre">QueueListener</span></code> to get messages from other kinds
|
|||
|
|
of queues, for example a ZeroMQ ‘subscribe’ socket. Here’s an example:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">ZeroMQSocketListener</span><span class="p">(</span><span class="n">QueueListener</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">uri</span><span class="p">,</span> <span class="o">*</span><span class="n">handlers</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">ctx</span> <span class="o">=</span> <span class="n">kwargs</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'ctx'</span><span class="p">)</span> <span class="ow">or</span> <span class="n">zmq</span><span class="o">.</span><span class="n">Context</span><span class="p">()</span>
|
|||
|
|
<span class="n">socket</span> <span class="o">=</span> <span class="n">zmq</span><span class="o">.</span><span class="n">Socket</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">ctx</span><span class="p">,</span> <span class="n">zmq</span><span class="o">.</span><span class="n">SUB</span><span class="p">)</span>
|
|||
|
|
<span class="n">socket</span><span class="o">.</span><span class="n">setsockopt_string</span><span class="p">(</span><span class="n">zmq</span><span class="o">.</span><span class="n">SUBSCRIBE</span><span class="p">,</span> <span class="s1">''</span><span class="p">)</span> <span class="c1"># subscribe to everything</span>
|
|||
|
|
<span class="n">socket</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">uri</span><span class="p">)</span>
|
|||
|
|
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">socket</span><span class="p">,</span> <span class="o">*</span><span class="n">handlers</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">dequeue</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="n">msg</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">queue</span><span class="o">.</span><span class="n">recv_json</span><span class="p">()</span>
|
|||
|
|
<span class="k">return</span> <span class="n">logging</span><span class="o">.</span><span class="n">makeLogRecord</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<div class="admonition seealso">
|
|||
|
|
<p class="admonition-title">See also</p>
|
|||
|
|
<dl class="simple">
|
|||
|
|
<dt>Module <a class="reference internal" href="../library/logging.html#module-logging" title="logging: Flexible event logging system for applications."><code class="xref py py-mod docutils literal notranslate"><span class="pre">logging</span></code></a></dt><dd><p>API reference for the logging module.</p>
|
|||
|
|
</dd>
|
|||
|
|
<dt>Module <a class="reference internal" href="../library/logging.config.html#module-logging.config" title="logging.config: Configuration of the logging module."><code class="xref py py-mod docutils literal notranslate"><span class="pre">logging.config</span></code></a></dt><dd><p>Configuration API for the logging module.</p>
|
|||
|
|
</dd>
|
|||
|
|
<dt>Module <a class="reference internal" href="../library/logging.handlers.html#module-logging.handlers" title="logging.handlers: Handlers for the logging module."><code class="xref py py-mod docutils literal notranslate"><span class="pre">logging.handlers</span></code></a></dt><dd><p>Useful handlers included with the logging module.</p>
|
|||
|
|
</dd>
|
|||
|
|
</dl>
|
|||
|
|
<p><a class="reference internal" href="logging.html#logging-basic-tutorial"><span class="std std-ref">A basic logging tutorial</span></a></p>
|
|||
|
|
<p><a class="reference internal" href="logging.html#logging-advanced-tutorial"><span class="std std-ref">A more advanced logging tutorial</span></a></p>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="an-example-dictionary-based-configuration">
|
|||
|
|
<h2>An example dictionary-based configuration<a class="headerlink" href="#an-example-dictionary-based-configuration" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Below is an example of a logging configuration dictionary - it’s taken from
|
|||
|
|
the <a class="reference external" href="https://docs.djangoproject.com/en/1.9/topics/logging/#configuring-logging">documentation on the Django project</a>.
|
|||
|
|
This dictionary is passed to <a class="reference internal" href="../library/logging.config.html#logging.config.dictConfig" title="logging.config.dictConfig"><code class="xref py py-func docutils literal notranslate"><span class="pre">dictConfig()</span></code></a> to put the configuration into effect:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">LOGGING</span> <span class="o">=</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'version'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'disable_existing_loggers'</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatters'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'verbose'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'format'</span><span class="p">:</span> <span class="s1">'</span><span class="si">%(levelname)s</span><span class="s1"> </span><span class="si">%(asctime)s</span><span class="s1"> </span><span class="si">%(module)s</span><span class="s1"> </span><span class="si">%(process)d</span><span class="s1"> </span><span class="si">%(thread)d</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'simple'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'format'</span><span class="p">:</span> <span class="s1">'</span><span class="si">%(levelname)s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'filters'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'special'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'()'</span><span class="p">:</span> <span class="s1">'project.logging.SpecialFilter'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'foo'</span><span class="p">:</span> <span class="s1">'bar'</span><span class="p">,</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'null'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span><span class="s1">'DEBUG'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span><span class="s1">'django.utils.log.NullHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'console'</span><span class="p">:{</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span><span class="s1">'DEBUG'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span><span class="s1">'logging.StreamHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatter'</span><span class="p">:</span> <span class="s1">'simple'</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'mail_admins'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'ERROR'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'django.utils.log.AdminEmailHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'filters'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'special'</span><span class="p">]</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'loggers'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'django'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:[</span><span class="s1">'null'</span><span class="p">],</span>
|
|||
|
|
<span class="s1">'propagate'</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span><span class="s1">'INFO'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'django.request'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'mail_admins'</span><span class="p">],</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'ERROR'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'propagate'</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'myproject.custom'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'console'</span><span class="p">,</span> <span class="s1">'mail_admins'</span><span class="p">],</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'INFO'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'filters'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'special'</span><span class="p">]</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>For more information about this configuration, you can see the <a class="reference external" href="https://docs.djangoproject.com/en/1.9/topics/logging/#configuring-logging">relevant
|
|||
|
|
section</a>
|
|||
|
|
of the Django documentation.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="using-a-rotator-and-namer-to-customize-log-rotation-processing">
|
|||
|
|
<span id="cookbook-rotator-namer"></span><h2>Using a rotator and namer to customize log rotation processing<a class="headerlink" href="#using-a-rotator-and-namer-to-customize-log-rotation-processing" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>An example of how you can define a namer and rotator is given in the following
|
|||
|
|
snippet, which shows zlib-based compression of the log file:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">namer</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
|
|||
|
|
<span class="k">return</span> <span class="n">name</span> <span class="o">+</span> <span class="s2">".gz"</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">rotator</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">dest</span><span class="p">):</span>
|
|||
|
|
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="s2">"rb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">sf</span><span class="p">:</span>
|
|||
|
|
<span class="n">data</span> <span class="o">=</span> <span class="n">sf</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
|
|||
|
|
<span class="n">compressed</span> <span class="o">=</span> <span class="n">zlib</span><span class="o">.</span><span class="n">compress</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="mi">9</span><span class="p">)</span>
|
|||
|
|
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">dest</span><span class="p">,</span> <span class="s2">"wb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">df</span><span class="p">:</span>
|
|||
|
|
<span class="n">df</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">compressed</span><span class="p">)</span>
|
|||
|
|
<span class="n">os</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="n">rh</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">handlers</span><span class="o">.</span><span class="n">RotatingFileHandler</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
|
|||
|
|
<span class="n">rh</span><span class="o">.</span><span class="n">rotator</span> <span class="o">=</span> <span class="n">rotator</span>
|
|||
|
|
<span class="n">rh</span><span class="o">.</span><span class="n">namer</span> <span class="o">=</span> <span class="n">namer</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>These are not “true” .gz files, as they are bare compressed data, with no
|
|||
|
|
“container” such as you’d find in an actual gzip file. This snippet is just
|
|||
|
|
for illustration purposes.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="a-more-elaborate-multiprocessing-example">
|
|||
|
|
<h2>A more elaborate multiprocessing example<a class="headerlink" href="#a-more-elaborate-multiprocessing-example" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>The following working example shows how logging can be used with multiprocessing
|
|||
|
|
using configuration files. The configurations are fairly simple, but serve to
|
|||
|
|
illustrate how more complex ones could be implemented in a real multiprocessing
|
|||
|
|
scenario.</p>
|
|||
|
|
<p>In the example, the main process spawns a listener process and some worker
|
|||
|
|
processes. Each of the main process, the listener and the workers have three
|
|||
|
|
separate configurations (the workers all share the same configuration). We can
|
|||
|
|
see logging in the main process, how the workers log to a QueueHandler and how
|
|||
|
|
the listener implements a QueueListener and a more complex logging
|
|||
|
|
configuration, and arranges to dispatch events received via the queue to the
|
|||
|
|
handlers specified in the configuration. Note that these configurations are
|
|||
|
|
purely illustrative, but you should be able to adapt this example to your own
|
|||
|
|
scenario.</p>
|
|||
|
|
<p>Here’s the script - the docstrings and the comments hopefully explain how it
|
|||
|
|
works:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging.config</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging.handlers</span>
|
|||
|
|
<span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Queue</span><span class="p">,</span> <span class="n">Event</span><span class="p">,</span> <span class="n">current_process</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">os</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">random</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">time</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">MyHandler</span><span class="p">:</span>
|
|||
|
|
<span class="sd">"""</span>
|
|||
|
|
<span class="sd"> A simple handler for logging events. It runs in the listener process and</span>
|
|||
|
|
<span class="sd"> dispatches events to loggers based on the name in the received record,</span>
|
|||
|
|
<span class="sd"> which then get dispatched, by the logging system, to the handlers</span>
|
|||
|
|
<span class="sd"> configured for those loggers.</span>
|
|||
|
|
<span class="sd"> """</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">handle</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">record</span><span class="p">):</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="n">record</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># The process name is transformed just to show that it's the listener</span>
|
|||
|
|
<span class="c1"># doing the logging to files and console</span>
|
|||
|
|
<span class="n">record</span><span class="o">.</span><span class="n">processName</span> <span class="o">=</span> <span class="s1">'</span><span class="si">%s</span><span class="s1"> (for </span><span class="si">%s</span><span class="s1">)'</span> <span class="o">%</span> <span class="p">(</span><span class="n">current_process</span><span class="p">()</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">record</span><span class="o">.</span><span class="n">processName</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">handle</span><span class="p">(</span><span class="n">record</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">listener_process</span><span class="p">(</span><span class="n">q</span><span class="p">,</span> <span class="n">stop_event</span><span class="p">,</span> <span class="n">config</span><span class="p">):</span>
|
|||
|
|
<span class="sd">"""</span>
|
|||
|
|
<span class="sd"> This could be done in the main process, but is just done in a separate</span>
|
|||
|
|
<span class="sd"> process for illustrative purposes.</span>
|
|||
|
|
|
|||
|
|
<span class="sd"> This initialises logging according to the specified configuration,</span>
|
|||
|
|
<span class="sd"> starts the listener and waits for the main process to signal completion</span>
|
|||
|
|
<span class="sd"> via the event. The listener is then stopped, and the process exits.</span>
|
|||
|
|
<span class="sd"> """</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">dictConfig</span><span class="p">(</span><span class="n">config</span><span class="p">)</span>
|
|||
|
|
<span class="n">listener</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">handlers</span><span class="o">.</span><span class="n">QueueListener</span><span class="p">(</span><span class="n">q</span><span class="p">,</span> <span class="n">MyHandler</span><span class="p">())</span>
|
|||
|
|
<span class="n">listener</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
|
|||
|
|
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">name</span> <span class="o">==</span> <span class="s1">'posix'</span><span class="p">:</span>
|
|||
|
|
<span class="c1"># On POSIX, the setup logger will have been configured in the</span>
|
|||
|
|
<span class="c1"># parent process, but should have been disabled following the</span>
|
|||
|
|
<span class="c1"># dictConfig call.</span>
|
|||
|
|
<span class="c1"># On Windows, since fork isn't used, the setup logger won't</span>
|
|||
|
|
<span class="c1"># exist in the child, so it would be created and the message</span>
|
|||
|
|
<span class="c1"># would appear - hence the "if posix" clause.</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'setup'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">critical</span><span class="p">(</span><span class="s1">'Should not appear, because of disabled logger ...'</span><span class="p">)</span>
|
|||
|
|
<span class="n">stop_event</span><span class="o">.</span><span class="n">wait</span><span class="p">()</span>
|
|||
|
|
<span class="n">listener</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">worker_process</span><span class="p">(</span><span class="n">config</span><span class="p">):</span>
|
|||
|
|
<span class="sd">"""</span>
|
|||
|
|
<span class="sd"> A number of these are spawned for the purpose of illustration. In</span>
|
|||
|
|
<span class="sd"> practice, they could be a heterogeneous bunch of processes rather than</span>
|
|||
|
|
<span class="sd"> ones which are identical to each other.</span>
|
|||
|
|
|
|||
|
|
<span class="sd"> This initialises logging according to the specified configuration,</span>
|
|||
|
|
<span class="sd"> and logs a hundred messages with random levels to randomly selected</span>
|
|||
|
|
<span class="sd"> loggers.</span>
|
|||
|
|
|
|||
|
|
<span class="sd"> A small sleep is added to allow other processes a chance to run. This</span>
|
|||
|
|
<span class="sd"> is not strictly needed, but it mixes the output from the different</span>
|
|||
|
|
<span class="sd"> processes a bit more than if it's left out.</span>
|
|||
|
|
<span class="sd"> """</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">dictConfig</span><span class="p">(</span><span class="n">config</span><span class="p">)</span>
|
|||
|
|
<span class="n">levels</span> <span class="o">=</span> <span class="p">[</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">WARNING</span><span class="p">,</span> <span class="n">logging</span><span class="o">.</span><span class="n">ERROR</span><span class="p">,</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">CRITICAL</span><span class="p">]</span>
|
|||
|
|
<span class="n">loggers</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'foo'</span><span class="p">,</span> <span class="s1">'foo.bar'</span><span class="p">,</span> <span class="s1">'foo.bar.baz'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'spam'</span><span class="p">,</span> <span class="s1">'spam.ham'</span><span class="p">,</span> <span class="s1">'spam.ham.eggs'</span><span class="p">]</span>
|
|||
|
|
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">name</span> <span class="o">==</span> <span class="s1">'posix'</span><span class="p">:</span>
|
|||
|
|
<span class="c1"># On POSIX, the setup logger will have been configured in the</span>
|
|||
|
|
<span class="c1"># parent process, but should have been disabled following the</span>
|
|||
|
|
<span class="c1"># dictConfig call.</span>
|
|||
|
|
<span class="c1"># On Windows, since fork isn't used, the setup logger won't</span>
|
|||
|
|
<span class="c1"># exist in the child, so it would be created and the message</span>
|
|||
|
|
<span class="c1"># would appear - hence the "if posix" clause.</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'setup'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">critical</span><span class="p">(</span><span class="s1">'Should not appear, because of disabled logger ...'</span><span class="p">)</span>
|
|||
|
|
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">100</span><span class="p">):</span>
|
|||
|
|
<span class="n">lvl</span> <span class="o">=</span> <span class="n">random</span><span class="o">.</span><span class="n">choice</span><span class="p">(</span><span class="n">levels</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="n">random</span><span class="o">.</span><span class="n">choice</span><span class="p">(</span><span class="n">loggers</span><span class="p">))</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">lvl</span><span class="p">,</span> <span class="s1">'Message no. </span><span class="si">%d</span><span class="s1">'</span><span class="p">,</span> <span class="n">i</span><span class="p">)</span>
|
|||
|
|
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">0.01</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
|
|||
|
|
<span class="n">q</span> <span class="o">=</span> <span class="n">Queue</span><span class="p">()</span>
|
|||
|
|
<span class="c1"># The main process gets a simple configuration which prints to the console.</span>
|
|||
|
|
<span class="n">config_initial</span> <span class="o">=</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'version'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatters'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'detailed'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.Formatter'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'format'</span><span class="p">:</span> <span class="s1">'</span><span class="si">%(asctime)s</span><span class="s1"> </span><span class="si">%(name)-15s</span><span class="s1"> </span><span class="si">%(levelname)-8s</span><span class="s1"> </span><span class="si">%(processName)-10s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'console'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.StreamHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'INFO'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'root'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'DEBUG'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'console'</span><span class="p">]</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="c1"># The worker process configuration is just a QueueHandler attached to the</span>
|
|||
|
|
<span class="c1"># root logger, which allows all messages to be sent to the queue.</span>
|
|||
|
|
<span class="c1"># We disable existing loggers to disable the "setup" logger used in the</span>
|
|||
|
|
<span class="c1"># parent process. This is needed on POSIX because the logger will</span>
|
|||
|
|
<span class="c1"># be there in the child following a fork().</span>
|
|||
|
|
<span class="n">config_worker</span> <span class="o">=</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'version'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'disable_existing_loggers'</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'queue'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.handlers.QueueHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'queue'</span><span class="p">:</span> <span class="n">q</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'root'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'DEBUG'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'queue'</span><span class="p">]</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="c1"># The listener process configuration shows that the full flexibility of</span>
|
|||
|
|
<span class="c1"># logging configuration is available to dispatch events to handlers however</span>
|
|||
|
|
<span class="c1"># you want.</span>
|
|||
|
|
<span class="c1"># We disable existing loggers to disable the "setup" logger used in the</span>
|
|||
|
|
<span class="c1"># parent process. This is needed on POSIX because the logger will</span>
|
|||
|
|
<span class="c1"># be there in the child following a fork().</span>
|
|||
|
|
<span class="n">config_listener</span> <span class="o">=</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'version'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'disable_existing_loggers'</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatters'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'detailed'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.Formatter'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'format'</span><span class="p">:</span> <span class="s1">'</span><span class="si">%(asctime)s</span><span class="s1"> </span><span class="si">%(name)-15s</span><span class="s1"> </span><span class="si">%(levelname)-8s</span><span class="s1"> </span><span class="si">%(processName)-10s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'simple'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.Formatter'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'format'</span><span class="p">:</span> <span class="s1">'</span><span class="si">%(name)-15s</span><span class="s1"> </span><span class="si">%(levelname)-8s</span><span class="s1"> </span><span class="si">%(processName)-10s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'console'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.StreamHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'INFO'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatter'</span><span class="p">:</span> <span class="s1">'simple'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'file'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.FileHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'filename'</span><span class="p">:</span> <span class="s1">'mplog.log'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'mode'</span><span class="p">:</span> <span class="s1">'w'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatter'</span><span class="p">:</span> <span class="s1">'detailed'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'foofile'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.FileHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'filename'</span><span class="p">:</span> <span class="s1">'mplog-foo.log'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'mode'</span><span class="p">:</span> <span class="s1">'w'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatter'</span><span class="p">:</span> <span class="s1">'detailed'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'errors'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.FileHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'filename'</span><span class="p">:</span> <span class="s1">'mplog-errors.log'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'mode'</span><span class="p">:</span> <span class="s1">'w'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'ERROR'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatter'</span><span class="p">:</span> <span class="s1">'detailed'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'loggers'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'foo'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'foofile'</span><span class="p">]</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'root'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'DEBUG'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'console'</span><span class="p">,</span> <span class="s1">'file'</span><span class="p">,</span> <span class="s1">'errors'</span><span class="p">]</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="c1"># Log some initial events, just to show that logging in the parent works</span>
|
|||
|
|
<span class="c1"># normally.</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">dictConfig</span><span class="p">(</span><span class="n">config_initial</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'setup'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'About to create workers ...'</span><span class="p">)</span>
|
|||
|
|
<span class="n">workers</span> <span class="o">=</span> <span class="p">[]</span>
|
|||
|
|
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">):</span>
|
|||
|
|
<span class="n">wp</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">worker_process</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s1">'worker </span><span class="si">%d</span><span class="s1">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">),</span>
|
|||
|
|
<span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">config_worker</span><span class="p">,))</span>
|
|||
|
|
<span class="n">workers</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">wp</span><span class="p">)</span>
|
|||
|
|
<span class="n">wp</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'Started worker: </span><span class="si">%s</span><span class="s1">'</span><span class="p">,</span> <span class="n">wp</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'About to create listener ...'</span><span class="p">)</span>
|
|||
|
|
<span class="n">stop_event</span> <span class="o">=</span> <span class="n">Event</span><span class="p">()</span>
|
|||
|
|
<span class="n">lp</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">listener_process</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s1">'listener'</span><span class="p">,</span>
|
|||
|
|
<span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">q</span><span class="p">,</span> <span class="n">stop_event</span><span class="p">,</span> <span class="n">config_listener</span><span class="p">))</span>
|
|||
|
|
<span class="n">lp</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'Started listener'</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># We now hang around for the workers to finish their work.</span>
|
|||
|
|
<span class="k">for</span> <span class="n">wp</span> <span class="ow">in</span> <span class="n">workers</span><span class="p">:</span>
|
|||
|
|
<span class="n">wp</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
|
|||
|
|
<span class="c1"># Workers all done, listening can now stop.</span>
|
|||
|
|
<span class="c1"># Logging in the parent still works normally.</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'Telling listener to stop ...'</span><span class="p">)</span>
|
|||
|
|
<span class="n">stop_event</span><span class="o">.</span><span class="n">set</span><span class="p">()</span>
|
|||
|
|
<span class="n">lp</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'All done.'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">main</span><span class="p">()</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="inserting-a-bom-into-messages-sent-to-a-sysloghandler">
|
|||
|
|
<h2>Inserting a BOM into messages sent to a SysLogHandler<a class="headerlink" href="#inserting-a-bom-into-messages-sent-to-a-sysloghandler" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p><span class="target" id="index-0"></span><a class="rfc reference external" href="https://tools.ietf.org/html/rfc5424.html"><strong>RFC 5424</strong></a> requires that a
|
|||
|
|
Unicode message be sent to a syslog daemon as a set of bytes which have the
|
|||
|
|
following structure: an optional pure-ASCII component, followed by a UTF-8 Byte
|
|||
|
|
Order Mark (BOM), followed by Unicode encoded using UTF-8. (See the
|
|||
|
|
<span class="target" id="index-1"></span><a class="rfc reference external" href="https://tools.ietf.org/html/rfc5424.html#section-6"><strong>relevant section of the specification</strong></a>.)</p>
|
|||
|
|
<p>In Python 3.1, code was added to
|
|||
|
|
<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.SysLogHandler" title="logging.handlers.SysLogHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">SysLogHandler</span></code></a> to insert a BOM into the message, but
|
|||
|
|
unfortunately, it was implemented incorrectly, with the BOM appearing at the
|
|||
|
|
beginning of the message and hence not allowing any pure-ASCII component to
|
|||
|
|
appear before it.</p>
|
|||
|
|
<p>As this behaviour is broken, the incorrect BOM insertion code is being removed
|
|||
|
|
from Python 3.2.4 and later. However, it is not being replaced, and if you
|
|||
|
|
want to produce <span class="target" id="index-2"></span><a class="rfc reference external" href="https://tools.ietf.org/html/rfc5424.html"><strong>RFC 5424</strong></a>-compliant messages which include a BOM, an optional
|
|||
|
|
pure-ASCII sequence before it and arbitrary Unicode after it, encoded using
|
|||
|
|
UTF-8, then you need to do the following:</p>
|
|||
|
|
<ol class="arabic">
|
|||
|
|
<li><p>Attach a <a class="reference internal" href="../library/logging.html#logging.Formatter" title="logging.Formatter"><code class="xref py py-class docutils literal notranslate"><span class="pre">Formatter</span></code></a> instance to your
|
|||
|
|
<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.SysLogHandler" title="logging.handlers.SysLogHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">SysLogHandler</span></code></a> instance, with a format string
|
|||
|
|
such as:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="s1">'ASCII section</span><span class="se">\ufeff</span><span class="s1">Unicode section'</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>The Unicode code point U+FEFF, when encoded using UTF-8, will be
|
|||
|
|
encoded as a UTF-8 BOM – the byte-string <code class="docutils literal notranslate"><span class="pre">b'\xef\xbb\xbf'</span></code>.</p>
|
|||
|
|
</li>
|
|||
|
|
<li><p>Replace the ASCII section with whatever placeholders you like, but make sure
|
|||
|
|
that the data that appears in there after substitution is always ASCII (that
|
|||
|
|
way, it will remain unchanged after UTF-8 encoding).</p></li>
|
|||
|
|
<li><p>Replace the Unicode section with whatever placeholders you like; if the data
|
|||
|
|
which appears there after substitution contains characters outside the ASCII
|
|||
|
|
range, that’s fine – it will be encoded using UTF-8.</p></li>
|
|||
|
|
</ol>
|
|||
|
|
<p>The formatted message <em>will</em> be encoded using UTF-8 encoding by
|
|||
|
|
<code class="docutils literal notranslate"><span class="pre">SysLogHandler</span></code>. If you follow the above rules, you should be able to produce
|
|||
|
|
<span class="target" id="index-3"></span><a class="rfc reference external" href="https://tools.ietf.org/html/rfc5424.html"><strong>RFC 5424</strong></a>-compliant messages. If you don’t, logging may not complain, but your
|
|||
|
|
messages will not be RFC 5424-compliant, and your syslog daemon may complain.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="implementing-structured-logging">
|
|||
|
|
<h2>Implementing structured logging<a class="headerlink" href="#implementing-structured-logging" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Although most logging messages are intended for reading by humans, and thus not
|
|||
|
|
readily machine-parseable, there might be circumstances where you want to output
|
|||
|
|
messages in a structured format which <em>is</em> capable of being parsed by a program
|
|||
|
|
(without needing complex regular expressions to parse the log message). This is
|
|||
|
|
straightforward to achieve using the logging package. There are a number of
|
|||
|
|
ways in which this could be achieved, but the following is a simple approach
|
|||
|
|
which uses JSON to serialise the event in a machine-parseable manner:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">json</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">StructuredMessage</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">message</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">message</span> <span class="o">=</span> <span class="n">message</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">kwargs</span> <span class="o">=</span> <span class="n">kwargs</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="k">return</span> <span class="s1">'</span><span class="si">%s</span><span class="s1"> >>> </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">message</span><span class="p">,</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">kwargs</span><span class="p">))</span>
|
|||
|
|
|
|||
|
|
<span class="n">_</span> <span class="o">=</span> <span class="n">StructuredMessage</span> <span class="c1"># optional, to improve readability</span>
|
|||
|
|
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">basicConfig</span><span class="p">(</span><span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">,</span> <span class="nb">format</span><span class="o">=</span><span class="s1">'</span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="n">_</span><span class="p">(</span><span class="s1">'message 1'</span><span class="p">,</span> <span class="n">foo</span><span class="o">=</span><span class="s1">'bar'</span><span class="p">,</span> <span class="n">bar</span><span class="o">=</span><span class="s1">'baz'</span><span class="p">,</span> <span class="n">num</span><span class="o">=</span><span class="mi">123</span><span class="p">,</span> <span class="n">fnum</span><span class="o">=</span><span class="mf">123.456</span><span class="p">))</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>If the above script is run, it prints:</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>message 1 >>> {"fnum": 123.456, "num": 123, "bar": "baz", "foo": "bar"}
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Note that the order of items might be different according to the version of
|
|||
|
|
Python used.</p>
|
|||
|
|
<p>If you need more specialised processing, you can use a custom JSON encoder,
|
|||
|
|
as in the following complete example:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">__future__</span> <span class="k">import</span> <span class="n">unicode_literals</span>
|
|||
|
|
|
|||
|
|
<span class="kn">import</span> <span class="nn">json</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
|
|||
|
|
<span class="c1"># This next bit is to ensure the script runs unchanged on 2.x and 3.x</span>
|
|||
|
|
<span class="k">try</span><span class="p">:</span>
|
|||
|
|
<span class="n">unicode</span>
|
|||
|
|
<span class="k">except</span> <span class="ne">NameError</span><span class="p">:</span>
|
|||
|
|
<span class="n">unicode</span> <span class="o">=</span> <span class="nb">str</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">Encoder</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">JSONEncoder</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">default</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">o</span><span class="p">):</span>
|
|||
|
|
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">o</span><span class="p">,</span> <span class="nb">set</span><span class="p">):</span>
|
|||
|
|
<span class="k">return</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">o</span><span class="p">)</span>
|
|||
|
|
<span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">o</span><span class="p">,</span> <span class="n">unicode</span><span class="p">):</span>
|
|||
|
|
<span class="k">return</span> <span class="n">o</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">'unicode_escape'</span><span class="p">)</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s1">'ascii'</span><span class="p">)</span>
|
|||
|
|
<span class="k">return</span> <span class="nb">super</span><span class="p">(</span><span class="n">Encoder</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">default</span><span class="p">(</span><span class="n">o</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">StructuredMessage</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">message</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">message</span> <span class="o">=</span> <span class="n">message</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">kwargs</span> <span class="o">=</span> <span class="n">kwargs</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="n">s</span> <span class="o">=</span> <span class="n">Encoder</span><span class="p">()</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">kwargs</span><span class="p">)</span>
|
|||
|
|
<span class="k">return</span> <span class="s1">'</span><span class="si">%s</span><span class="s1"> >>> </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">message</span><span class="p">,</span> <span class="n">s</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="n">_</span> <span class="o">=</span> <span class="n">StructuredMessage</span> <span class="c1"># optional, to improve readability</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">basicConfig</span><span class="p">(</span><span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">,</span> <span class="nb">format</span><span class="o">=</span><span class="s1">'</span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="n">_</span><span class="p">(</span><span class="s1">'message 1'</span><span class="p">,</span> <span class="n">set_value</span><span class="o">=</span><span class="p">{</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">},</span> <span class="n">snowman</span><span class="o">=</span><span class="s1">'</span><span class="se">\u2603</span><span class="s1">'</span><span class="p">))</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">main</span><span class="p">()</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>When the above script is run, it prints:</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>message 1 >>> {"snowman": "\u2603", "set_value": [1, 2, 3]}
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Note that the order of items might be different according to the version of
|
|||
|
|
Python used.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="customizing-handlers-with-dictconfig">
|
|||
|
|
<span id="custom-handlers"></span><h2>Customizing handlers with <a class="reference internal" href="../library/logging.config.html#logging.config.dictConfig" title="logging.config.dictConfig"><code class="xref py py-func docutils literal notranslate"><span class="pre">dictConfig()</span></code></a><a class="headerlink" href="#customizing-handlers-with-dictconfig" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>There are times when you want to customize logging handlers in particular ways,
|
|||
|
|
and if you use <a class="reference internal" href="../library/logging.config.html#logging.config.dictConfig" title="logging.config.dictConfig"><code class="xref py py-func docutils literal notranslate"><span class="pre">dictConfig()</span></code></a> you may be able to do this without
|
|||
|
|
subclassing. As an example, consider that you may want to set the ownership of a
|
|||
|
|
log file. On POSIX, this is easily done using <a class="reference internal" href="../library/shutil.html#shutil.chown" title="shutil.chown"><code class="xref py py-func docutils literal notranslate"><span class="pre">shutil.chown()</span></code></a>, but the file
|
|||
|
|
handlers in the stdlib don’t offer built-in support. You can customize handler
|
|||
|
|
creation using a plain function such as:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">owned_file_handler</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s1">'a'</span><span class="p">,</span> <span class="n">encoding</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">owner</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
|||
|
|
<span class="k">if</span> <span class="n">owner</span><span class="p">:</span>
|
|||
|
|
<span class="k">if</span> <span class="ow">not</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="n">filename</span><span class="p">):</span>
|
|||
|
|
<span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="s1">'a'</span><span class="p">)</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
|
|||
|
|
<span class="n">shutil</span><span class="o">.</span><span class="n">chown</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="o">*</span><span class="n">owner</span><span class="p">)</span>
|
|||
|
|
<span class="k">return</span> <span class="n">logging</span><span class="o">.</span><span class="n">FileHandler</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">mode</span><span class="p">,</span> <span class="n">encoding</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>You can then specify, in a logging configuration passed to <a class="reference internal" href="../library/logging.config.html#logging.config.dictConfig" title="logging.config.dictConfig"><code class="xref py py-func docutils literal notranslate"><span class="pre">dictConfig()</span></code></a>,
|
|||
|
|
that a logging handler be created by calling this function:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">LOGGING</span> <span class="o">=</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'version'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'disable_existing_loggers'</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatters'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'default'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'format'</span><span class="p">:</span> <span class="s1">'</span><span class="si">%(asctime)s</span><span class="s1"> </span><span class="si">%(levelname)s</span><span class="s1"> </span><span class="si">%(name)s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'file'</span><span class="p">:{</span>
|
|||
|
|
<span class="c1"># The values below are popped from this dictionary and</span>
|
|||
|
|
<span class="c1"># used to create the handler, set the handler's level and</span>
|
|||
|
|
<span class="c1"># its formatter.</span>
|
|||
|
|
<span class="s1">'()'</span><span class="p">:</span> <span class="n">owned_file_handler</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span><span class="s1">'DEBUG'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatter'</span><span class="p">:</span> <span class="s1">'default'</span><span class="p">,</span>
|
|||
|
|
<span class="c1"># The values below are passed to the handler creator callable</span>
|
|||
|
|
<span class="c1"># as keyword arguments.</span>
|
|||
|
|
<span class="s1">'owner'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'pulse'</span><span class="p">,</span> <span class="s1">'pulse'</span><span class="p">],</span>
|
|||
|
|
<span class="s1">'filename'</span><span class="p">:</span> <span class="s1">'chowntest.log'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'mode'</span><span class="p">:</span> <span class="s1">'w'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'encoding'</span><span class="p">:</span> <span class="s1">'utf-8'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'root'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'file'</span><span class="p">],</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'DEBUG'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>In this example I am setting the ownership using the <code class="docutils literal notranslate"><span class="pre">pulse</span></code> user and group,
|
|||
|
|
just for the purposes of illustration. Putting it together into a working
|
|||
|
|
script, <code class="docutils literal notranslate"><span class="pre">chowntest.py</span></code>:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span><span class="o">,</span> <span class="nn">logging.config</span><span class="o">,</span> <span class="nn">os</span><span class="o">,</span> <span class="nn">shutil</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">owned_file_handler</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s1">'a'</span><span class="p">,</span> <span class="n">encoding</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">owner</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
|||
|
|
<span class="k">if</span> <span class="n">owner</span><span class="p">:</span>
|
|||
|
|
<span class="k">if</span> <span class="ow">not</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="n">filename</span><span class="p">):</span>
|
|||
|
|
<span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="s1">'a'</span><span class="p">)</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
|
|||
|
|
<span class="n">shutil</span><span class="o">.</span><span class="n">chown</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="o">*</span><span class="n">owner</span><span class="p">)</span>
|
|||
|
|
<span class="k">return</span> <span class="n">logging</span><span class="o">.</span><span class="n">FileHandler</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">mode</span><span class="p">,</span> <span class="n">encoding</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="n">LOGGING</span> <span class="o">=</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'version'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'disable_existing_loggers'</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatters'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'default'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'format'</span><span class="p">:</span> <span class="s1">'</span><span class="si">%(asctime)s</span><span class="s1"> </span><span class="si">%(levelname)s</span><span class="s1"> </span><span class="si">%(name)s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'file'</span><span class="p">:{</span>
|
|||
|
|
<span class="c1"># The values below are popped from this dictionary and</span>
|
|||
|
|
<span class="c1"># used to create the handler, set the handler's level and</span>
|
|||
|
|
<span class="c1"># its formatter.</span>
|
|||
|
|
<span class="s1">'()'</span><span class="p">:</span> <span class="n">owned_file_handler</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span><span class="s1">'DEBUG'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatter'</span><span class="p">:</span> <span class="s1">'default'</span><span class="p">,</span>
|
|||
|
|
<span class="c1"># The values below are passed to the handler creator callable</span>
|
|||
|
|
<span class="c1"># as keyword arguments.</span>
|
|||
|
|
<span class="s1">'owner'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'pulse'</span><span class="p">,</span> <span class="s1">'pulse'</span><span class="p">],</span>
|
|||
|
|
<span class="s1">'filename'</span><span class="p">:</span> <span class="s1">'chowntest.log'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'mode'</span><span class="p">:</span> <span class="s1">'w'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'encoding'</span><span class="p">:</span> <span class="s1">'utf-8'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'root'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'file'</span><span class="p">],</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'DEBUG'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">dictConfig</span><span class="p">(</span><span class="n">LOGGING</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'mylogger'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'A debug message'</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>To run this, you will probably need to run as <code class="docutils literal notranslate"><span class="pre">root</span></code>:</p>
|
|||
|
|
<div class="highlight-shell-session notranslate"><div class="highlight"><pre><span></span><span class="gp">$</span> sudo python3.3 chowntest.py
|
|||
|
|
<span class="gp">$</span> cat chowntest.log
|
|||
|
|
<span class="go">2013-11-05 09:34:51,128 DEBUG mylogger A debug message</span>
|
|||
|
|
<span class="gp">$</span> ls -l chowntest.log
|
|||
|
|
<span class="go">-rw-r--r-- 1 pulse pulse 55 2013-11-05 09:34 chowntest.log</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Note that this example uses Python 3.3 because that’s where <a class="reference internal" href="../library/shutil.html#shutil.chown" title="shutil.chown"><code class="xref py py-func docutils literal notranslate"><span class="pre">shutil.chown()</span></code></a>
|
|||
|
|
makes an appearance. This approach should work with any Python version that
|
|||
|
|
supports <a class="reference internal" href="../library/logging.config.html#logging.config.dictConfig" title="logging.config.dictConfig"><code class="xref py py-func docutils literal notranslate"><span class="pre">dictConfig()</span></code></a> - namely, Python 2.7, 3.2 or later. With pre-3.3
|
|||
|
|
versions, you would need to implement the actual ownership change using e.g.
|
|||
|
|
<a class="reference internal" href="../library/os.html#os.chown" title="os.chown"><code class="xref py py-func docutils literal notranslate"><span class="pre">os.chown()</span></code></a>.</p>
|
|||
|
|
<p>In practice, the handler-creating function may be in a utility module somewhere
|
|||
|
|
in your project. Instead of the line in the configuration:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="s1">'()'</span><span class="p">:</span> <span class="n">owned_file_handler</span><span class="p">,</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>you could use e.g.:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="s1">'()'</span><span class="p">:</span> <span class="s1">'ext://project.util.owned_file_handler'</span><span class="p">,</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>where <code class="docutils literal notranslate"><span class="pre">project.util</span></code> can be replaced with the actual name of the package
|
|||
|
|
where the function resides. In the above working script, using
|
|||
|
|
<code class="docutils literal notranslate"><span class="pre">'ext://__main__.owned_file_handler'</span></code> should work. Here, the actual callable
|
|||
|
|
is resolved by <a class="reference internal" href="../library/logging.config.html#logging.config.dictConfig" title="logging.config.dictConfig"><code class="xref py py-func docutils literal notranslate"><span class="pre">dictConfig()</span></code></a> from the <code class="docutils literal notranslate"><span class="pre">ext://</span></code> specification.</p>
|
|||
|
|
<p>This example hopefully also points the way to how you could implement other
|
|||
|
|
types of file change - e.g. setting specific POSIX permission bits - in the
|
|||
|
|
same way, using <a class="reference internal" href="../library/os.html#os.chmod" title="os.chmod"><code class="xref py py-func docutils literal notranslate"><span class="pre">os.chmod()</span></code></a>.</p>
|
|||
|
|
<p>Of course, the approach could also be extended to types of handler other than a
|
|||
|
|
<a class="reference internal" href="../library/logging.handlers.html#logging.FileHandler" title="logging.FileHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">FileHandler</span></code></a> - for example, one of the rotating file handlers,
|
|||
|
|
or a different type of handler altogether.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="using-particular-formatting-styles-throughout-your-application">
|
|||
|
|
<span id="formatting-styles"></span><h2>Using particular formatting styles throughout your application<a class="headerlink" href="#using-particular-formatting-styles-throughout-your-application" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>In Python 3.2, the <a class="reference internal" href="../library/logging.html#logging.Formatter" title="logging.Formatter"><code class="xref py py-class docutils literal notranslate"><span class="pre">Formatter</span></code></a> gained a <code class="docutils literal notranslate"><span class="pre">style</span></code> keyword
|
|||
|
|
parameter which, while defaulting to <code class="docutils literal notranslate"><span class="pre">%</span></code> for backward compatibility, allowed
|
|||
|
|
the specification of <code class="docutils literal notranslate"><span class="pre">{</span></code> or <code class="docutils literal notranslate"><span class="pre">$</span></code> to support the formatting approaches
|
|||
|
|
supported by <a class="reference internal" href="../library/stdtypes.html#str.format" title="str.format"><code class="xref py py-meth docutils literal notranslate"><span class="pre">str.format()</span></code></a> and <a class="reference internal" href="../library/string.html#string.Template" title="string.Template"><code class="xref py py-class docutils literal notranslate"><span class="pre">string.Template</span></code></a>. Note that this
|
|||
|
|
governs the formatting of logging messages for final output to logs, and is
|
|||
|
|
completely orthogonal to how an individual logging message is constructed.</p>
|
|||
|
|
<p>Logging calls (<a class="reference internal" href="../library/logging.html#logging.Logger.debug" title="logging.Logger.debug"><code class="xref py py-meth docutils literal notranslate"><span class="pre">debug()</span></code></a>, <a class="reference internal" href="../library/logging.html#logging.Logger.info" title="logging.Logger.info"><code class="xref py py-meth docutils literal notranslate"><span class="pre">info()</span></code></a> etc.) only take
|
|||
|
|
positional parameters for the actual logging message itself, with keyword
|
|||
|
|
parameters used only for determining options for how to handle the logging call
|
|||
|
|
(e.g. the <code class="docutils literal notranslate"><span class="pre">exc_info</span></code> keyword parameter to indicate that traceback information
|
|||
|
|
should be logged, or the <code class="docutils literal notranslate"><span class="pre">extra</span></code> keyword parameter to indicate additional
|
|||
|
|
contextual information to be added to the log). So you cannot directly make
|
|||
|
|
logging calls using <a class="reference internal" href="../library/stdtypes.html#str.format" title="str.format"><code class="xref py py-meth docutils literal notranslate"><span class="pre">str.format()</span></code></a> or <a class="reference internal" href="../library/string.html#string.Template" title="string.Template"><code class="xref py py-class docutils literal notranslate"><span class="pre">string.Template</span></code></a> syntax,
|
|||
|
|
because internally the logging package uses %-formatting to merge the format
|
|||
|
|
string and the variable arguments. There would no changing this while preserving
|
|||
|
|
backward compatibility, since all logging calls which are out there in existing
|
|||
|
|
code will be using %-format strings.</p>
|
|||
|
|
<p>There have been suggestions to associate format styles with specific loggers,
|
|||
|
|
but that approach also runs into backward compatibility problems because any
|
|||
|
|
existing code could be using a given logger name and using %-formatting.</p>
|
|||
|
|
<p>For logging to work interoperably between any third-party libraries and your
|
|||
|
|
code, decisions about formatting need to be made at the level of the
|
|||
|
|
individual logging call. This opens up a couple of ways in which alternative
|
|||
|
|
formatting styles can be accommodated.</p>
|
|||
|
|
<div class="section" id="using-logrecord-factories">
|
|||
|
|
<h3>Using LogRecord factories<a class="headerlink" href="#using-logrecord-factories" title="Permalink to this headline">¶</a></h3>
|
|||
|
|
<p>In Python 3.2, along with the <a class="reference internal" href="../library/logging.html#logging.Formatter" title="logging.Formatter"><code class="xref py py-class docutils literal notranslate"><span class="pre">Formatter</span></code></a> changes mentioned
|
|||
|
|
above, the logging package gained the ability to allow users to set their own
|
|||
|
|
<a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal notranslate"><span class="pre">LogRecord</span></code></a> subclasses, using the <a class="reference internal" href="../library/logging.html#logging.setLogRecordFactory" title="logging.setLogRecordFactory"><code class="xref py py-func docutils literal notranslate"><span class="pre">setLogRecordFactory()</span></code></a> function.
|
|||
|
|
You can use this to set your own subclass of <a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal notranslate"><span class="pre">LogRecord</span></code></a>, which does the
|
|||
|
|
Right Thing by overriding the <a class="reference internal" href="../library/logging.html#logging.LogRecord.getMessage" title="logging.LogRecord.getMessage"><code class="xref py py-meth docutils literal notranslate"><span class="pre">getMessage()</span></code></a> method. The base
|
|||
|
|
class implementation of this method is where the <code class="docutils literal notranslate"><span class="pre">msg</span> <span class="pre">%</span> <span class="pre">args</span></code> formatting
|
|||
|
|
happens, and where you can substitute your alternate formatting; however, you
|
|||
|
|
should be careful to support all formatting styles and allow %-formatting as
|
|||
|
|
the default, to ensure interoperability with other code. Care should also be
|
|||
|
|
taken to call <code class="docutils literal notranslate"><span class="pre">str(self.msg)</span></code>, just as the base implementation does.</p>
|
|||
|
|
<p>Refer to the reference documentation on <a class="reference internal" href="../library/logging.html#logging.setLogRecordFactory" title="logging.setLogRecordFactory"><code class="xref py py-func docutils literal notranslate"><span class="pre">setLogRecordFactory()</span></code></a> and
|
|||
|
|
<a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal notranslate"><span class="pre">LogRecord</span></code></a> for more information.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="using-custom-message-objects">
|
|||
|
|
<h3>Using custom message objects<a class="headerlink" href="#using-custom-message-objects" title="Permalink to this headline">¶</a></h3>
|
|||
|
|
<p>There is another, perhaps simpler way that you can use {}- and $- formatting to
|
|||
|
|
construct your individual log messages. You may recall (from
|
|||
|
|
<a class="reference internal" href="logging.html#arbitrary-object-messages"><span class="std std-ref">Using arbitrary objects as messages</span></a>) that when logging you can use an arbitrary
|
|||
|
|
object as a message format string, and that the logging package will call
|
|||
|
|
<a class="reference internal" href="../library/stdtypes.html#str" title="str"><code class="xref py py-func docutils literal notranslate"><span class="pre">str()</span></code></a> on that object to get the actual format string. Consider the
|
|||
|
|
following two classes:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">BraceMessage</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">fmt</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">fmt</span> <span class="o">=</span> <span class="n">fmt</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">args</span> <span class="o">=</span> <span class="n">args</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">kwargs</span> <span class="o">=</span> <span class="n">kwargs</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">fmt</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="o">*</span><span class="bp">self</span><span class="o">.</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="bp">self</span><span class="o">.</span><span class="n">kwargs</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">DollarMessage</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">fmt</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">fmt</span> <span class="o">=</span> <span class="n">fmt</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">kwargs</span> <span class="o">=</span> <span class="n">kwargs</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="kn">from</span> <span class="nn">string</span> <span class="k">import</span> <span class="n">Template</span>
|
|||
|
|
<span class="k">return</span> <span class="n">Template</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">fmt</span><span class="p">)</span><span class="o">.</span><span class="n">substitute</span><span class="p">(</span><span class="o">**</span><span class="bp">self</span><span class="o">.</span><span class="n">kwargs</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Either of these can be used in place of a format string, to allow {}- or
|
|||
|
|
$-formatting to be used to build the actual “message” part which appears in the
|
|||
|
|
formatted log output in place of “%(message)s” or “{message}” or “$message”.
|
|||
|
|
If you find it a little unwieldy to use the class names whenever you want to log
|
|||
|
|
something, you can make it more palatable if you use an alias such as <code class="docutils literal notranslate"><span class="pre">M</span></code> or
|
|||
|
|
<code class="docutils literal notranslate"><span class="pre">_</span></code> for the message (or perhaps <code class="docutils literal notranslate"><span class="pre">__</span></code>, if you are using <code class="docutils literal notranslate"><span class="pre">_</span></code> for
|
|||
|
|
localization).</p>
|
|||
|
|
<p>Examples of this approach are given below. Firstly, formatting with
|
|||
|
|
<a class="reference internal" href="../library/stdtypes.html#str.format" title="str.format"><code class="xref py py-meth docutils literal notranslate"><span class="pre">str.format()</span></code></a>:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">__</span> <span class="o">=</span> <span class="n">BraceMessage</span>
|
|||
|
|
<span class="gp">>>> </span><span class="nb">print</span><span class="p">(</span><span class="n">__</span><span class="p">(</span><span class="s1">'Message with </span><span class="si">{0}</span><span class="s1"> </span><span class="si">{1}</span><span class="s1">'</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="s1">'placeholders'</span><span class="p">))</span>
|
|||
|
|
<span class="go">Message with 2 placeholders</span>
|
|||
|
|
<span class="gp">>>> </span><span class="k">class</span> <span class="nc">Point</span><span class="p">:</span> <span class="k">pass</span>
|
|||
|
|
<span class="gp">...</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">p</span> <span class="o">=</span> <span class="n">Point</span><span class="p">()</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">p</span><span class="o">.</span><span class="n">x</span> <span class="o">=</span> <span class="mf">0.5</span>
|
|||
|
|
<span class="gp">>>> </span><span class="n">p</span><span class="o">.</span><span class="n">y</span> <span class="o">=</span> <span class="mf">0.5</span>
|
|||
|
|
<span class="gp">>>> </span><span class="nb">print</span><span class="p">(</span><span class="n">__</span><span class="p">(</span><span class="s1">'Message with coordinates: (</span><span class="si">{point.x:.2f}</span><span class="s1">, </span><span class="si">{point.y:.2f}</span><span class="s1">)'</span><span class="p">,</span> <span class="n">point</span><span class="o">=</span><span class="n">p</span><span class="p">))</span>
|
|||
|
|
<span class="go">Message with coordinates: (0.50, 0.50)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Secondly, formatting with <a class="reference internal" href="../library/string.html#string.Template" title="string.Template"><code class="xref py py-class docutils literal notranslate"><span class="pre">string.Template</span></code></a>:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">__</span> <span class="o">=</span> <span class="n">DollarMessage</span>
|
|||
|
|
<span class="gp">>>> </span><span class="nb">print</span><span class="p">(</span><span class="n">__</span><span class="p">(</span><span class="s1">'Message with $num $what'</span><span class="p">,</span> <span class="n">num</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">what</span><span class="o">=</span><span class="s1">'placeholders'</span><span class="p">))</span>
|
|||
|
|
<span class="go">Message with 2 placeholders</span>
|
|||
|
|
<span class="go">>>></span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>One thing to note is that you pay no significant performance penalty with this
|
|||
|
|
approach: the actual formatting happens not when you make the logging call, but
|
|||
|
|
when (and if) the logged message is actually about to be output to a log by a
|
|||
|
|
handler. So the only slightly unusual thing which might trip you up is that the
|
|||
|
|
parentheses go around the format string and the arguments, not just the format
|
|||
|
|
string. That’s because the __ notation is just syntax sugar for a constructor
|
|||
|
|
call to one of the <code class="docutils literal notranslate"><span class="pre">XXXMessage</span></code> classes shown above.</p>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="configuring-filters-with-dictconfig">
|
|||
|
|
<span id="filters-dictconfig"></span><h2>Configuring filters with <a class="reference internal" href="../library/logging.config.html#logging.config.dictConfig" title="logging.config.dictConfig"><code class="xref py py-func docutils literal notranslate"><span class="pre">dictConfig()</span></code></a><a class="headerlink" href="#configuring-filters-with-dictconfig" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>You <em>can</em> configure filters using <a class="reference internal" href="../library/logging.config.html#logging.config.dictConfig" title="logging.config.dictConfig"><code class="xref py py-func docutils literal notranslate"><span class="pre">dictConfig()</span></code></a>, though it
|
|||
|
|
might not be obvious at first glance how to do it (hence this recipe). Since
|
|||
|
|
<a class="reference internal" href="../library/logging.html#logging.Filter" title="logging.Filter"><code class="xref py py-class docutils literal notranslate"><span class="pre">Filter</span></code></a> is the only filter class included in the standard
|
|||
|
|
library, and it is unlikely to cater to many requirements (it’s only there as a
|
|||
|
|
base class), you will typically need to define your own <a class="reference internal" href="../library/logging.html#logging.Filter" title="logging.Filter"><code class="xref py py-class docutils literal notranslate"><span class="pre">Filter</span></code></a>
|
|||
|
|
subclass with an overridden <a class="reference internal" href="../library/logging.html#logging.Filter.filter" title="logging.Filter.filter"><code class="xref py py-meth docutils literal notranslate"><span class="pre">filter()</span></code></a> method. To do this,
|
|||
|
|
specify the <code class="docutils literal notranslate"><span class="pre">()</span></code> key in the configuration dictionary for the filter,
|
|||
|
|
specifying a callable which will be used to create the filter (a class is the
|
|||
|
|
most obvious, but you can provide any callable which returns a
|
|||
|
|
<a class="reference internal" href="../library/logging.html#logging.Filter" title="logging.Filter"><code class="xref py py-class docutils literal notranslate"><span class="pre">Filter</span></code></a> instance). Here is a complete example:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging.config</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">sys</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">MyFilter</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">Filter</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">param</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">param</span> <span class="o">=</span> <span class="n">param</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">filter</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">record</span><span class="p">):</span>
|
|||
|
|
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">param</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
|||
|
|
<span class="n">allow</span> <span class="o">=</span> <span class="kc">True</span>
|
|||
|
|
<span class="k">else</span><span class="p">:</span>
|
|||
|
|
<span class="n">allow</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">param</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">record</span><span class="o">.</span><span class="n">msg</span>
|
|||
|
|
<span class="k">if</span> <span class="n">allow</span><span class="p">:</span>
|
|||
|
|
<span class="n">record</span><span class="o">.</span><span class="n">msg</span> <span class="o">=</span> <span class="s1">'changed: '</span> <span class="o">+</span> <span class="n">record</span><span class="o">.</span><span class="n">msg</span>
|
|||
|
|
<span class="k">return</span> <span class="n">allow</span>
|
|||
|
|
|
|||
|
|
<span class="n">LOGGING</span> <span class="o">=</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'version'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'filters'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'myfilter'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'()'</span><span class="p">:</span> <span class="n">MyFilter</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'param'</span><span class="p">:</span> <span class="s1">'noshow'</span><span class="p">,</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'console'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.StreamHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'filters'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'myfilter'</span><span class="p">]</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'root'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'level'</span><span class="p">:</span> <span class="s1">'DEBUG'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'console'</span><span class="p">]</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">dictConfig</span><span class="p">(</span><span class="n">LOGGING</span><span class="p">)</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'hello'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'hello - noshow'</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>This example shows how you can pass configuration data to the callable which
|
|||
|
|
constructs the instance, in the form of keyword parameters. When run, the above
|
|||
|
|
script will print:</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>changed: hello
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>which shows that the filter is working as configured.</p>
|
|||
|
|
<p>A couple of extra points to note:</p>
|
|||
|
|
<ul class="simple">
|
|||
|
|
<li><p>If you can’t refer to the callable directly in the configuration (e.g. if it
|
|||
|
|
lives in a different module, and you can’t import it directly where the
|
|||
|
|
configuration dictionary is), you can use the form <code class="docutils literal notranslate"><span class="pre">ext://...</span></code> as described
|
|||
|
|
in <a class="reference internal" href="../library/logging.config.html#logging-config-dict-externalobj"><span class="std std-ref">Access to external objects</span></a>. For example, you could have used
|
|||
|
|
the text <code class="docutils literal notranslate"><span class="pre">'ext://__main__.MyFilter'</span></code> instead of <code class="docutils literal notranslate"><span class="pre">MyFilter</span></code> in the above
|
|||
|
|
example.</p></li>
|
|||
|
|
<li><p>As well as for filters, this technique can also be used to configure custom
|
|||
|
|
handlers and formatters. See <a class="reference internal" href="../library/logging.config.html#logging-config-dict-userdef"><span class="std std-ref">User-defined objects</span></a> for more
|
|||
|
|
information on how logging supports using user-defined objects in its
|
|||
|
|
configuration, and see the other cookbook recipe <a class="reference internal" href="#custom-handlers"><span class="std std-ref">Customizing handlers with dictConfig()</span></a> above.</p></li>
|
|||
|
|
</ul>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="customized-exception-formatting">
|
|||
|
|
<span id="custom-format-exception"></span><h2>Customized exception formatting<a class="headerlink" href="#customized-exception-formatting" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>There might be times when you want to do customized exception formatting - for
|
|||
|
|
argument’s sake, let’s say you want exactly one line per logged event, even
|
|||
|
|
when exception information is present. You can do this with a custom formatter
|
|||
|
|
class, as shown in the following example:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">OneLineExceptionFormatter</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">formatException</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">exc_info</span><span class="p">):</span>
|
|||
|
|
<span class="sd">"""</span>
|
|||
|
|
<span class="sd"> Format an exception so that it prints on a single line.</span>
|
|||
|
|
<span class="sd"> """</span>
|
|||
|
|
<span class="n">result</span> <span class="o">=</span> <span class="nb">super</span><span class="p">(</span><span class="n">OneLineExceptionFormatter</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">formatException</span><span class="p">(</span><span class="n">exc_info</span><span class="p">)</span>
|
|||
|
|
<span class="k">return</span> <span class="nb">repr</span><span class="p">(</span><span class="n">result</span><span class="p">)</span> <span class="c1"># or format into one line however you want to</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">format</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">record</span><span class="p">):</span>
|
|||
|
|
<span class="n">s</span> <span class="o">=</span> <span class="nb">super</span><span class="p">(</span><span class="n">OneLineExceptionFormatter</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">record</span><span class="p">)</span>
|
|||
|
|
<span class="k">if</span> <span class="n">record</span><span class="o">.</span><span class="n">exc_text</span><span class="p">:</span>
|
|||
|
|
<span class="n">s</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">'</span><span class="se">\n</span><span class="s1">'</span><span class="p">,</span> <span class="s1">''</span><span class="p">)</span> <span class="o">+</span> <span class="s1">'|'</span>
|
|||
|
|
<span class="k">return</span> <span class="n">s</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">configure_logging</span><span class="p">():</span>
|
|||
|
|
<span class="n">fh</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">FileHandler</span><span class="p">(</span><span class="s1">'output.txt'</span><span class="p">,</span> <span class="s1">'w'</span><span class="p">)</span>
|
|||
|
|
<span class="n">f</span> <span class="o">=</span> <span class="n">OneLineExceptionFormatter</span><span class="p">(</span><span class="s1">'</span><span class="si">%(asctime)s</span><span class="s1">|</span><span class="si">%(levelname)s</span><span class="s1">|</span><span class="si">%(message)s</span><span class="s1">|'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'</span><span class="si">%d</span><span class="s1">/%m/%Y %H:%M:%S'</span><span class="p">)</span>
|
|||
|
|
<span class="n">fh</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
|
|||
|
|
<span class="n">root</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">()</span>
|
|||
|
|
<span class="n">root</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
<span class="n">root</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">fh</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
|
|||
|
|
<span class="n">configure_logging</span><span class="p">()</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'Sample message'</span><span class="p">)</span>
|
|||
|
|
<span class="k">try</span><span class="p">:</span>
|
|||
|
|
<span class="n">x</span> <span class="o">=</span> <span class="mi">1</span> <span class="o">/</span> <span class="mi">0</span>
|
|||
|
|
<span class="k">except</span> <span class="ne">ZeroDivisionError</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">exception</span><span class="p">(</span><span class="s1">'ZeroDivisionError: </span><span class="si">%s</span><span class="s1">'</span><span class="p">,</span> <span class="n">e</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">main</span><span class="p">()</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>When run, this produces a file with exactly two lines:</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>28/01/2015 07:21:23|INFO|Sample message|
|
|||
|
|
28/01/2015 07:21:23|ERROR|ZeroDivisionError: integer division or modulo by zero|'Traceback (most recent call last):\n File "logtest7.py", line 30, in main\n x = 1 / 0\nZeroDivisionError: integer division or modulo by zero'|
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>While the above treatment is simplistic, it points the way to how exception
|
|||
|
|
information can be formatted to your liking. The <a class="reference internal" href="../library/traceback.html#module-traceback" title="traceback: Print or retrieve a stack traceback."><code class="xref py py-mod docutils literal notranslate"><span class="pre">traceback</span></code></a> module may be
|
|||
|
|
helpful for more specialized needs.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="speaking-logging-messages">
|
|||
|
|
<span id="spoken-messages"></span><h2>Speaking logging messages<a class="headerlink" href="#speaking-logging-messages" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>There might be situations when it is desirable to have logging messages rendered
|
|||
|
|
in an audible rather than a visible format. This is easy to do if you have
|
|||
|
|
text-to-speech (TTS) functionality available in your system, even if it doesn’t have
|
|||
|
|
a Python binding. Most TTS systems have a command line program you can run, and
|
|||
|
|
this can be invoked from a handler using <a class="reference internal" href="../library/subprocess.html#module-subprocess" title="subprocess: Subprocess management."><code class="xref py py-mod docutils literal notranslate"><span class="pre">subprocess</span></code></a>. It’s assumed here
|
|||
|
|
that TTS command line programs won’t expect to interact with users or take a
|
|||
|
|
long time to complete, and that the frequency of logged messages will be not so
|
|||
|
|
high as to swamp the user with messages, and that it’s acceptable to have the
|
|||
|
|
messages spoken one at a time rather than concurrently, The example implementation
|
|||
|
|
below waits for one message to be spoken before the next is processed, and this
|
|||
|
|
might cause other handlers to be kept waiting. Here is a short example showing
|
|||
|
|
the approach, which assumes that the <code class="docutils literal notranslate"><span class="pre">espeak</span></code> TTS package is available:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">subprocess</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">sys</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">TTSHandler</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">Handler</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">emit</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">record</span><span class="p">):</span>
|
|||
|
|
<span class="n">msg</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">record</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># Speak slowly in a female English voice</span>
|
|||
|
|
<span class="n">cmd</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'espeak'</span><span class="p">,</span> <span class="s1">'-s150'</span><span class="p">,</span> <span class="s1">'-ven+f3'</span><span class="p">,</span> <span class="n">msg</span><span class="p">]</span>
|
|||
|
|
<span class="n">p</span> <span class="o">=</span> <span class="n">subprocess</span><span class="o">.</span><span class="n">Popen</span><span class="p">(</span><span class="n">cmd</span><span class="p">,</span> <span class="n">stdout</span><span class="o">=</span><span class="n">subprocess</span><span class="o">.</span><span class="n">PIPE</span><span class="p">,</span>
|
|||
|
|
<span class="n">stderr</span><span class="o">=</span><span class="n">subprocess</span><span class="o">.</span><span class="n">STDOUT</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># wait for the program to finish</span>
|
|||
|
|
<span class="n">p</span><span class="o">.</span><span class="n">communicate</span><span class="p">()</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">configure_logging</span><span class="p">():</span>
|
|||
|
|
<span class="n">h</span> <span class="o">=</span> <span class="n">TTSHandler</span><span class="p">()</span>
|
|||
|
|
<span class="n">root</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">()</span>
|
|||
|
|
<span class="n">root</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">h</span><span class="p">)</span>
|
|||
|
|
<span class="c1"># the default formatter just returns the message</span>
|
|||
|
|
<span class="n">root</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'Hello'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'Goodbye'</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">configure_logging</span><span class="p">()</span>
|
|||
|
|
<span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="n">main</span><span class="p">())</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>When run, this script should say “Hello” and then “Goodbye” in a female voice.</p>
|
|||
|
|
<p>The above approach can, of course, be adapted to other TTS systems and even
|
|||
|
|
other systems altogether which can process messages via external programs run
|
|||
|
|
from a command line.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="buffering-logging-messages-and-outputting-them-conditionally">
|
|||
|
|
<span id="buffered-logging"></span><h2>Buffering logging messages and outputting them conditionally<a class="headerlink" href="#buffering-logging-messages-and-outputting-them-conditionally" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>There might be situations where you want to log messages in a temporary area
|
|||
|
|
and only output them if a certain condition occurs. For example, you may want to
|
|||
|
|
start logging debug events in a function, and if the function completes without
|
|||
|
|
errors, you don’t want to clutter the log with the collected debug information,
|
|||
|
|
but if there is an error, you want all the debug information to be output as well
|
|||
|
|
as the error.</p>
|
|||
|
|
<p>Here is an example which shows how you could do this using a decorator for your
|
|||
|
|
functions where you want logging to behave this way. It makes use of the
|
|||
|
|
<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.MemoryHandler" title="logging.handlers.MemoryHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">logging.handlers.MemoryHandler</span></code></a>, which allows buffering of logged events
|
|||
|
|
until some condition occurs, at which point the buffered events are <code class="docutils literal notranslate"><span class="pre">flushed</span></code>
|
|||
|
|
- passed to another handler (the <code class="docutils literal notranslate"><span class="pre">target</span></code> handler) for processing. By default,
|
|||
|
|
the <code class="docutils literal notranslate"><span class="pre">MemoryHandler</span></code> flushed when its buffer gets filled up or an event whose
|
|||
|
|
level is greater than or equal to a specified threshold is seen. You can use this
|
|||
|
|
recipe with a more specialised subclass of <code class="docutils literal notranslate"><span class="pre">MemoryHandler</span></code> if you want custom
|
|||
|
|
flushing behavior.</p>
|
|||
|
|
<p>The example script has a simple function, <code class="docutils literal notranslate"><span class="pre">foo</span></code>, which just cycles through
|
|||
|
|
all the logging levels, writing to <code class="docutils literal notranslate"><span class="pre">sys.stderr</span></code> to say what level it’s about
|
|||
|
|
to log at, and then actually logging a message at that level. You can pass a
|
|||
|
|
parameter to <code class="docutils literal notranslate"><span class="pre">foo</span></code> which, if true, will log at ERROR and CRITICAL levels -
|
|||
|
|
otherwise, it only logs at DEBUG, INFO and WARNING levels.</p>
|
|||
|
|
<p>The script just arranges to decorate <code class="docutils literal notranslate"><span class="pre">foo</span></code> with a decorator which will do the
|
|||
|
|
conditional logging that’s required. The decorator takes a logger as a parameter
|
|||
|
|
and attaches a memory handler for the duration of the call to the decorated
|
|||
|
|
function. The decorator can be additionally parameterised using a target handler,
|
|||
|
|
a level at which flushing should occur, and a capacity for the buffer (number of
|
|||
|
|
records buffered). These default to a <a class="reference internal" href="../library/logging.handlers.html#logging.StreamHandler" title="logging.StreamHandler"><code class="xref py py-class docutils literal notranslate"><span class="pre">StreamHandler</span></code></a> which
|
|||
|
|
writes to <code class="docutils literal notranslate"><span class="pre">sys.stderr</span></code>, <code class="docutils literal notranslate"><span class="pre">logging.ERROR</span></code> and <code class="docutils literal notranslate"><span class="pre">100</span></code> respectively.</p>
|
|||
|
|
<p>Here’s the script:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">from</span> <span class="nn">logging.handlers</span> <span class="k">import</span> <span class="n">MemoryHandler</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">sys</span>
|
|||
|
|
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">NullHandler</span><span class="p">())</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">log_if_errors</span><span class="p">(</span><span class="n">logger</span><span class="p">,</span> <span class="n">target_handler</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">flush_level</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">capacity</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
|||
|
|
<span class="k">if</span> <span class="n">target_handler</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
|||
|
|
<span class="n">target_handler</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">StreamHandler</span><span class="p">()</span>
|
|||
|
|
<span class="k">if</span> <span class="n">flush_level</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
|||
|
|
<span class="n">flush_level</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">ERROR</span>
|
|||
|
|
<span class="k">if</span> <span class="n">capacity</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
|||
|
|
<span class="n">capacity</span> <span class="o">=</span> <span class="mi">100</span>
|
|||
|
|
<span class="n">handler</span> <span class="o">=</span> <span class="n">MemoryHandler</span><span class="p">(</span><span class="n">capacity</span><span class="p">,</span> <span class="n">flushLevel</span><span class="o">=</span><span class="n">flush_level</span><span class="p">,</span> <span class="n">target</span><span class="o">=</span><span class="n">target_handler</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">decorator</span><span class="p">(</span><span class="n">fn</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">handler</span><span class="p">)</span>
|
|||
|
|
<span class="k">try</span><span class="p">:</span>
|
|||
|
|
<span class="k">return</span> <span class="n">fn</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
|||
|
|
<span class="k">except</span> <span class="ne">Exception</span><span class="p">:</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">exception</span><span class="p">(</span><span class="s1">'call failed'</span><span class="p">)</span>
|
|||
|
|
<span class="k">raise</span>
|
|||
|
|
<span class="k">finally</span><span class="p">:</span>
|
|||
|
|
<span class="nb">super</span><span class="p">(</span><span class="n">MemoryHandler</span><span class="p">,</span> <span class="n">handler</span><span class="p">)</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">removeHandler</span><span class="p">(</span><span class="n">handler</span><span class="p">)</span>
|
|||
|
|
<span class="k">return</span> <span class="n">wrapper</span>
|
|||
|
|
|
|||
|
|
<span class="k">return</span> <span class="n">decorator</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">write_line</span><span class="p">(</span><span class="n">s</span><span class="p">):</span>
|
|||
|
|
<span class="n">sys</span><span class="o">.</span><span class="n">stderr</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s1">'</span><span class="si">%s</span><span class="se">\n</span><span class="s1">'</span> <span class="o">%</span> <span class="n">s</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">fail</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
|
|||
|
|
<span class="n">write_line</span><span class="p">(</span><span class="s1">'about to log at DEBUG ...'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'Actually logged at DEBUG'</span><span class="p">)</span>
|
|||
|
|
<span class="n">write_line</span><span class="p">(</span><span class="s1">'about to log at INFO ...'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'Actually logged at INFO'</span><span class="p">)</span>
|
|||
|
|
<span class="n">write_line</span><span class="p">(</span><span class="s1">'about to log at WARNING ...'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="s1">'Actually logged at WARNING'</span><span class="p">)</span>
|
|||
|
|
<span class="k">if</span> <span class="n">fail</span><span class="p">:</span>
|
|||
|
|
<span class="n">write_line</span><span class="p">(</span><span class="s1">'about to log at ERROR ...'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s1">'Actually logged at ERROR'</span><span class="p">)</span>
|
|||
|
|
<span class="n">write_line</span><span class="p">(</span><span class="s1">'about to log at CRITICAL ...'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">critical</span><span class="p">(</span><span class="s1">'Actually logged at CRITICAL'</span><span class="p">)</span>
|
|||
|
|
<span class="k">return</span> <span class="n">fail</span>
|
|||
|
|
|
|||
|
|
<span class="n">decorated_foo</span> <span class="o">=</span> <span class="n">log_if_errors</span><span class="p">(</span><span class="n">logger</span><span class="p">)(</span><span class="n">foo</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
|
|||
|
|
<span class="n">write_line</span><span class="p">(</span><span class="s1">'Calling undecorated foo with False'</span><span class="p">)</span>
|
|||
|
|
<span class="k">assert</span> <span class="ow">not</span> <span class="n">foo</span><span class="p">(</span><span class="kc">False</span><span class="p">)</span>
|
|||
|
|
<span class="n">write_line</span><span class="p">(</span><span class="s1">'Calling undecorated foo with True'</span><span class="p">)</span>
|
|||
|
|
<span class="k">assert</span> <span class="n">foo</span><span class="p">(</span><span class="kc">True</span><span class="p">)</span>
|
|||
|
|
<span class="n">write_line</span><span class="p">(</span><span class="s1">'Calling decorated foo with False'</span><span class="p">)</span>
|
|||
|
|
<span class="k">assert</span> <span class="ow">not</span> <span class="n">decorated_foo</span><span class="p">(</span><span class="kc">False</span><span class="p">)</span>
|
|||
|
|
<span class="n">write_line</span><span class="p">(</span><span class="s1">'Calling decorated foo with True'</span><span class="p">)</span>
|
|||
|
|
<span class="k">assert</span> <span class="n">decorated_foo</span><span class="p">(</span><span class="kc">True</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>When this script is run, the following output should be observed:</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>Calling undecorated foo with False
|
|||
|
|
about to log at DEBUG ...
|
|||
|
|
about to log at INFO ...
|
|||
|
|
about to log at WARNING ...
|
|||
|
|
Calling undecorated foo with True
|
|||
|
|
about to log at DEBUG ...
|
|||
|
|
about to log at INFO ...
|
|||
|
|
about to log at WARNING ...
|
|||
|
|
about to log at ERROR ...
|
|||
|
|
about to log at CRITICAL ...
|
|||
|
|
Calling decorated foo with False
|
|||
|
|
about to log at DEBUG ...
|
|||
|
|
about to log at INFO ...
|
|||
|
|
about to log at WARNING ...
|
|||
|
|
Calling decorated foo with True
|
|||
|
|
about to log at DEBUG ...
|
|||
|
|
about to log at INFO ...
|
|||
|
|
about to log at WARNING ...
|
|||
|
|
about to log at ERROR ...
|
|||
|
|
Actually logged at DEBUG
|
|||
|
|
Actually logged at INFO
|
|||
|
|
Actually logged at WARNING
|
|||
|
|
Actually logged at ERROR
|
|||
|
|
about to log at CRITICAL ...
|
|||
|
|
Actually logged at CRITICAL
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>As you can see, actual logging output only occurs when an event is logged whose
|
|||
|
|
severity is ERROR or greater, but in that case, any previous events at lower
|
|||
|
|
severities are also logged.</p>
|
|||
|
|
<p>You can of course use the conventional means of decoration:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="nd">@log_if_errors</span><span class="p">(</span><span class="n">logger</span><span class="p">)</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">fail</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
|
|||
|
|
<span class="o">...</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="formatting-times-using-utc-gmt-via-configuration">
|
|||
|
|
<span id="utc-formatting"></span><h2>Formatting times using UTC (GMT) via configuration<a class="headerlink" href="#formatting-times-using-utc-gmt-via-configuration" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>Sometimes you want to format times using UTC, which can be done using a class
|
|||
|
|
such as <cite>UTCFormatter</cite>, shown below:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">time</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">UTCFormatter</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">):</span>
|
|||
|
|
<span class="n">converter</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">gmtime</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>and you can then use the <code class="docutils literal notranslate"><span class="pre">UTCFormatter</span></code> in your code instead of
|
|||
|
|
<a class="reference internal" href="../library/logging.html#logging.Formatter" title="logging.Formatter"><code class="xref py py-class docutils literal notranslate"><span class="pre">Formatter</span></code></a>. If you want to do that via configuration, you can
|
|||
|
|
use the <a class="reference internal" href="../library/logging.config.html#logging.config.dictConfig" title="logging.config.dictConfig"><code class="xref py py-func docutils literal notranslate"><span class="pre">dictConfig()</span></code></a> API with an approach illustrated by
|
|||
|
|
the following complete example:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">logging.config</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">time</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">UTCFormatter</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">):</span>
|
|||
|
|
<span class="n">converter</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">gmtime</span>
|
|||
|
|
|
|||
|
|
<span class="n">LOGGING</span> <span class="o">=</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'version'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'disable_existing_loggers'</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatters'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'utc'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'()'</span><span class="p">:</span> <span class="n">UTCFormatter</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'format'</span><span class="p">:</span> <span class="s1">'</span><span class="si">%(asctime)s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'local'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'format'</span><span class="p">:</span> <span class="s1">'</span><span class="si">%(asctime)s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">'</span><span class="p">,</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'console1'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.StreamHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatter'</span><span class="p">:</span> <span class="s1">'utc'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'console2'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'class'</span><span class="p">:</span> <span class="s1">'logging.StreamHandler'</span><span class="p">,</span>
|
|||
|
|
<span class="s1">'formatter'</span><span class="p">:</span> <span class="s1">'local'</span><span class="p">,</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="p">},</span>
|
|||
|
|
<span class="s1">'root'</span><span class="p">:</span> <span class="p">{</span>
|
|||
|
|
<span class="s1">'handlers'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'console1'</span><span class="p">,</span> <span class="s1">'console2'</span><span class="p">],</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
<span class="p">}</span>
|
|||
|
|
|
|||
|
|
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">dictConfig</span><span class="p">(</span><span class="n">LOGGING</span><span class="p">)</span>
|
|||
|
|
<span class="n">logging</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="s1">'The local time is </span><span class="si">%s</span><span class="s1">'</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">asctime</span><span class="p">())</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>When this script is run, it should print something like:</p>
|
|||
|
|
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>2015-10-17 12:53:29,501 The local time is Sat Oct 17 13:53:29 2015
|
|||
|
|
2015-10-17 13:53:29,501 The local time is Sat Oct 17 13:53:29 2015
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>showing how the time is formatted both as local time and UTC, one for each
|
|||
|
|
handler.</p>
|
|||
|
|
</div>
|
|||
|
|
<div class="section" id="using-a-context-manager-for-selective-logging">
|
|||
|
|
<span id="context-manager"></span><h2>Using a context manager for selective logging<a class="headerlink" href="#using-a-context-manager-for-selective-logging" title="Permalink to this headline">¶</a></h2>
|
|||
|
|
<p>There are times when it would be useful to temporarily change the logging
|
|||
|
|
configuration and revert it back after doing something. For this, a context
|
|||
|
|
manager is the most obvious way of saving and restoring the logging context.
|
|||
|
|
Here is a simple example of such a context manager, which allows you to
|
|||
|
|
optionally change the logging level and add a logging handler purely in the
|
|||
|
|
scope of the context manager:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
|
|||
|
|
<span class="kn">import</span> <span class="nn">sys</span>
|
|||
|
|
|
|||
|
|
<span class="k">class</span> <span class="nc">LoggingContext</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
|
|||
|
|
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">logger</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">handler</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">close</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span> <span class="o">=</span> <span class="n">logger</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">level</span> <span class="o">=</span> <span class="n">level</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">handler</span> <span class="o">=</span> <span class="n">handler</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">close</span> <span class="o">=</span> <span class="n">close</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">__enter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
|||
|
|
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">level</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">old_level</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">level</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">level</span><span class="p">)</span>
|
|||
|
|
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">handler</span><span class="p">:</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">handler</span><span class="p">)</span>
|
|||
|
|
|
|||
|
|
<span class="k">def</span> <span class="nf">__exit__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">et</span><span class="p">,</span> <span class="n">ev</span><span class="p">,</span> <span class="n">tb</span><span class="p">):</span>
|
|||
|
|
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">level</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">old_level</span><span class="p">)</span>
|
|||
|
|
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">handler</span><span class="p">:</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">removeHandler</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">handler</span><span class="p">)</span>
|
|||
|
|
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">handler</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">close</span><span class="p">:</span>
|
|||
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">handler</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
|
|||
|
|
<span class="c1"># implicit return of None => don't swallow exceptions</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>If you specify a level value, the logger’s level is set to that value in the
|
|||
|
|
scope of the with block covered by the context manager. If you specify a
|
|||
|
|
handler, it is added to the logger on entry to the block and removed on exit
|
|||
|
|
from the block. You can also ask the manager to close the handler for you on
|
|||
|
|
block exit - you could do this if you don’t need the handler any more.</p>
|
|||
|
|
<p>To illustrate how it works, we can add the following block of code to the
|
|||
|
|
above:</p>
|
|||
|
|
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
|||
|
|
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">StreamHandler</span><span class="p">())</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'1. This should appear just once on stderr.'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'2. This should not appear.'</span><span class="p">)</span>
|
|||
|
|
<span class="k">with</span> <span class="n">LoggingContext</span><span class="p">(</span><span class="n">logger</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">):</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'3. This should appear once on stderr.'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'4. This should not appear.'</span><span class="p">)</span>
|
|||
|
|
<span class="n">h</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">StreamHandler</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="p">)</span>
|
|||
|
|
<span class="k">with</span> <span class="n">LoggingContext</span><span class="p">(</span><span class="n">logger</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">,</span> <span class="n">handler</span><span class="o">=</span><span class="n">h</span><span class="p">,</span> <span class="n">close</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'5. This should appear twice - once on stderr and once on stdout.'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'6. This should appear just once on stderr.'</span><span class="p">)</span>
|
|||
|
|
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">'7. This should not appear.'</span><span class="p">)</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>We initially set the logger’s level to <code class="docutils literal notranslate"><span class="pre">INFO</span></code>, so message #1 appears and
|
|||
|
|
message #2 doesn’t. We then change the level to <code class="docutils literal notranslate"><span class="pre">DEBUG</span></code> temporarily in the
|
|||
|
|
following <code class="docutils literal notranslate"><span class="pre">with</span></code> block, and so message #3 appears. After the block exits, the
|
|||
|
|
logger’s level is restored to <code class="docutils literal notranslate"><span class="pre">INFO</span></code> and so message #4 doesn’t appear. In the
|
|||
|
|
next <code class="docutils literal notranslate"><span class="pre">with</span></code> block, we set the level to <code class="docutils literal notranslate"><span class="pre">DEBUG</span></code> again but also add a handler
|
|||
|
|
writing to <code class="docutils literal notranslate"><span class="pre">sys.stdout</span></code>. Thus, message #5 appears twice on the console (once
|
|||
|
|
via <code class="docutils literal notranslate"><span class="pre">stderr</span></code> and once via <code class="docutils literal notranslate"><span class="pre">stdout</span></code>). After the <code class="docutils literal notranslate"><span class="pre">with</span></code> statement’s
|
|||
|
|
completion, the status is as it was before so message #6 appears (like message
|
|||
|
|
#1) whereas message #7 doesn’t (just like message #2).</p>
|
|||
|
|
<p>If we run the resulting script, the result is as follows:</p>
|
|||
|
|
<div class="highlight-shell-session notranslate"><div class="highlight"><pre><span></span><span class="gp">$</span> python logctx.py
|
|||
|
|
<span class="go">1. This should appear just once on stderr.</span>
|
|||
|
|
<span class="go">3. This should appear once on stderr.</span>
|
|||
|
|
<span class="go">5. This should appear twice - once on stderr and once on stdout.</span>
|
|||
|
|
<span class="go">5. This should appear twice - once on stderr and once on stdout.</span>
|
|||
|
|
<span class="go">6. This should appear just once on stderr.</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>If we run it again, but pipe <code class="docutils literal notranslate"><span class="pre">stderr</span></code> to <code class="docutils literal notranslate"><span class="pre">/dev/null</span></code>, we see the following,
|
|||
|
|
which is the only message written to <code class="docutils literal notranslate"><span class="pre">stdout</span></code>:</p>
|
|||
|
|
<div class="highlight-shell-session notranslate"><div class="highlight"><pre><span></span><span class="gp">$</span> python logctx.py <span class="m">2</span>>/dev/null
|
|||
|
|
<span class="go">5. This should appear twice - once on stderr and once on stdout.</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>Once again, but piping <code class="docutils literal notranslate"><span class="pre">stdout</span></code> to <code class="docutils literal notranslate"><span class="pre">/dev/null</span></code>, we get:</p>
|
|||
|
|
<div class="highlight-shell-session notranslate"><div class="highlight"><pre><span></span><span class="gp">$</span> python logctx.py >/dev/null
|
|||
|
|
<span class="go">1. This should appear just once on stderr.</span>
|
|||
|
|
<span class="go">3. This should appear once on stderr.</span>
|
|||
|
|
<span class="go">5. This should appear twice - once on stderr and once on stdout.</span>
|
|||
|
|
<span class="go">6. This should appear just once on stderr.</span>
|
|||
|
|
</pre></div>
|
|||
|
|
</div>
|
|||
|
|
<p>In this case, the message #5 printed to <code class="docutils literal notranslate"><span class="pre">stdout</span></code> doesn’t appear, as expected.</p>
|
|||
|
|
<p>Of course, the approach described here can be generalised, for example to attach
|
|||
|
|
logging filters temporarily. Note that the above code works in Python 2 as well
|
|||
|
|
as Python 3.</p>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
|
|||
|
|
<div class="sphinxsidebarwrapper">
|
|||
|
|
<h3><a href="../contents.html">Table of Contents</a></h3>
|
|||
|
|
<ul>
|
|||
|
|
<li><a class="reference internal" href="#">Logging Cookbook</a><ul>
|
|||
|
|
<li><a class="reference internal" href="#using-logging-in-multiple-modules">Using logging in multiple modules</a></li>
|
|||
|
|
<li><a class="reference internal" href="#logging-from-multiple-threads">Logging from multiple threads</a></li>
|
|||
|
|
<li><a class="reference internal" href="#multiple-handlers-and-formatters">Multiple handlers and formatters</a></li>
|
|||
|
|
<li><a class="reference internal" href="#logging-to-multiple-destinations">Logging to multiple destinations</a></li>
|
|||
|
|
<li><a class="reference internal" href="#configuration-server-example">Configuration server example</a></li>
|
|||
|
|
<li><a class="reference internal" href="#dealing-with-handlers-that-block">Dealing with handlers that block</a></li>
|
|||
|
|
<li><a class="reference internal" href="#sending-and-receiving-logging-events-across-a-network">Sending and receiving logging events across a network</a></li>
|
|||
|
|
<li><a class="reference internal" href="#adding-contextual-information-to-your-logging-output">Adding contextual information to your logging output</a><ul>
|
|||
|
|
<li><a class="reference internal" href="#using-loggeradapters-to-impart-contextual-information">Using LoggerAdapters to impart contextual information</a><ul>
|
|||
|
|
<li><a class="reference internal" href="#using-objects-other-than-dicts-to-pass-contextual-information">Using objects other than dicts to pass contextual information</a></li>
|
|||
|
|
</ul>
|
|||
|
|
</li>
|
|||
|
|
<li><a class="reference internal" href="#using-filters-to-impart-contextual-information">Using Filters to impart contextual information</a></li>
|
|||
|
|
</ul>
|
|||
|
|
</li>
|
|||
|
|
<li><a class="reference internal" href="#logging-to-a-single-file-from-multiple-processes">Logging to a single file from multiple processes</a></li>
|
|||
|
|
<li><a class="reference internal" href="#using-file-rotation">Using file rotation</a></li>
|
|||
|
|
<li><a class="reference internal" href="#use-of-alternative-formatting-styles">Use of alternative formatting styles</a></li>
|
|||
|
|
<li><a class="reference internal" href="#customizing-logrecord">Customizing <code class="docutils literal notranslate"><span class="pre">LogRecord</span></code></a></li>
|
|||
|
|
<li><a class="reference internal" href="#subclassing-queuehandler-a-zeromq-example">Subclassing QueueHandler - a ZeroMQ example</a></li>
|
|||
|
|
<li><a class="reference internal" href="#subclassing-queuelistener-a-zeromq-example">Subclassing QueueListener - a ZeroMQ example</a></li>
|
|||
|
|
<li><a class="reference internal" href="#an-example-dictionary-based-configuration">An example dictionary-based configuration</a></li>
|
|||
|
|
<li><a class="reference internal" href="#using-a-rotator-and-namer-to-customize-log-rotation-processing">Using a rotator and namer to customize log rotation processing</a></li>
|
|||
|
|
<li><a class="reference internal" href="#a-more-elaborate-multiprocessing-example">A more elaborate multiprocessing example</a></li>
|
|||
|
|
<li><a class="reference internal" href="#inserting-a-bom-into-messages-sent-to-a-sysloghandler">Inserting a BOM into messages sent to a SysLogHandler</a></li>
|
|||
|
|
<li><a class="reference internal" href="#implementing-structured-logging">Implementing structured logging</a></li>
|
|||
|
|
<li><a class="reference internal" href="#customizing-handlers-with-dictconfig">Customizing handlers with <code class="xref py py-func docutils literal notranslate"><span class="pre">dictConfig()</span></code></a></li>
|
|||
|
|
<li><a class="reference internal" href="#using-particular-formatting-styles-throughout-your-application">Using particular formatting styles throughout your application</a><ul>
|
|||
|
|
<li><a class="reference internal" href="#using-logrecord-factories">Using LogRecord factories</a></li>
|
|||
|
|
<li><a class="reference internal" href="#using-custom-message-objects">Using custom message objects</a></li>
|
|||
|
|
</ul>
|
|||
|
|
</li>
|
|||
|
|
<li><a class="reference internal" href="#configuring-filters-with-dictconfig">Configuring filters with <code class="xref py py-func docutils literal notranslate"><span class="pre">dictConfig()</span></code></a></li>
|
|||
|
|
<li><a class="reference internal" href="#customized-exception-formatting">Customized exception formatting</a></li>
|
|||
|
|
<li><a class="reference internal" href="#speaking-logging-messages">Speaking logging messages</a></li>
|
|||
|
|
<li><a class="reference internal" href="#buffering-logging-messages-and-outputting-them-conditionally">Buffering logging messages and outputting them conditionally</a></li>
|
|||
|
|
<li><a class="reference internal" href="#formatting-times-using-utc-gmt-via-configuration">Formatting times using UTC (GMT) via configuration</a></li>
|
|||
|
|
<li><a class="reference internal" href="#using-a-context-manager-for-selective-logging">Using a context manager for selective logging</a></li>
|
|||
|
|
</ul>
|
|||
|
|
</li>
|
|||
|
|
</ul>
|
|||
|
|
|
|||
|
|
<h4>Previous topic</h4>
|
|||
|
|
<p class="topless"><a href="logging.html"
|
|||
|
|
title="previous chapter">Logging HOWTO</a></p>
|
|||
|
|
<h4>Next topic</h4>
|
|||
|
|
<p class="topless"><a href="regex.html"
|
|||
|
|
title="next chapter">Regular Expression HOWTO</a></p>
|
|||
|
|
<div role="note" aria-label="source link">
|
|||
|
|
<h3>This Page</h3>
|
|||
|
|
<ul class="this-page-menu">
|
|||
|
|
<li><a href="../bugs.html">Report a Bug</a></li>
|
|||
|
|
<li>
|
|||
|
|
<a href="https://github.com/python/cpython/blob/3.7/Doc/howto/logging-cookbook.rst"
|
|||
|
|
rel="nofollow">Show Source
|
|||
|
|
</a>
|
|||
|
|
</li>
|
|||
|
|
</ul>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
<div class="clearer"></div>
|
|||
|
|
</div>
|
|||
|
|
<div class="related" role="navigation" aria-label="related navigation">
|
|||
|
|
<h3>Navigation</h3>
|
|||
|
|
<ul>
|
|||
|
|
<li class="right" style="margin-right: 10px">
|
|||
|
|
<a href="../genindex.html" title="General Index"
|
|||
|
|
>index</a></li>
|
|||
|
|
<li class="right" >
|
|||
|
|
<a href="../py-modindex.html" title="Python Module Index"
|
|||
|
|
>modules</a> |</li>
|
|||
|
|
<li class="right" >
|
|||
|
|
<a href="regex.html" title="Regular Expression HOWTO"
|
|||
|
|
>next</a> |</li>
|
|||
|
|
<li class="right" >
|
|||
|
|
<a href="logging.html" title="Logging HOWTO"
|
|||
|
|
>previous</a> |</li>
|
|||
|
|
<li><img src="../_static/py.png" alt=""
|
|||
|
|
style="vertical-align: middle; margin-top: -1px"/></li>
|
|||
|
|
<li><a href="https://www.python.org/">Python</a> »</li>
|
|||
|
|
<li>
|
|||
|
|
<span class="language_switcher_placeholder">en</span>
|
|||
|
|
<span class="version_switcher_placeholder">3.7.4</span>
|
|||
|
|
<a href="../index.html">Documentation </a> »
|
|||
|
|
</li>
|
|||
|
|
|
|||
|
|
<li class="nav-item nav-item-1"><a href="index.html" >Python HOWTOs</a> »</li>
|
|||
|
|
<li class="right">
|
|||
|
|
|
|||
|
|
|
|||
|
|
<div class="inline-search" style="display: none" role="search">
|
|||
|
|
<form class="inline-search" action="../search.html" method="get">
|
|||
|
|
<input placeholder="Quick search" type="text" name="q" />
|
|||
|
|
<input type="submit" value="Go" />
|
|||
|
|
<input type="hidden" name="check_keywords" value="yes" />
|
|||
|
|
<input type="hidden" name="area" value="default" />
|
|||
|
|
</form>
|
|||
|
|
</div>
|
|||
|
|
<script type="text/javascript">$('.inline-search').show(0);</script>
|
|||
|
|
|
|
|||
|
|
</li>
|
|||
|
|
|
|||
|
|
</ul>
|
|||
|
|
</div>
|
|||
|
|
<div class="footer">
|
|||
|
|
© <a href="../copyright.html">Copyright</a> 2001-2019, Python Software Foundation.
|
|||
|
|
<br />
|
|||
|
|
The Python Software Foundation is a non-profit corporation.
|
|||
|
|
<a href="https://www.python.org/psf/donations/">Please donate.</a>
|
|||
|
|
<br />
|
|||
|
|
Last updated on Jul 13, 2019.
|
|||
|
|
<a href="../bugs.html">Found a bug</a>?
|
|||
|
|
<br />
|
|||
|
|
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 2.0.1.
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
</body>
|
|||
|
|
</html>
|