479 lines
36 KiB
HTML
479 lines
36 KiB
HTML
|
||
<!DOCTYPE html>
|
||
|
||
<html xmlns="http://www.w3.org/1999/xhtml">
|
||
<head>
|
||
<meta charset="utf-8" />
|
||
<title>heapq — Heap queue algorithm — Python 3.7.4 documentation</title>
|
||
<link rel="stylesheet" href="../_static/pydoctheme.css" type="text/css" />
|
||
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
|
||
|
||
<script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
|
||
<script type="text/javascript" src="../_static/jquery.js"></script>
|
||
<script type="text/javascript" src="../_static/underscore.js"></script>
|
||
<script type="text/javascript" src="../_static/doctools.js"></script>
|
||
<script type="text/javascript" src="../_static/language_data.js"></script>
|
||
|
||
<script type="text/javascript" src="../_static/sidebar.js"></script>
|
||
|
||
<link rel="search" type="application/opensearchdescription+xml"
|
||
title="Search within Python 3.7.4 documentation"
|
||
href="../_static/opensearch.xml"/>
|
||
<link rel="author" title="About these documents" href="../about.html" />
|
||
<link rel="index" title="Index" href="../genindex.html" />
|
||
<link rel="search" title="Search" href="../search.html" />
|
||
<link rel="copyright" title="Copyright" href="../copyright.html" />
|
||
<link rel="next" title="bisect — Array bisection algorithm" href="bisect.html" />
|
||
<link rel="prev" title="collections.abc — Abstract Base Classes for Containers" href="collections.abc.html" />
|
||
<link rel="shortcut icon" type="image/png" href="../_static/py.png" />
|
||
<link rel="canonical" href="https://docs.python.org/3/library/heapq.html" />
|
||
|
||
<script type="text/javascript" src="../_static/copybutton.js"></script>
|
||
<script type="text/javascript" src="../_static/switchers.js"></script>
|
||
|
||
|
||
|
||
<style>
|
||
@media only screen {
|
||
table.full-width-table {
|
||
width: 100%;
|
||
}
|
||
}
|
||
</style>
|
||
|
||
|
||
</head><body>
|
||
|
||
<div class="related" role="navigation" aria-label="related navigation">
|
||
<h3>Navigation</h3>
|
||
<ul>
|
||
<li class="right" style="margin-right: 10px">
|
||
<a href="../genindex.html" title="General Index"
|
||
accesskey="I">index</a></li>
|
||
<li class="right" >
|
||
<a href="../py-modindex.html" title="Python Module Index"
|
||
>modules</a> |</li>
|
||
<li class="right" >
|
||
<a href="bisect.html" title="bisect — Array bisection algorithm"
|
||
accesskey="N">next</a> |</li>
|
||
<li class="right" >
|
||
<a href="collections.abc.html" title="collections.abc — Abstract Base Classes for Containers"
|
||
accesskey="P">previous</a> |</li>
|
||
<li><img src="../_static/py.png" alt=""
|
||
style="vertical-align: middle; margin-top: -1px"/></li>
|
||
<li><a href="https://www.python.org/">Python</a> »</li>
|
||
<li>
|
||
<span class="language_switcher_placeholder">en</span>
|
||
<span class="version_switcher_placeholder">3.7.4</span>
|
||
<a href="../index.html">Documentation </a> »
|
||
</li>
|
||
|
||
<li class="nav-item nav-item-1"><a href="index.html" >The Python Standard Library</a> »</li>
|
||
<li class="nav-item nav-item-2"><a href="datatypes.html" accesskey="U">Data Types</a> »</li>
|
||
<li class="right">
|
||
|
||
|
||
<div class="inline-search" style="display: none" role="search">
|
||
<form class="inline-search" action="../search.html" method="get">
|
||
<input placeholder="Quick search" type="text" name="q" />
|
||
<input type="submit" value="Go" />
|
||
<input type="hidden" name="check_keywords" value="yes" />
|
||
<input type="hidden" name="area" value="default" />
|
||
</form>
|
||
</div>
|
||
<script type="text/javascript">$('.inline-search').show(0);</script>
|
||
|
|
||
</li>
|
||
|
||
</ul>
|
||
</div>
|
||
|
||
<div class="document">
|
||
<div class="documentwrapper">
|
||
<div class="bodywrapper">
|
||
<div class="body" role="main">
|
||
|
||
<div class="section" id="module-heapq">
|
||
<span id="heapq-heap-queue-algorithm"></span><h1><a class="reference internal" href="#module-heapq" title="heapq: Heap queue algorithm (a.k.a. priority queue)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">heapq</span></code></a> — Heap queue algorithm<a class="headerlink" href="#module-heapq" title="Permalink to this headline">¶</a></h1>
|
||
<p><strong>Source code:</strong> <a class="reference external" href="https://github.com/python/cpython/tree/3.7/Lib/heapq.py">Lib/heapq.py</a></p>
|
||
<hr class="docutils" />
|
||
<p>This module provides an implementation of the heap queue algorithm, also known
|
||
as the priority queue algorithm.</p>
|
||
<p>Heaps are binary trees for which every parent node has a value less than or
|
||
equal to any of its children. This implementation uses arrays for which
|
||
<code class="docutils literal notranslate"><span class="pre">heap[k]</span> <span class="pre"><=</span> <span class="pre">heap[2*k+1]</span></code> and <code class="docutils literal notranslate"><span class="pre">heap[k]</span> <span class="pre"><=</span> <span class="pre">heap[2*k+2]</span></code> for all <em>k</em>, counting
|
||
elements from zero. For the sake of comparison, non-existing elements are
|
||
considered to be infinite. The interesting property of a heap is that its
|
||
smallest element is always the root, <code class="docutils literal notranslate"><span class="pre">heap[0]</span></code>.</p>
|
||
<p>The API below differs from textbook heap algorithms in two aspects: (a) We use
|
||
zero-based indexing. This makes the relationship between the index for a node
|
||
and the indexes for its children slightly less obvious, but is more suitable
|
||
since Python uses zero-based indexing. (b) Our pop method returns the smallest
|
||
item, not the largest (called a “min heap” in textbooks; a “max heap” is more
|
||
common in texts because of its suitability for in-place sorting).</p>
|
||
<p>These two make it possible to view the heap as a regular Python list without
|
||
surprises: <code class="docutils literal notranslate"><span class="pre">heap[0]</span></code> is the smallest item, and <code class="docutils literal notranslate"><span class="pre">heap.sort()</span></code> maintains the
|
||
heap invariant!</p>
|
||
<p>To create a heap, use a list initialized to <code class="docutils literal notranslate"><span class="pre">[]</span></code>, or you can transform a
|
||
populated list into a heap via function <a class="reference internal" href="#heapq.heapify" title="heapq.heapify"><code class="xref py py-func docutils literal notranslate"><span class="pre">heapify()</span></code></a>.</p>
|
||
<p>The following functions are provided:</p>
|
||
<dl class="function">
|
||
<dt id="heapq.heappush">
|
||
<code class="descclassname">heapq.</code><code class="descname">heappush</code><span class="sig-paren">(</span><em>heap</em>, <em>item</em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.heappush" title="Permalink to this definition">¶</a></dt>
|
||
<dd><p>Push the value <em>item</em> onto the <em>heap</em>, maintaining the heap invariant.</p>
|
||
</dd></dl>
|
||
|
||
<dl class="function">
|
||
<dt id="heapq.heappop">
|
||
<code class="descclassname">heapq.</code><code class="descname">heappop</code><span class="sig-paren">(</span><em>heap</em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.heappop" title="Permalink to this definition">¶</a></dt>
|
||
<dd><p>Pop and return the smallest item from the <em>heap</em>, maintaining the heap
|
||
invariant. If the heap is empty, <a class="reference internal" href="exceptions.html#IndexError" title="IndexError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">IndexError</span></code></a> is raised. To access the
|
||
smallest item without popping it, use <code class="docutils literal notranslate"><span class="pre">heap[0]</span></code>.</p>
|
||
</dd></dl>
|
||
|
||
<dl class="function">
|
||
<dt id="heapq.heappushpop">
|
||
<code class="descclassname">heapq.</code><code class="descname">heappushpop</code><span class="sig-paren">(</span><em>heap</em>, <em>item</em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.heappushpop" title="Permalink to this definition">¶</a></dt>
|
||
<dd><p>Push <em>item</em> on the heap, then pop and return the smallest item from the
|
||
<em>heap</em>. The combined action runs more efficiently than <a class="reference internal" href="#heapq.heappush" title="heapq.heappush"><code class="xref py py-func docutils literal notranslate"><span class="pre">heappush()</span></code></a>
|
||
followed by a separate call to <a class="reference internal" href="#heapq.heappop" title="heapq.heappop"><code class="xref py py-func docutils literal notranslate"><span class="pre">heappop()</span></code></a>.</p>
|
||
</dd></dl>
|
||
|
||
<dl class="function">
|
||
<dt id="heapq.heapify">
|
||
<code class="descclassname">heapq.</code><code class="descname">heapify</code><span class="sig-paren">(</span><em>x</em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.heapify" title="Permalink to this definition">¶</a></dt>
|
||
<dd><p>Transform list <em>x</em> into a heap, in-place, in linear time.</p>
|
||
</dd></dl>
|
||
|
||
<dl class="function">
|
||
<dt id="heapq.heapreplace">
|
||
<code class="descclassname">heapq.</code><code class="descname">heapreplace</code><span class="sig-paren">(</span><em>heap</em>, <em>item</em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.heapreplace" title="Permalink to this definition">¶</a></dt>
|
||
<dd><p>Pop and return the smallest item from the <em>heap</em>, and also push the new <em>item</em>.
|
||
The heap size doesn’t change. If the heap is empty, <a class="reference internal" href="exceptions.html#IndexError" title="IndexError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">IndexError</span></code></a> is raised.</p>
|
||
<p>This one step operation is more efficient than a <a class="reference internal" href="#heapq.heappop" title="heapq.heappop"><code class="xref py py-func docutils literal notranslate"><span class="pre">heappop()</span></code></a> followed by
|
||
<a class="reference internal" href="#heapq.heappush" title="heapq.heappush"><code class="xref py py-func docutils literal notranslate"><span class="pre">heappush()</span></code></a> and can be more appropriate when using a fixed-size heap.
|
||
The pop/push combination always returns an element from the heap and replaces
|
||
it with <em>item</em>.</p>
|
||
<p>The value returned may be larger than the <em>item</em> added. If that isn’t
|
||
desired, consider using <a class="reference internal" href="#heapq.heappushpop" title="heapq.heappushpop"><code class="xref py py-func docutils literal notranslate"><span class="pre">heappushpop()</span></code></a> instead. Its push/pop
|
||
combination returns the smaller of the two values, leaving the larger value
|
||
on the heap.</p>
|
||
</dd></dl>
|
||
|
||
<p>The module also offers three general purpose functions based on heaps.</p>
|
||
<dl class="function">
|
||
<dt id="heapq.merge">
|
||
<code class="descclassname">heapq.</code><code class="descname">merge</code><span class="sig-paren">(</span><em>*iterables</em>, <em>key=None</em>, <em>reverse=False</em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.merge" title="Permalink to this definition">¶</a></dt>
|
||
<dd><p>Merge multiple sorted inputs into a single sorted output (for example, merge
|
||
timestamped entries from multiple log files). Returns an <a class="reference internal" href="../glossary.html#term-iterator"><span class="xref std std-term">iterator</span></a>
|
||
over the sorted values.</p>
|
||
<p>Similar to <code class="docutils literal notranslate"><span class="pre">sorted(itertools.chain(*iterables))</span></code> but returns an iterable, does
|
||
not pull the data into memory all at once, and assumes that each of the input
|
||
streams is already sorted (smallest to largest).</p>
|
||
<p>Has two optional arguments which must be specified as keyword arguments.</p>
|
||
<p><em>key</em> specifies a <a class="reference internal" href="../glossary.html#term-key-function"><span class="xref std std-term">key function</span></a> of one argument that is used to
|
||
extract a comparison key from each input element. The default value is
|
||
<code class="docutils literal notranslate"><span class="pre">None</span></code> (compare the elements directly).</p>
|
||
<p><em>reverse</em> is a boolean value. If set to <code class="docutils literal notranslate"><span class="pre">True</span></code>, then the input elements
|
||
are merged as if each comparison were reversed. To achieve behavior similar
|
||
to <code class="docutils literal notranslate"><span class="pre">sorted(itertools.chain(*iterables),</span> <span class="pre">reverse=True)</span></code>, all iterables must
|
||
be sorted from largest to smallest.</p>
|
||
<div class="versionchanged">
|
||
<p><span class="versionmodified changed">Changed in version 3.5: </span>Added the optional <em>key</em> and <em>reverse</em> parameters.</p>
|
||
</div>
|
||
</dd></dl>
|
||
|
||
<dl class="function">
|
||
<dt id="heapq.nlargest">
|
||
<code class="descclassname">heapq.</code><code class="descname">nlargest</code><span class="sig-paren">(</span><em>n</em>, <em>iterable</em>, <em>key=None</em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.nlargest" title="Permalink to this definition">¶</a></dt>
|
||
<dd><p>Return a list with the <em>n</em> largest elements from the dataset defined by
|
||
<em>iterable</em>. <em>key</em>, if provided, specifies a function of one argument that is
|
||
used to extract a comparison key from each element in <em>iterable</em> (for example,
|
||
<code class="docutils literal notranslate"><span class="pre">key=str.lower</span></code>). Equivalent to: <code class="docutils literal notranslate"><span class="pre">sorted(iterable,</span> <span class="pre">key=key,</span>
|
||
<span class="pre">reverse=True)[:n]</span></code>.</p>
|
||
</dd></dl>
|
||
|
||
<dl class="function">
|
||
<dt id="heapq.nsmallest">
|
||
<code class="descclassname">heapq.</code><code class="descname">nsmallest</code><span class="sig-paren">(</span><em>n</em>, <em>iterable</em>, <em>key=None</em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.nsmallest" title="Permalink to this definition">¶</a></dt>
|
||
<dd><p>Return a list with the <em>n</em> smallest elements from the dataset defined by
|
||
<em>iterable</em>. <em>key</em>, if provided, specifies a function of one argument that is
|
||
used to extract a comparison key from each element in <em>iterable</em> (for example,
|
||
<code class="docutils literal notranslate"><span class="pre">key=str.lower</span></code>). Equivalent to: <code class="docutils literal notranslate"><span class="pre">sorted(iterable,</span> <span class="pre">key=key)[:n]</span></code>.</p>
|
||
</dd></dl>
|
||
|
||
<p>The latter two functions perform best for smaller values of <em>n</em>. For larger
|
||
values, it is more efficient to use the <a class="reference internal" href="functions.html#sorted" title="sorted"><code class="xref py py-func docutils literal notranslate"><span class="pre">sorted()</span></code></a> function. Also, when
|
||
<code class="docutils literal notranslate"><span class="pre">n==1</span></code>, it is more efficient to use the built-in <a class="reference internal" href="functions.html#min" title="min"><code class="xref py py-func docutils literal notranslate"><span class="pre">min()</span></code></a> and <a class="reference internal" href="functions.html#max" title="max"><code class="xref py py-func docutils literal notranslate"><span class="pre">max()</span></code></a>
|
||
functions. If repeated usage of these functions is required, consider turning
|
||
the iterable into an actual heap.</p>
|
||
<div class="section" id="basic-examples">
|
||
<h2>Basic Examples<a class="headerlink" href="#basic-examples" title="Permalink to this headline">¶</a></h2>
|
||
<p>A <a class="reference external" href="https://en.wikipedia.org/wiki/Heapsort">heapsort</a> can be implemented by
|
||
pushing all values onto a heap and then popping off the smallest values one at a
|
||
time:</p>
|
||
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">def</span> <span class="nf">heapsort</span><span class="p">(</span><span class="n">iterable</span><span class="p">):</span>
|
||
<span class="gp">... </span> <span class="n">h</span> <span class="o">=</span> <span class="p">[]</span>
|
||
<span class="gp">... </span> <span class="k">for</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">iterable</span><span class="p">:</span>
|
||
<span class="gp">... </span> <span class="n">heappush</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
|
||
<span class="gp">... </span> <span class="k">return</span> <span class="p">[</span><span class="n">heappop</span><span class="p">(</span><span class="n">h</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">h</span><span class="p">))]</span>
|
||
<span class="gp">...</span>
|
||
<span class="gp">>>> </span><span class="n">heapsort</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">0</span><span class="p">])</span>
|
||
<span class="go">[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>This is similar to <code class="docutils literal notranslate"><span class="pre">sorted(iterable)</span></code>, but unlike <a class="reference internal" href="functions.html#sorted" title="sorted"><code class="xref py py-func docutils literal notranslate"><span class="pre">sorted()</span></code></a>, this
|
||
implementation is not stable.</p>
|
||
<p>Heap elements can be tuples. This is useful for assigning comparison values
|
||
(such as task priorities) alongside the main record being tracked:</p>
|
||
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">h</span> <span class="o">=</span> <span class="p">[]</span>
|
||
<span class="gp">>>> </span><span class="n">heappush</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="s1">'write code'</span><span class="p">))</span>
|
||
<span class="gp">>>> </span><span class="n">heappush</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="p">(</span><span class="mi">7</span><span class="p">,</span> <span class="s1">'release product'</span><span class="p">))</span>
|
||
<span class="gp">>>> </span><span class="n">heappush</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s1">'write spec'</span><span class="p">))</span>
|
||
<span class="gp">>>> </span><span class="n">heappush</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="s1">'create tests'</span><span class="p">))</span>
|
||
<span class="gp">>>> </span><span class="n">heappop</span><span class="p">(</span><span class="n">h</span><span class="p">)</span>
|
||
<span class="go">(1, 'write spec')</span>
|
||
</pre></div>
|
||
</div>
|
||
</div>
|
||
<div class="section" id="priority-queue-implementation-notes">
|
||
<h2>Priority Queue Implementation Notes<a class="headerlink" href="#priority-queue-implementation-notes" title="Permalink to this headline">¶</a></h2>
|
||
<p>A <a class="reference external" href="https://en.wikipedia.org/wiki/Priority_queue">priority queue</a> is common use
|
||
for a heap, and it presents several implementation challenges:</p>
|
||
<ul class="simple">
|
||
<li><p>Sort stability: how do you get two tasks with equal priorities to be returned
|
||
in the order they were originally added?</p></li>
|
||
<li><p>Tuple comparison breaks for (priority, task) pairs if the priorities are equal
|
||
and the tasks do not have a default comparison order.</p></li>
|
||
<li><p>If the priority of a task changes, how do you move it to a new position in
|
||
the heap?</p></li>
|
||
<li><p>Or if a pending task needs to be deleted, how do you find it and remove it
|
||
from the queue?</p></li>
|
||
</ul>
|
||
<p>A solution to the first two challenges is to store entries as 3-element list
|
||
including the priority, an entry count, and the task. The entry count serves as
|
||
a tie-breaker so that two tasks with the same priority are returned in the order
|
||
they were added. And since no two entry counts are the same, the tuple
|
||
comparison will never attempt to directly compare two tasks.</p>
|
||
<p>Another solution to the problem of non-comparable tasks is to create a wrapper
|
||
class that ignores the task item and only compares the priority field:</p>
|
||
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">dataclasses</span> <span class="k">import</span> <span class="n">dataclass</span><span class="p">,</span> <span class="n">field</span>
|
||
<span class="kn">from</span> <span class="nn">typing</span> <span class="k">import</span> <span class="n">Any</span>
|
||
|
||
<span class="nd">@dataclass</span><span class="p">(</span><span class="n">order</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||
<span class="k">class</span> <span class="nc">PrioritizedItem</span><span class="p">:</span>
|
||
<span class="n">priority</span><span class="p">:</span> <span class="nb">int</span>
|
||
<span class="n">item</span><span class="p">:</span> <span class="n">Any</span><span class="o">=</span><span class="n">field</span><span class="p">(</span><span class="n">compare</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>The remaining challenges revolve around finding a pending task and making
|
||
changes to its priority or removing it entirely. Finding a task can be done
|
||
with a dictionary pointing to an entry in the queue.</p>
|
||
<p>Removing the entry or changing its priority is more difficult because it would
|
||
break the heap structure invariants. So, a possible solution is to mark the
|
||
entry as removed and add a new entry with the revised priority:</p>
|
||
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">pq</span> <span class="o">=</span> <span class="p">[]</span> <span class="c1"># list of entries arranged in a heap</span>
|
||
<span class="n">entry_finder</span> <span class="o">=</span> <span class="p">{}</span> <span class="c1"># mapping of tasks to entries</span>
|
||
<span class="n">REMOVED</span> <span class="o">=</span> <span class="s1">'<removed-task>'</span> <span class="c1"># placeholder for a removed task</span>
|
||
<span class="n">counter</span> <span class="o">=</span> <span class="n">itertools</span><span class="o">.</span><span class="n">count</span><span class="p">()</span> <span class="c1"># unique sequence count</span>
|
||
|
||
<span class="k">def</span> <span class="nf">add_task</span><span class="p">(</span><span class="n">task</span><span class="p">,</span> <span class="n">priority</span><span class="o">=</span><span class="mi">0</span><span class="p">):</span>
|
||
<span class="s1">'Add a new task or update the priority of an existing task'</span>
|
||
<span class="k">if</span> <span class="n">task</span> <span class="ow">in</span> <span class="n">entry_finder</span><span class="p">:</span>
|
||
<span class="n">remove_task</span><span class="p">(</span><span class="n">task</span><span class="p">)</span>
|
||
<span class="n">count</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">counter</span><span class="p">)</span>
|
||
<span class="n">entry</span> <span class="o">=</span> <span class="p">[</span><span class="n">priority</span><span class="p">,</span> <span class="n">count</span><span class="p">,</span> <span class="n">task</span><span class="p">]</span>
|
||
<span class="n">entry_finder</span><span class="p">[</span><span class="n">task</span><span class="p">]</span> <span class="o">=</span> <span class="n">entry</span>
|
||
<span class="n">heappush</span><span class="p">(</span><span class="n">pq</span><span class="p">,</span> <span class="n">entry</span><span class="p">)</span>
|
||
|
||
<span class="k">def</span> <span class="nf">remove_task</span><span class="p">(</span><span class="n">task</span><span class="p">):</span>
|
||
<span class="s1">'Mark an existing task as REMOVED. Raise KeyError if not found.'</span>
|
||
<span class="n">entry</span> <span class="o">=</span> <span class="n">entry_finder</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">task</span><span class="p">)</span>
|
||
<span class="n">entry</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">REMOVED</span>
|
||
|
||
<span class="k">def</span> <span class="nf">pop_task</span><span class="p">():</span>
|
||
<span class="s1">'Remove and return the lowest priority task. Raise KeyError if empty.'</span>
|
||
<span class="k">while</span> <span class="n">pq</span><span class="p">:</span>
|
||
<span class="n">priority</span><span class="p">,</span> <span class="n">count</span><span class="p">,</span> <span class="n">task</span> <span class="o">=</span> <span class="n">heappop</span><span class="p">(</span><span class="n">pq</span><span class="p">)</span>
|
||
<span class="k">if</span> <span class="n">task</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">REMOVED</span><span class="p">:</span>
|
||
<span class="k">del</span> <span class="n">entry_finder</span><span class="p">[</span><span class="n">task</span><span class="p">]</span>
|
||
<span class="k">return</span> <span class="n">task</span>
|
||
<span class="k">raise</span> <span class="ne">KeyError</span><span class="p">(</span><span class="s1">'pop from an empty priority queue'</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
</div>
|
||
<div class="section" id="theory">
|
||
<h2>Theory<a class="headerlink" href="#theory" title="Permalink to this headline">¶</a></h2>
|
||
<p>Heaps are arrays for which <code class="docutils literal notranslate"><span class="pre">a[k]</span> <span class="pre"><=</span> <span class="pre">a[2*k+1]</span></code> and <code class="docutils literal notranslate"><span class="pre">a[k]</span> <span class="pre"><=</span> <span class="pre">a[2*k+2]</span></code> for all
|
||
<em>k</em>, counting elements from 0. For the sake of comparison, non-existing
|
||
elements are considered to be infinite. The interesting property of a heap is
|
||
that <code class="docutils literal notranslate"><span class="pre">a[0]</span></code> is always its smallest element.</p>
|
||
<p>The strange invariant above is meant to be an efficient memory representation
|
||
for a tournament. The numbers below are <em>k</em>, not <code class="docutils literal notranslate"><span class="pre">a[k]</span></code>:</p>
|
||
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span> <span class="mi">0</span>
|
||
|
||
<span class="mi">1</span> <span class="mi">2</span>
|
||
|
||
<span class="mi">3</span> <span class="mi">4</span> <span class="mi">5</span> <span class="mi">6</span>
|
||
|
||
<span class="mi">7</span> <span class="mi">8</span> <span class="mi">9</span> <span class="mi">10</span> <span class="mi">11</span> <span class="mi">12</span> <span class="mi">13</span> <span class="mi">14</span>
|
||
|
||
<span class="mi">15</span> <span class="mi">16</span> <span class="mi">17</span> <span class="mi">18</span> <span class="mi">19</span> <span class="mi">20</span> <span class="mi">21</span> <span class="mi">22</span> <span class="mi">23</span> <span class="mi">24</span> <span class="mi">25</span> <span class="mi">26</span> <span class="mi">27</span> <span class="mi">28</span> <span class="mi">29</span> <span class="mi">30</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>In the tree above, each cell <em>k</em> is topping <code class="docutils literal notranslate"><span class="pre">2*k+1</span></code> and <code class="docutils literal notranslate"><span class="pre">2*k+2</span></code>. In a usual
|
||
binary tournament we see in sports, each cell is the winner over the two cells
|
||
it tops, and we can trace the winner down the tree to see all opponents s/he
|
||
had. However, in many computer applications of such tournaments, we do not need
|
||
to trace the history of a winner. To be more memory efficient, when a winner is
|
||
promoted, we try to replace it by something else at a lower level, and the rule
|
||
becomes that a cell and the two cells it tops contain three different items, but
|
||
the top cell “wins” over the two topped cells.</p>
|
||
<p>If this heap invariant is protected at all time, index 0 is clearly the overall
|
||
winner. The simplest algorithmic way to remove it and find the “next” winner is
|
||
to move some loser (let’s say cell 30 in the diagram above) into the 0 position,
|
||
and then percolate this new 0 down the tree, exchanging values, until the
|
||
invariant is re-established. This is clearly logarithmic on the total number of
|
||
items in the tree. By iterating over all items, you get an O(n log n) sort.</p>
|
||
<p>A nice feature of this sort is that you can efficiently insert new items while
|
||
the sort is going on, provided that the inserted items are not “better” than the
|
||
last 0’th element you extracted. This is especially useful in simulation
|
||
contexts, where the tree holds all incoming events, and the “win” condition
|
||
means the smallest scheduled time. When an event schedules other events for
|
||
execution, they are scheduled into the future, so they can easily go into the
|
||
heap. So, a heap is a good structure for implementing schedulers (this is what
|
||
I used for my MIDI sequencer :-).</p>
|
||
<p>Various structures for implementing schedulers have been extensively studied,
|
||
and heaps are good for this, as they are reasonably speedy, the speed is almost
|
||
constant, and the worst case is not much different than the average case.
|
||
However, there are other representations which are more efficient overall, yet
|
||
the worst cases might be terrible.</p>
|
||
<p>Heaps are also very useful in big disk sorts. You most probably all know that a
|
||
big sort implies producing “runs” (which are pre-sorted sequences, whose size is
|
||
usually related to the amount of CPU memory), followed by a merging passes for
|
||
these runs, which merging is often very cleverly organised <a class="footnote-reference brackets" href="#id2" id="id1">1</a>. It is very
|
||
important that the initial sort produces the longest runs possible. Tournaments
|
||
are a good way to achieve that. If, using all the memory available to hold a
|
||
tournament, you replace and percolate items that happen to fit the current run,
|
||
you’ll produce runs which are twice the size of the memory for random input, and
|
||
much better for input fuzzily ordered.</p>
|
||
<p>Moreover, if you output the 0’th item on disk and get an input which may not fit
|
||
in the current tournament (because the value “wins” over the last output value),
|
||
it cannot fit in the heap, so the size of the heap decreases. The freed memory
|
||
could be cleverly reused immediately for progressively building a second heap,
|
||
which grows at exactly the same rate the first heap is melting. When the first
|
||
heap completely vanishes, you switch heaps and start a new run. Clever and
|
||
quite effective!</p>
|
||
<p>In a word, heaps are useful memory structures to know. I use them in a few
|
||
applications, and I think it is good to keep a ‘heap’ module around. :-)</p>
|
||
<p class="rubric">Footnotes</p>
|
||
<dl class="footnote brackets">
|
||
<dt class="label" id="id2"><span class="brackets"><a class="fn-backref" href="#id1">1</a></span></dt>
|
||
<dd><p>The disk balancing algorithms which are current, nowadays, are more annoying
|
||
than clever, and this is a consequence of the seeking capabilities of the disks.
|
||
On devices which cannot seek, like big tape drives, the story was quite
|
||
different, and one had to be very clever to ensure (far in advance) that each
|
||
tape movement will be the most effective possible (that is, will best
|
||
participate at “progressing” the merge). Some tapes were even able to read
|
||
backwards, and this was also used to avoid the rewinding time. Believe me, real
|
||
good tape sorts were quite spectacular to watch! From all times, sorting has
|
||
always been a Great Art! :-)</p>
|
||
</dd>
|
||
</dl>
|
||
</div>
|
||
</div>
|
||
|
||
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
|
||
<div class="sphinxsidebarwrapper">
|
||
<h3><a href="../contents.html">Table of Contents</a></h3>
|
||
<ul>
|
||
<li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">heapq</span></code> — Heap queue algorithm</a><ul>
|
||
<li><a class="reference internal" href="#basic-examples">Basic Examples</a></li>
|
||
<li><a class="reference internal" href="#priority-queue-implementation-notes">Priority Queue Implementation Notes</a></li>
|
||
<li><a class="reference internal" href="#theory">Theory</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
|
||
<h4>Previous topic</h4>
|
||
<p class="topless"><a href="collections.abc.html"
|
||
title="previous chapter"><code class="xref py py-mod docutils literal notranslate"><span class="pre">collections.abc</span></code> — Abstract Base Classes for Containers</a></p>
|
||
<h4>Next topic</h4>
|
||
<p class="topless"><a href="bisect.html"
|
||
title="next chapter"><code class="xref py py-mod docutils literal notranslate"><span class="pre">bisect</span></code> — Array bisection algorithm</a></p>
|
||
<div role="note" aria-label="source link">
|
||
<h3>This Page</h3>
|
||
<ul class="this-page-menu">
|
||
<li><a href="../bugs.html">Report a Bug</a></li>
|
||
<li>
|
||
<a href="https://github.com/python/cpython/blob/3.7/Doc/library/heapq.rst"
|
||
rel="nofollow">Show Source
|
||
</a>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="clearer"></div>
|
||
</div>
|
||
<div class="related" role="navigation" aria-label="related navigation">
|
||
<h3>Navigation</h3>
|
||
<ul>
|
||
<li class="right" style="margin-right: 10px">
|
||
<a href="../genindex.html" title="General Index"
|
||
>index</a></li>
|
||
<li class="right" >
|
||
<a href="../py-modindex.html" title="Python Module Index"
|
||
>modules</a> |</li>
|
||
<li class="right" >
|
||
<a href="bisect.html" title="bisect — Array bisection algorithm"
|
||
>next</a> |</li>
|
||
<li class="right" >
|
||
<a href="collections.abc.html" title="collections.abc — Abstract Base Classes for Containers"
|
||
>previous</a> |</li>
|
||
<li><img src="../_static/py.png" alt=""
|
||
style="vertical-align: middle; margin-top: -1px"/></li>
|
||
<li><a href="https://www.python.org/">Python</a> »</li>
|
||
<li>
|
||
<span class="language_switcher_placeholder">en</span>
|
||
<span class="version_switcher_placeholder">3.7.4</span>
|
||
<a href="../index.html">Documentation </a> »
|
||
</li>
|
||
|
||
<li class="nav-item nav-item-1"><a href="index.html" >The Python Standard Library</a> »</li>
|
||
<li class="nav-item nav-item-2"><a href="datatypes.html" >Data Types</a> »</li>
|
||
<li class="right">
|
||
|
||
|
||
<div class="inline-search" style="display: none" role="search">
|
||
<form class="inline-search" action="../search.html" method="get">
|
||
<input placeholder="Quick search" type="text" name="q" />
|
||
<input type="submit" value="Go" />
|
||
<input type="hidden" name="check_keywords" value="yes" />
|
||
<input type="hidden" name="area" value="default" />
|
||
</form>
|
||
</div>
|
||
<script type="text/javascript">$('.inline-search').show(0);</script>
|
||
|
|
||
</li>
|
||
|
||
</ul>
|
||
</div>
|
||
<div class="footer">
|
||
© <a href="../copyright.html">Copyright</a> 2001-2019, Python Software Foundation.
|
||
<br />
|
||
The Python Software Foundation is a non-profit corporation.
|
||
<a href="https://www.python.org/psf/donations/">Please donate.</a>
|
||
<br />
|
||
Last updated on Jul 13, 2019.
|
||
<a href="../bugs.html">Found a bug</a>?
|
||
<br />
|
||
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 2.0.1.
|
||
</div>
|
||
|
||
</body>
|
||
</html> |