Lesson 11: collections and itertools

collections

Counter

Counts items. most_common(n) gives the top n.

from collections import Counter
c = Counter("mississippi")
print(c)
print(c.most_common(2))
print(c["s"], c["z"])
print(Counter(["a", "b", "a"]) + Counter(["a"]))

Output:

Counter({'i': 4, 's': 4, 'p': 2, 'm': 1})
[('i', 4), ('s', 4)]
4 0
Counter({'a': 3, 'b': 1})

defaultdict

A dictionary that creates a missing value automatically, so you need no get or setdefault.

from collections import defaultdict
by_length = defaultdict(list)
for w in ["a", "bb", "cc", "d", "eee"]:
    by_length[len(w)].append(w)
print(dict(by_length))
count = defaultdict(int)
for ch in "hello":
    count[ch] += 1
print(dict(count))

Output:

{1: ['a', 'd'], 2: ['bb', 'cc'], 3: ['eee']}
{'h': 1, 'e': 1, 'l': 2, 'o': 1}

deque

A list-like queue that is fast at both ends: append, appendleft, pop, popleft, rotate.

from collections import deque
q = deque([1, 2, 3])
q.append(4)
q.appendleft(0)
print(q)
print(q.popleft(), q.pop())
q.rotate(1)
print(q)
print(list(deque(range(5), maxlen=3)))

Output:

deque([0, 1, 2, 3, 4])
0 4
deque([3, 1, 2])
[2, 3, 4]

ChainMap and OrderedDict

ChainMap looks up several dictionaries as one (settings with defaults). A normal dictionary already keeps order, so OrderedDict is rarely needed now.

from collections import ChainMap
defaults = {"colour": "blue", "size": 10}
user = {"size": 14}
settings = ChainMap(user, defaults)
print(settings["colour"], settings["size"])

Output:

blue 14

itertools

chain(a, b, ...)

Walk several iterables one after another.

from itertools import chain
print(list(chain([1, 2], "ab", (9,))))

Output:

[1, 2, 'a', 'b', 9]

product(a, b)

Every combination of one item from each (like nested loops).

from itertools import product
print(list(product("ab", [1, 2])))
print(len(list(product(range(3), repeat=3))))

Output:

[('a', 1), ('a', 2), ('b', 1), ('b', 2)]
27

permutations(x, r) and combinations(x, r)

Orderings of items, and choices of items where order does not matter.

from itertools import combinations, permutations
print(list(permutations("abc", 2)))
print(list(combinations("abc", 2)))
print(len(list(permutations(range(5)))))

Output:

[('a', 'b'), ('a', 'c'), ('b', 'a'), ('b', 'c'), ('c', 'a'), ('c', 'b')]
[('a', 'b'), ('a', 'c'), ('b', 'c')]
120

accumulate(x)

Running totals (or any running result).

from itertools import accumulate
print(list(accumulate([1, 2, 3, 4])))
print(list(accumulate([1, 2, 3, 4], lambda a, b: a * b)))

Output:

[1, 3, 6, 10]
[1, 2, 6, 24]

groupby(iterable, key)

Groups neighbouring items with the same key: sort first if the items are not already together.

from itertools import groupby
words = sorted(["apple", "avocado", "banana", "blueberry", "cherry"])
for letter, group in groupby(words, key=lambda w: w[0]):
    print(letter, list(group))

Output:

a ['apple', 'avocado']
b ['banana', 'blueberry']
c ['cherry']

islice, cycle, repeat, zip_longest

Slice an iterator, repeat a cycle forever, repeat a value, or zip to the longest.

from itertools import cycle, islice, repeat, zip_longest
print(list(islice(range(100), 3, 8)))
print(list(islice(cycle("ab"), 5)))
print(list(repeat("x", 3)))
print(list(zip_longest([1, 2, 3], "a", fillvalue="-")))

Output:

[3, 4, 5, 6, 7]
['a', 'b', 'a', 'b', 'a']
['x', 'x', 'x']
[(1, 'a'), (2, '-'), (3, '-')]

heapq: smallest and largest items

nlargest and nsmallest find the top items without sorting everything.

import heapq
n = [5, 1, 8, 3, 9, 2]
print(heapq.nlargest(2, n))
print(heapq.nsmallest(3, n))

Output:

[9, 8]
[1, 2, 3]

bisect: searching sorted lists

Binary search for a position in a sorted list.

import bisect
grades = [40, 60, 80]
print(bisect.bisect(grades, 65))
marks = [40, 60, 80]
bisect.insort(marks, 70)
print(marks)

Output:

2
[40, 60, 70, 80]

Try it yourself

1. Find the three most common words in a paragraph with Counter.
2. List all the 2-letter passwords you can make from the letters a, b, c with repeats (product).
3. Use deque to simulate a queue of customers.

Final lesson: errors, introspection and type tools.