The collections Module

Lists, tuples and dictionaries can do almost anything, but some jobs come up so often that Python ships ready-made containers for them. Counting things, grouping things, running a queue, giving names to the parts of a record and layering settings are all one-liners with the standard collections module.

You saw Counter, defaultdict and deque briefly in the Standard Library Tour. This lesson goes deeper into each of them and adds namedtuple, OrderedDict and ChainMap, so you know exactly when each one beats a plain dictionary.

Counter: counting and comparing

A Counter is a dictionary that maps each item to how many times it appears. Two things make it better than a hand-written counting loop. A missing item gives 0 instead of a KeyError, and you can add and subtract whole counters.

from collections import Counter

monday = Counter(samosa=40, chai=75, jalebi=20)
tuesday = Counter(samosa=35, chai=90, kachori=15)

print(monday + tuesday)
print(tuesday - monday)
print("Dosa sold on Monday:", monday["dosa"])
print("Items sold in two days:", (monday + tuesday).total())

words = "the train to Pune and the bus to Jaipur".split()
print(Counter(words).most_common(2))

Counter({'chai': 165, 'samosa': 75, 'jalebi': 20, 'kachori': 15})
Counter({'chai': 15, 'kachori': 15})
Dosa sold on Monday: 0
Items sold in two days: 275
[('the', 2), ('to', 2)]

Adding counters combines the counts for each item. Subtracting keeps only positive results, so tuesday - monday shows what sold more on Tuesday: 15 extra cups of chai and 15 kachoris that Monday did not sell at all. total() (Python 3.10 and later) adds up every count. You can also call update() to add more items to an existing counter.

defaultdict: no more "if key not in"

A defaultdict takes a factory: a function it calls to create a value whenever you use a key that does not exist yet. The three factories you will use most are int (gives 0), list (gives []) and set (gives an empty set).

from collections import defaultdict

orders = [("Meera", "dosa"), ("Kabir", "idli"), ("Meera", "vada"),
          ("Meera", "dosa"), ("Kabir", "idli")]

plates = defaultdict(int)
dishes = defaultdict(set)
for name, dish in orders:
    plates[name] += 1
    dishes[name].add(dish)

print(dict(plates))
print(sorted(dishes["Meera"]))

print(plates["Farhan"])
print(dict(plates))

{'Meera': 3, 'Kabir': 2}
['dosa', 'vada']
0
{'Meera': 3, 'Kabir': 2, 'Farhan': 0}

plates[name] += 1 works on the very first order because the missing count starts at 0. Using a set keeps each dish only once per person.

The last two lines show the side effect you must know about: simply reading plates["Farhan"] created a new key. If you only want to look something up, use plates.get("Farhan", 0), which never adds a key. With a normal dict, d.setdefault(key, []).append(x) does a similar job, but defaultdict is clearer when every key needs the same kind of starting value.

deque: a fast queue

A queue serves items in the order they arrived, like people at a railway ticket counter. You could use a list with pop(0), but removing the first item of a list makes Python shift every other item along, which gets slow for long lists. A deque (double-ended queue, said "deck") adds and removes at both ends instantly.

from collections import deque

counter = deque(["Aarav", "Priya", "Rohan"])
counter.append("Isha")                # joins at the back
counter.appendleft("Senior citizen")  # priority: front of the line

while counter:
    print("Now serving:", counter.popleft())

turns = deque(["Ananya", "Farhan", "Meera"])
turns.rotate(-1)                      # next player's turn
print(list(turns))

Now serving: Senior citizen
Now serving: Aarav
Now serving: Priya
Now serving: Rohan
Now serving: Isha
['Farhan', 'Meera', 'Ananya']

popleft() takes from the front, and rotate(-1) moves the first item to the back, which is perfect for taking turns in a game. Add maxlen=n when creating a deque to keep only the last n items. Use a deque when you add or remove at the front; for everything else, a list is fine.

namedtuple: tuples with field names

A tuple like ("12951", "Mumbai Rajdhani", 1384) is hard to read: what is item 2? A named tuple lets you use names instead of positions, while staying a light, unchangeable tuple. The modern way to write one is a class that inherits from typing.NamedTuple, with a type hint for each field.

from typing import NamedTuple
from collections import namedtuple

class Train(NamedTuple):
    number: str
    name: str
    distance_km: int

rajdhani = Train("12951", "Mumbai Rajdhani", 1384)
print(rajdhani.name, rajdhani.distance_km)
print(rajdhani[0])

number, name, km = rajdhani        # unpacks like any tuple
print(rajdhani._replace(distance_km=1386))
print(rajdhani._asdict())

Point = namedtuple("Point", ["lat", "lon"])   # older function style
print(Point(26.9, 75.8))

Mumbai Rajdhani 1384
12951
Train(number='12951', name='Mumbai Rajdhani', distance_km=1386)
{'number': '12951', 'name': 'Mumbai Rajdhani', 'distance_km': 1384}
Point(lat=26.9, lon=75.8)

You can read fields by name or by position, and unpack the tuple into variables. Because it is immutable, _replace() returns a new tuple with one field changed. _asdict() gives a dictionary, handy before saving to JSON. The namedtuple() function is the older way that you will see in a lot of existing code; it builds the same kind of class from a list of field names.

If you need to change fields after creating a record, or want default values and methods, use a dataclass instead (see Dataclasses & Enums).

OrderedDict: mostly history now

Before Python 3.7, a normal dictionary did not promise to keep keys in the order you added them, so OrderedDict was the only way to get a dependable order. Today every dict keeps insertion order, so for most code a plain dict is the right choice.

OrderedDict still has two abilities a dict lacks: it can move a key to either end, and its equality check cares about order.

from collections import OrderedDict

recent = OrderedDict(Rohtak=1, Jaipur=2, Kochi=3)
recent.move_to_end("Rohtak")         # just viewed again
print(list(recent))
print(recent.popitem(last=False))    # remove the oldest

a = OrderedDict(x=1, y=2)
b = OrderedDict(y=2, x=1)
print(a == b, dict(a) == dict(b))

['Jaipur', 'Kochi', 'Rohtak']
('Jaipur', 2)
False True

Moving a key to the end and removing from the front is exactly what a "recently viewed" list needs. Two plain dicts with the same items are equal in any order, but two OrderedDict objects are only equal if the order matches too.

ChainMap: layered settings

A ChainMap groups several dictionaries and searches them in order, returning the first match. It does not copy anything, so it is a cheap way to model settings with priorities: values typed on the command line beat values from a config file, which beat the built-in defaults.

from collections import ChainMap

defaults = {"city": "Lucknow", "language": "English", "theme": "light"}
saved = {"theme": "dark"}
command_line = {"city": "Kochi"}

settings = ChainMap(command_line, saved, defaults)
print(settings["city"], settings["theme"], settings["language"])

defaults["language"] = "Hindi"
print(settings["language"])
print(dict(settings))

Kochi dark English
Hindi
{'city': 'Kochi', 'language': 'Hindi', 'theme': 'dark'}

Because the chain only looks into the original dictionaries, changing defaults shows up at once. Writing settings["x"] = 1 changes the first dictionary only.

Which container should you use?

Common mistakes

• Reading a missing key from a defaultdict "just to check". It adds the key. Use in or .get().
• Using list.pop(0) in a loop over a long queue. Each call shifts every item; use deque.popleft().
• Trying to change a named tuple field. train.name = "x" raises AttributeError; use _replace() or a dataclass.
• Reaching for OrderedDict out of habit. Plain dicts keep order already.
• Printing a defaultdict directly in output for users. Its text shows the factory; convert with dict() first.

Key points

• Counter counts items, returns 0 for missing ones, and supports +, -, most_common() and total().
• defaultdict(factory) creates starting values for new keys, but even reading a missing key adds it.
• deque is the right tool for queues: fast append, appendleft, popleft and rotate.
• typing.NamedTuple gives tuples readable field names; namedtuple() is the older form.
• Plain dicts keep insertion order, so OrderedDict is only for move_to_end() and order-aware equality.
• ChainMap searches several dictionaries in order, ideal for layered settings.

from collections import Counter, defaultdict, deque

loans = [("Isha", "science"), ("Rohan", "comics"), ("Isha", "history"),
         ("Meera", "science"), ("Rohan", "comics"), ("Isha", "science")]

genres = Counter(genre for _, genre in loans)
print(genres.most_common(1))

by_student = defaultdict(list)
last_three = deque(maxlen=3)
for student, genre in loans:
    by_student[student].append(genre)
    last_three.append((student, genre))

print(dict(by_student))
print(list(last_three))

[('science', 3)]
{'Isha': ['science', 'history', 'science'], 'Rohan': ['comics', 'comics'], 'Meera': ['science']}
[('Meera', 'science'), ('Rohan', 'comics'), ('Isha', 'science')]