Associative containers in Python — practice with a play counter, tastes that overlap and a playlist library (and the loop that complained the dictionary was changing)

After fixing my play counter with get, I wanted to tidy up: delete from the dictionary the songs that had been left with 0 plays because I had reset their count by hand. It seemed like the most natural thing in the world:

for song in counts:
    if counts[song] == 0:
        del counts[song]

And Python answered with this:

RuntimeError: dictionary changed size during iteration

“What do you mean it changed size? That’s what I asked it to do.” Precisely. The for goes through the keys of the dictionary one by one, and I was removing keys while it was going through them. Python does not know where to continue and prefers to stop rather than skip songs or read one that no longer exists. The fix is to go through a copy of the keys, for song in list(counts):, or, even better, to build a new dictionary with only the ones you want to keep.

Today we practise associative containers with three complete programs from a music app: a play counter with several questions about the history, a comparison of tastes between two friends with set algebra and a library with users, playlists and songs stored in nested dictionaries. This practice builds on the associative containers theory. If you still hesitate with lists inside other structures, review mutability, copies and matrices first: they show up in almost every line here.

How to use this article

Each program is complete in a single file. Copy it, save it with the name shown and launch it from the terminal with python3 name.py. Below each program you have the real output, a drawing or a table with what happens inside and an explanation in parts. Before looking at the output, try to predict it: with dictionaries, knowing which key exists at each moment is exactly what you need to train.

Program 1 · The play counter

The history is a list of tuples (song, artist), in the order in which they were played. We want to answer three questions: how many times each song has been played, which one is the most played and which songs each artist has.

counter.py

HISTORY = [
    ("Lights of Neon", "Marea Alta"),
    ("Coffee at 3", "Los Turnos"),
    ("Lights of Neon", "Marea Alta"),
    ("Kilometre 0", "Marea Alta"),
    ("Easy Sunday", "Noa Vela"),
    ("Lights of Neon", "Marea Alta"),
    ("Coffee at 3", "Los Turnos"),
]


def count(history):
    """Returns a dictionary song -> times it has been played."""
    counts = {}                             # starts empty
    for song, artist in history:            # each tuple is unpacked into two
        # get(song, 0): the times so far, or 0 if it is the first time it appears
        counts[song] = counts.get(song, 0) + 1
    return counts


def most_played(counts):
    """Returns the (song, times) pair with the most plays."""
    best = None                             # we have not seen any yet
    for song, times in counts.items():
        # best[1] is the times of the best one so far
        if best is None or times > best[1]:
            best = (song, times)
    return best


def by_artist(history):
    """Returns a dictionary artist -> list of their songs, without repeats."""
    result = {}
    for song, artist in history:
        if artist not in result:            # first time the artist appears:
            result[artist] = []             # we create its empty list
        if song not in result[artist]:      # this way the song is not repeated
            result[artist].append(song)
    return result


counts = count(HISTORY)
print(counts)
print("Most played:", most_played(counts))
for artist, songs in by_artist(HISTORY).items():
    print(f"{artist}: {', '.join(songs)}")

Launch it with python3 counter.py:

{'Lights of Neon': 3, 'Coffee at 3': 2, 'Kilometre 0': 1, 'Easy Sunday': 1}
Most played: ('Lights of Neon', 3)
Marea Alta: Lights of Neon, Kilometre 0
Los Turnos: Coffee at 3
Noa Vela: Easy Sunday

The count function, turn by turn

count is the counter pattern from the theory, applied to a real history. The only new thing is that each element is a tuple and the for unpacks it into song and artist (the artist is not used here, but you still have to collect it so the unpacking fits).

Notice a few things in the drawing:

• The first time a song appears, get returns 0 and the line counts[song] = 0 + 1 creates the key. That is why the dictionary grows in turns 1, 2, 4 and 5.
• The other times, get returns what it had and the same line replaces the value. In turns 3, 6 and 7 the dictionary does not grow: only one number changes.
• The order of the keys in the result is the order in which they first appeared: Lights of Neon, Coffee at 3, Kilometre 0, Easy Sunday.

The most_played function: the maximum by hand

def most_played(counts):
    """Returns the (song, times) pair with the most plays."""
    best = None                                   # we have not seen any yet
    for song, times in counts.items():
        # best[1] is the times of the best one so far
        if best is None or times > best[1]:
            best = (song, times)
    return best

Why not use max(counts.values()) directly? Because that gives you the 3, but it does not tell you which song it belongs to. We need to go through the pairs with items() and keep the best pair we have seen:

• best starts at None, which means “I have not seen any yet”.
• The condition best is None or times > best[1] has two parts. The first one is only true on the first turn and makes the first song the provisional best. The second one compares with the times of the best one (best[1], the second element of the tuple).
• The order of the two parts matters: if best is None, the or is already true and Python does not evaluate best[1], which would raise an error.
• With a strict >, in case of a tie the one that appeared first stays.

Turn	song, times	Wins?	best afterwards
1	'Lights of Neon', 3	Yes (best was None)	('Lights of Neon', 3)
2	'Coffee at 3', 2	No (2 > 3 is false)	('Lights of Neon', 3)
3	'Kilometre 0', 1	No	('Lights of Neon', 3)
4	'Easy Sunday', 1	No	('Lights of Neon', 3)

The by_artist function: a dictionary of lists

def by_artist(history):
    """Returns a dictionary artist -> list of their songs, without repeats."""
    result = {}
    for song, artist in history:
        if artist not in result:            # first time the artist appears:
            result[artist] = []             # we create its empty list
        if song not in result[artist]:      # this way the song is not repeated
            result[artist].append(song)
    return result

Here the value of each key is a list that keeps growing. The mistake everyone makes the first time is to write result[artist].append(song) directly: as soon as a new artist appears, that line tries to read result[artist] to do the append, and the KeyError jumps out. That is why there are two if statements:

1. The first one makes sure the artist has their list. If it is the first time they appear, it creates an empty one.
2. The second one avoids repeating songs: Lights of Neon appears three times in the history, but in Marea Alta’s list it only appears once.

At the end, ', '.join(songs) joins the list of titles into a single string separated by commas, which is what gets printed.

Try it yourself

• Add ("Pixel", "Bit Bang") to the history four times. What does most_played return now? Predict it and check it.
• Change > to >= in most_played and make Coffee at 3 tie with Lights of Neon (add one more play of Coffee). Which one wins with each operator, and why?
• Write seconds_by_artist(history, durations) that returns a dictionary artist → total seconds played, using the counter pattern with get(artist, 0) + duration.

Program 2 · What do Ana and I have in common? Tastes with sets

Ana and Luis have each saved their songs. Since each song can only be saved once, the natural choice is to use sets, and then every interesting question is a single operation.

tastes.py

ANA = {"Lights of Neon", "Coffee at 3", "Pixel", "Easy Sunday", "Kilometre 0"}
LUIS = {"Pixel", "Coffee at 3", "Way Back Home", "Kilometre 0", "Below Zero"}
CATALOGUE = ANA | LUIS | {"March", "Signal Lost"}   # everything in the app


def show(title, songs):
    """Shows a set sorted (a set has no fixed order)."""
    print(f"{title} ({len(songs)}): {', '.join(sorted(songs))}")


show("In common", ANA & LUIS)             # intersection
show("Between the two", ANA | LUIS)       # union
show("Only Ana", ANA - LUIS)              # difference
show("Only Luis", LUIS - ANA)             # difference the other way round
show("Not shared", ANA ^ LUIS)            # symmetric difference

# Affinity: which part of everything the two listen to is shared (from 0 to 1)
affinity = len(ANA & LUIS) / len(ANA | LUIS)
print(f"Affinity: {affinity:.0%}")

# What is in the app and neither of them has saved
show("No listeners", CATALOGUE - (ANA | LUIS))

print(ANA <= CATALOGUE, (ANA & LUIS) < ANA)   # inclusion

Launch it with python3 tastes.py:

In common (3): Coffee at 3, Kilometre 0, Pixel
Between the two (7): Below Zero, Coffee at 3, Easy Sunday, Kilometre 0, Lights of Neon, Pixel, Way Back Home
Only Ana (2): Easy Sunday, Lights of Neon
Only Luis (2): Below Zero, Way Back Home
Not shared (4): Below Zero, Easy Sunday, Lights of Neon, Way Back Home
Affinity: 43%
No listeners (2): March, Signal Lost
True True

Before reading the explanation, look at the drawing: each area is the result of one of the lines of the program.

Between the lines:

• CATALOGUE is built with a union of three sets: Ana’s, Luis’s and two that nobody has saved. That way we make sure everything saved is in the catalogue.
• show receives a title and a set. It uses len for the number in brackets and sorted to always print in the same order. Notice that sorted returns a list, and that is why join can join it.
• In common (&): the 3 in the central area of the drawing.
• Between the two (|): 7 different songs. Ana has 5 and Luis has 5, but 5 + 5 = 10 counts the 3 shared ones twice: 10 − 3 = 7.
• Only Ana and Only Luis (-): the side areas. Changing the order of the operands changes the result.
• Not shared (^): the 4 of both sides together. Check the count: 7 between the two = 3 in common + 4 not shared.
• Affinity: 3 in common out of 7 in total gives 0.4285…, and the :.0% format multiplies it by 100, rounds it with no decimals and adds the symbol: 43%.
• No listeners: the brackets are important. CATALOGUE - (ANA | LUIS) first joins what the two have and then removes it from the catalogue.
• Inclusion: ANA <= CATALOGUE is True because everything Ana has is in the catalogue. (ANA & LUIS) < ANA too, because what they share is inside Ana’s songs and Ana also has more.

Try it yourself

• Add "March" to LUIS. Which lines of the output change? Predict all seven before running it.
• What would happen to the affinity if Ana and Luis had exactly the same songs? And if they did not share any? Work it out by hand and then check it.
• Write all_except(sets, excluded) that receives a dictionary user → set and returns the union of everyone’s sets except the excluded one.

Program 3 · The library: users, playlists and songs

To finish, a more realistic structure: a dictionary of users where each user has their playlists, and each playlist is a list of song codes. The information about each song is in another dictionary, SONGS, and the code is the bridge between the two.

library.py

SONGS = {
    "c01": {"title": "Lights of Neon", "artist": "Marea Alta", "seconds": 215},
    "c02": {"title": "Coffee at 3", "artist": "Los Turnos", "seconds": 187},
    "c03": {"title": "Easy Sunday", "artist": "Noa Vela", "seconds": 242},
    "c04": {"title": "Kilometre 0", "artist": "Marea Alta", "seconds": 201},
    "c05": {"title": "Pixel", "artist": "Bit Bang", "seconds": 176},
}

USERS = {
    "ana": {
        "name": "Ana",
        "playlists": {"Study": ["c03", "c02"], "Gym": ["c01", "c05", "c04"]},
    },
    "luis": {
        "name": "Luis",
        "playlists": {"Car": ["c04", "c01"]},
    },
}


def mm_ss(seconds):
    """Converts 592 seconds into '9:52'."""
    return f"{seconds // 60}:{seconds % 60:02d}"


def playlist_duration(user, playlist_name):
    """Total duration of a user's playlist (0 if it does not exist)."""
    playlists = USERS[user]["playlists"]        # we go down two levels
    codes = playlists.get(playlist_name, [])    # third level, with a default value
    total = 0
    for code in codes:
        total += SONGS["seconds"]         # from the code, to its duration
    return total


def artists_of(user):
    """Set of artists that appear in the user's playlists."""
    artists = set()                             # set(), not {}
    for codes in USERS[user]["playlists"].values():   # each list of codes
        for code in codes:
            artists.add(SONGS["artist"])        # the set does not repeat
    return artists


def who_has(code):
    """Names of the users with that song in some playlist."""
    names = []
    for data in USERS.values():                 # we have no key: we have to go through
        for codes in data["playlists"].values():
            if code in codes:
                names.append(data["name"])
                break                           # one playlist is enough
    return names


print(USERS["ana"]["playlists"]["Gym"])
print("Ana's Gym:", mm_ss(playlist_duration("ana", "Gym")))
print("Ana's Party:", mm_ss(playlist_duration("ana", "Party")))
print("Ana's artists:", sorted(artists_of("ana")))
print("Who has c04:", who_has("c04"))
print("Who has c03:", who_has("c03"))

Launch it with python3 library.py:

['c01', 'c05', 'c04']
Ana's Gym: 9:52
Ana's Party: 0:00
Ana's artists: ['Bit Bang', 'Los Turnos', 'Marea Alta', 'Noa Vela']
Who has c04: ['Ana', 'Luis']
Who has c03: ['Ana']

How the structure is put together

The first line of the output is the best way to understand it. USERS["ana"]["playlists"]["Gym"] goes down three levels:

Expression	Type	Value
USERS["ana"]	dict	{"name": "Ana", "playlists": {…}}
USERS["ana"]["playlists"]	dict	{"Study": […], "Gym": […]}
USERS["ana"]["playlists"]["Gym"]	list	['c01', 'c05', 'c04']
SONGS["c05"]	dict	{"title": "Pixel", "artist": "Bit Bang", "seconds": 176}
SONGS["c05"]["seconds"]	int	176

playlist_duration: three levels and a default value

def playlist_duration(user, playlist_name):
    """Total duration of a user's playlist (0 if it does not exist)."""
    playlists = USERS[user]["playlists"]        # we go down two levels
    codes = playlists.get(playlist_name, [])    # third level, with a default value
    total = 0
    for code in codes:
        total += SONGS["seconds"]         # from the code, to its duration
    return total

• Line 28 goes down two levels with normal brackets: here we assume the user exists, and if they did not we would rather get the KeyError so we find out.
• Line 29 goes down the third level with get(playlist_name, []). If the playlist does not exist, it returns an empty list: the loop does not run at all and the duration is 0. That is why "Ana's Party" comes out as 0:00 instead of an error.
• Inside the loop, each code becomes its duration by jumping to the other dictionary: SONGS["seconds"].

With the Gym playlist, the total goes like this: 215 (c01) + 176 (c05) + 201 (c04) = 592 seconds. mm_ss converts it: 592 // 60 is 9 minutes and 592 % 60 is 52 seconds. The :02d format pads with a zero on the left when needed, so that 0 seconds comes out as 0:00 and not 0:0.

artists_of: going through two levels without repeats

def artists_of(user):
    """Set of artists that appear in the user's playlists."""
    artists = set()                                # set(), not {}
    for codes in USERS[user]["playlists"].values():   # each list of codes
        for code in codes:
            artists.add(SONGS["artist"])        # the set does not repeat
    return artists

• USERS[user]["playlists"].values() gives the lists of codes of each playlist (we are not interested in the names of the playlists, that is why we use values() and not items()).
• The second for goes through the codes of each list, and each code becomes an artist.
• We use a set because Marea Alta appears twice (c01 and c04) and we want it only once. With a set, the repeated add does nothing and no if is needed.
• It is created with set(), not with {}: the empty braces would be a dictionary and the add would fail.

who_has: when you do not have the key

def who_has(code):
    """Names of the users with that song in some playlist."""
    names = []
    for data in USERS.values():                 # we have no key: we have to go through
        for codes in data["playlists"].values():
            if code in codes:
                names.append(data["name"])
                break                           # one playlist is enough
    return names

This function is the opposite of the previous ones: we have a code and we want to know which users have it. The codes are not keys of USERS, they are hidden inside the lists, so there is no shortcut: we have to go through all the users and all their playlists.

• The break stops the playlists loop as soon as it finds the code. Without it, if Ana had c04 in two playlists, her name would appear twice.
• With c04 we get Ana (in Gym) and Luis (in Car). With c03, only Ana (in Study).

Try it yourself

• Call playlist_duration("pablo", "Gym"). Which error comes out and on which line? How would you avoid it with get on the first level?
• Write titles(user, playlist_name) that returns the list of titles (not codes) of a playlist.
• Write invert(users) that builds the dictionary code → set of names from the previous note, and check that with it who_has is answered in one line.

Watch it step by step in Python Tutor

Python Tutor runs the code line by line and draws each dictionary as a table of keys and values. Choose Python 3 and paste this counter, which stores the count in an intermediate variable so you can see it:

history = ["Pixel", "March", "Pixel"]
counts = {}
for song in history:
    current = counts.get(song, 0)
    counts[song] = current + 1
print(counts)

Expected output:

{'Pixel': 2, 'March': 1}

What to watch, step by step:

1. Before the loop, counts appears as an empty dictionary: a box with no rows.
2. On the first turn, song is "Pixel" and current is 0: get did not find the key and returned the default value. Notice that counts is still empty after the get, because get only reads.
3. On the next line the first row of the dictionary appears: "Pixel" → 1.
4. On the second turn the same happens with "March". On the third, "Pixel" again: now current is 1, and Pixel's row changes its value to 2 instead of creating a new row.
5. When the loop ends, the dictionary has two rows in the order in which the songs first appeared.

To finish

Today you have used associative containers in three different situations: a counter dictionary and a dictionary of lists to summarise a history, sets to compare tastes with a single operation and nested dictionaries with codes that act as a bridge between two structures. If something did not click, go back to the theory; and when you feel comfortable, move on to the exercises, where you will build a small recommender and catch three very typical bugs.

Next step: exercisesAssociative containers in Python — exercises with durations, music genres and a song recommender (and the empty braces that were not a set)

Did this article help you? Here is how you can support SergioLearns →