Modules and packages in Python — theory with grade utilities, import and __main__ (and the program that printed things I never asked for)

I had written a grades.py file with my functions and, at the end, a few lines to test them while I was writing it. All good. Days later, in another file, I wrote import grades to reuse the average function, ran my new program and this is what came out:

Testing average...
7.0
My program starts here
7.0

“I only asked for the average of 5, 7 and 9… where does that «Testing average…» line come from, before anything starts?” I spent a while staring at my new program, which had no trace of those lines. What happens is that import runs the whole file, and I had left my tests lying around at the end of the module.

Today we put everything in order. You will see where it makes sense to write your functions, what a module is, how to use it with import and with from … import, why the prefix exists (namespaces), what if __name__ == "__main__": really means and how to group several modules into a package. All with one running example: a few utilities for working with grades. If you are coming from functions in Python, this is the natural next step: you already know how to write functions; now you will learn where to keep them so they can be reused. And when you finish, the modules and packages practice is waiting with three complete programs.

Where do I write my functions?

So far you have written your functions in the same file as the program that uses them. For small programs that is normal and works perfectly:

GRADE_MAX = 10
PASSING = 5.0


def average(a, b, c):
    """Average of three grades."""
    return (a + b + c) / 3


def is_passed(grade):
    return grade >= PASSING


m = average(6.0, 7.5, 7.5)
print("Average:", m)
print("Passed?", is_passed(m))

Average: 7.0
Passed? True

The problem comes when the program grows. Imagine that tomorrow you need average in a different program: you would have to copy and paste the function. And if you later fix a bug in one copy, the other still has it. Copying code is the fastest way to have the same bug in two places.

Python gives you another option: write the functions in another file and ask your program to use them. That is modular programming: splitting a program into independent parts, each with its own functionality, that you can reuse and swap. In Python, those parts are called modules.

A module is a .py file

A module is simply a .py file, and its name is the file name without the extension. If we save the functions and constants in grades.py…

grades.py

GRADE_MAX = 10
PASSING = 5.0


def average(a, b, c):
    """Average of three grades."""
    return (a + b + c) / 3


def is_passed(grade):
    return grade >= PASSING

…any other file in the same folder can use them with import:

program.py

import grades

m = grades.average(6.0, 7.5, 7.5)
print("Average:", m)
print("Passed?", grades.is_passed(m))
print("Maximum possible grade:", grades.GRADE_MAX)

Average: 7.0
Passed? True
Maximum possible grade: 10

Between the lines:

• import grades looks for a file called grades.py and loads it. We write grades, without .py.
• To use something from the module you write it with the prefix: grades.average(...), grades.GRADE_MAX. It works for functions and also for variables and constants.
• program.py does not contain a single line of average, and yet it uses it. If tomorrow you improve average in grades.py, every program that imports it benefits.

Traceback (most recent call last):
  File "prefix_error.py", line 3, in <module>
    m = average(6.0, 7.5, 7.5)
        ^^^^^^^
NameError: name 'average' is not defined

Namespaces: why the prefix is needed

Why won’t it let me use plain average? Because every module is a namespace: a box with its own names. What is defined inside grades.py lives in the grades box, which is why you need to say which box you are taking it from.

This lets two modules have things with the same name without bothering each other. Imagine that, besides grades.py, we have attendance.py, which also has a function called average, but to compute the average attendance as a percentage:

grades.py

def average(a, b, c):
    """Average of three grades."""
    return (a + b + c) / 3

attendance.py

def average(p1, p2):
    """Average attendance (in %) over two terms, rounded."""
    return round((p1 + p2) / 2)

Both are called average, but there is no conflict because they live in different boxes:

import grades
import attendance

print("Average grade:", grades.average(6.0, 7.5, 7.5))
print("Average attendance:", attendance.average(100, 70), "%")

Average grade: 7.0
Average attendance: 85 %

grades.average and attendance.average are different functions that happen to share a name. Thanks to the prefix, you always know which one you are calling.

from … import and aliases

Writing grades. in front every time can be tedious. Python offers two shortcuts:

from grades import average, is_passed    # bare names, no prefix
import grades as gr                        # short alias for the module
from grades import GRADE_MAX as MAXIMUM      # alias for a single name

m = average(6.0, 7.5, 7.5)
print("Average:", m)
print("Passed?", is_passed(m))
print("Maximum grade:", gr.GRADE_MAX, MAXIMUM)
print("Same function?", average is gr.average)

Average: 7.0
Passed? True
Maximum grade: 10 10
Same function? True

Form	How you use it afterwards	When it fits
import grades	grades.average(...)	The default: you can see where everything comes from
from grades import average	average(...)	When you use the module a little and no name will repeat
import grades as gr	gr.average(...)	Modules with a long name
from grades import average as grades_average	grades_average(...)	Two modules with a repeated name

Look at the last line of the output: average is gr.average gives True. With from … import you do not copy the function: you put another label on the same function. Aliases work the same way.

overwritten.py

from grades import average
from attendance import average     # overwrites the previous one!

print(average(6.0, 7.5, 7.5))

Traceback (most recent call last):
  File "overwritten.py", line 4, in <module>
    print(average(6.0, 7.5, 7.5))
          ^^^^^^^^^^^^^^^^^^^^^^
TypeError: average() takes 2 positional arguments but 3 were given

Here average is no longer the one from grades, but the one from attendance (the second one imported), which takes two parameters instead of three. And the worst part: if both had taken the same number of parameters, there would be no error: you would simply get wrong results.

The main module and stray code

Back to the mystery at the beginning. When Python imports a module, it runs the file from top to bottom, the first time it is imported. Defining functions and constants is harmless, but any loose statement (a print, a test, a call) also runs. The grades.py from that story looked like this:

grades.py (with loose tests)

GRADE_MAX = 10


def average(a, b, c):
    return (a + b + c) / 3


# Quick test while I was writing the module
print("Testing average...")
print(average(6.0, 7.5, 7.5))

What we need is for those tests to run when **I launch grades.py directly**, but not when another program imports it. Python allows this because every module has a special variable, __name__:

• If you launch the file directly (python3 grades.py), Python sets __name__ to "__main__": it is the main module, the one that starts everything.
• If the file is loaded with import, __name__ is the module’s name ("grades").

So it is enough to put the tests inside if __name__ == "__main__":. To see it, the module below also prints __name__ in each case:

grades.py (fixed)

GRADE_MAX = 10

print("__name__ is:", __name__)


def average(a, b, c):
    return (a + b + c) / 3


if __name__ == "__main__":
    # Only runs if you launch THIS file directly
    print("Testing average...")
    print(average(6.0, 7.5, 7.5))

If we import it from program.py (the same one as before), the tests no longer sneak in:

__name__ is: grades
My program starts here
7.0

And if we launch grades.py directly, the tests do run:

__name__ is: __main__
Testing average...
7.0

How it runs	Value of __name__	Does it enter the if?
python3 grades.py	"__main__"	Yes
import grades from another file	"grades"	No

Packages: folders of modules

When you have many modules, it is convenient to group them. A package is a folder with modules inside and a special file called __init__.py (it can be empty). That is what tells Python: “this folder is a package”. Packages can in turn contain other packages, the subpackages.

Our utilities grow and we make a university package, with a scholarships subpackage:

program.py
university/
├── __init__.py
├── grades.py
├── attendance.py
└── scholarships/
    ├── __init__.py
    └── calculation.py

The dot in the import reads like a folder slash: university.scholarships.calculation is the file university/scholarships/calculation.py. These are the new modules (grades.py and attendance.py are the same as before):

university/scholarships/calculation.py

from university.grades import PASSING


def score(grade, attendance):
    """Scholarship score: 80 % grade and 20 % attendance (out of 10)."""
    if grade < PASSING:
        return 0
    return round(0.8 * grade + 0.2 * attendance / 10, 2)

Notice that calculation.py imports something from another module in the package using the full path: from university.grades import PASSING. And this is the program that uses it all:

program.py

from university.grades import average
import university.attendance
from university.scholarships.calculation import score

m = average(6.0, 7.5, 7.5)
a = university.attendance.average(100, 70)
print("Average:", m, "| Attendance:", a, "%")
print("Scholarship score:", score(m, a))

Average: 7.0 | Attendance: 85 %
Scholarship score: 7.3

Look at the three ways of importing:

• from university.grades import average: brings the bare function; we use it without a prefix.
• import university.attendance: brings the module; you must write the full path (university.attendance.average) every time.
• from university.scholarships.calculation import score: goes down two levels (package, subpackage, module) and takes a single function.

Traceback (most recent call last):
  File "path_error.py", line 1, in <module>
    import university.grade
ModuleNotFoundError: No module named 'university.grade'

Traceback (most recent call last):
  File "university/scholarships/calculation.py", line 1, in <module>
    from university.grades import PASSING
ModuleNotFoundError: No module named 'university'

Watch it step by step in Python Tutor

Python Tutor runs your program line by line and draws the variables and functions in memory. Choose Python 3. It has one limitation: it only accepts one file, so it cannot import your own modules. But we can simulate the important part: that a name is a label pointing to a function, and that from … import only changes which function it points to.

def average(a, b, c):
    return (a + b + c) / 3


def attendance_average(p1, p2):
    return round((p1 + p2) / 2)


grades_average = average           # alias: another label for the same function
print(grades_average(6.0, 7.5, 7.5))

average = attendance_average      # now "average" points to the other function
print(average(100, 70))
print(grades_average(6.0, 7.5, 7.5))   # the alias still points to the original

if __name__ == "__main__":
    print("I am the main program")

Expected output:

7.0
85
7.0
I am the main program

What to watch, step by step:

1. After defining the two functions, the global variables area shows average and attendance_average, each with an arrow to its own function box.
2. In grades_average = average no new function is created: a second arrow appears to the same box. That is what an alias or a from … import … as … does.
3. In average = attendance_average, the arrow of average jumps to the other box. That is what the second from … import average of the earlier error does: the name stays with the last one.
4. The last call, through grades_average, still gives 7.0: the alias kept its original arrow. That is why aliases are the way out of name conflicts.
5. In the final block, Python Tutor runs your code as the main program, so the condition __name__ == "__main__" is met and you enter the if.

Summary

Concept	The essentials
Module	A .py file; its name is the file name without the extension
import module	Used with a prefix: module.function()
from module import f	Used without a prefix; careful with repeated names
Alias (as)	Another label for the module or the function
Namespace	Each module is a box: a.average and b.average do not clash
if __name__ == "__main__":	Code that only runs when the file is launched directly
Package	Folder with modules and an __init__.py; imported with dots

Three ideas to take with you:

1. Importing runs the file. Leave only definitions at the top level and everything else under if __name__ == "__main__":.
2. The prefix is an advantage, not a nuisance. It says where each thing comes from and prevents name clashes. If you drop it with from, do it knowing which names come in.
3. A dot is a folder. university.scholarships.calculation is university/scholarships/calculation.py, and it always runs from outside the package.

In the modules and packages practice you will build a report card with your own module, a module that tests itself and a package with a subpackage. After that, the exercises will tell you whether it has stuck. And in the next topic we get into sequences: lists and tuples.

Next step: practiceModules and packages in Python — practice with a report card, a self-testing module and a package (and the assert that brought my program down on import)

Did this article help you? Here is how you can support SergioLearns →