Files and Modules in Python

Everything a program keeps in variables disappears when it ends. To remember data between runs (a to-do list, settings, scores) or to process data someone else produced, a program reads and writes files.

The built-in open(path, mode) returns a file object. The mode says what you intend to do:

• "r": read (the default); the file must exist.
• "w": write; creates the file, or empties an existing one first.
• "a": append; writes at the end and keeps what is there.

Always open files inside a with statement. It closes the file automatically when the block ends, even if an exception occurs, so data is flushed to disk and the file is released. Also pass encoding="utf-8" for text, so the program behaves the same on every operating system.

f.read() returns the whole file as one string, which is fine for small files. For anything that might be large, loop over the file object itself: it yields one line at a time, so memory use stays small however big the file is.

Each line keeps its newline character at the end. line.rstrip("\n") (or line.strip() when surrounding spaces do not matter) removes it before you process the text. f.readlines() returns all lines as a list when you really need them all at once.

f.write(text) writes exactly the text you give it and returns the number of characters written. It does not add a newline, so include \n yourself. print(..., file=f) is a convenient alternative that adds the newline and the spaces between values for you.

Append mode is the natural choice for logs: every run adds lines and nothing is lost.

CSV (comma-separated values) is the most common format for tables: one record per line, fields separated by commas. Splitting on commas yourself breaks as soon as a field contains a comma, which CSV handles by quoting the field ("Lovelace, Ada"). The standard csv module does the quoting and unquoting correctly.

• csv.writer(f).writerow([...]) writes a row; csv.DictWriter writes dictionaries.
• csv.reader(f) yields each row as a list of strings; csv.DictReader(f) yields dictionaries keyed by the header line.

Open CSV files with newline="" as the csv documentation recommends, and remember that every field you read is a string: convert numbers yourself.

JSON stores nested data (dictionaries, lists, strings, numbers, booleans and null) as text. It is the format of most web APIs and a simple way to save a program's state.

• json.dump(data, f) writes Python data to a file; json.load(f) reads it back.
• json.dumps(data) and json.loads(text) do the same with strings.

Python's True, False and None become true, false and null. Tuples become lists, and dictionary keys always become strings. indent=2 makes the file readable for people.

The pathlib module represents file paths as objects: Path("data") / "report.csv" joins parts with the right separator for the operating system, and methods such as exists(), suffix, stem, read_text() and write_text() cover the common cases in one call.

A relative path such as "notes.txt" is resolved from the current working directory, the folder the program was started from, which is not necessarily the folder the script lives in. When a program cannot find its data file, this is usually why.

Opening a file that does not exist raises FileNotFoundError. Handle it where a sensible fallback exists, for example starting with an empty list the first time a program runs.

A module is a file of Python code that other code can import. Python ships with a large standard library of modules, so many problems are already solved before you install anything.

There are three common import forms:

• import math then math.sqrt(2): the module name keeps it clear where a name comes from.
• from math import sqrt then sqrt(2): shorter, for names you use often.
• import statistics as st: an alias for long module names.

Avoid from module import *: it dumps every name into your file and makes it unclear where anything came from.

Any .py file is a module. As a program grows, move related functions into their own file and import them where they are needed. A typical small project looks like this:

expenses/
    main.py        # menu and input/output
    storage.py     # load and save the data file
    reports.py     # calculations and summaries

# reports.py
def total(expenses):
    return sum(e["amount"] for e in expenses)

# main.py
from reports import total
import storage

expenses = storage.load("expenses.json")
print("Spent so far:", total(expenses))

Each file now has one responsibility, can be read on its own, and can be tested without running the whole program. Python finds reports.py because it sits in the same folder as the script being run.

When a module is imported, Python runs all of its top-level code. That is a problem if the file also contains a demo or a menu loop that should only run when the file is started directly.

Python sets the special variable __name__ to "__main__" for the file you run, and to the module's own name when it is imported. Putting the start-up code under if __name__ == "__main__": makes a file usable both as a program and as an importable module. Wrapping that code in a main() function keeps its variables out of the module's global names.

Beyond the standard library, thousands of packages are published on the Python Package Index (PyPI) and installed with pip: python -m pip install requests. Install them inside a virtual environment so each project has its own versions:

python -m venv .venv
.venv\Scripts\activate      (Windows)
source .venv/bin/activate   (macOS / Linux)
python -m pip install requests

Record what a project needs in a requirements.txt file (python -m pip freeze > requirements.txt) so someone else can recreate the same environment. Everything in this course uses only the standard library, so no installation is needed for the exercises.

Read quoted CSV with the csv module, group a sales export like a data-analysis library would, and parse an HTML page with the standard library's html.parser.