In real life, as well as programming, there are some impossible operations. Can you divide seven by yellow? Can you set fire to a sound? These don’t make sense. The same is true in programming.

Throughout this sprint we are going to look at some code which you can find in the types directory of the Module-Tools repository.

In that file, is half("22") hoping to return 11 (because the string should be converted to a number)? Or return 2 (because it’s the first half of the string)? Or error, because it doesn’t make sense?

What if we tried to run half("hello")? Try to give part of a word, or error because it can’t be split evenly in half? Does this input even make sense? What if we did double("hello") instead? What do you expect it to return?

How about second(22)? Should it treat 22 like a stringified version of the decimal representation of the number 22 and return 2? If so - 22 is the same as 0x16. Should second(0x16) convert 0x16 to decimal before returning the second character? Or should it remember that the original number was input as hexadecimal and return 6?

Intent

The intent of these functions is probably that half and double are expected to operate on numbers, and second is expected to operate on strings (and/or maybe lists). We don’t know for sure what the author intended just by looking at the function names.

But Python lets us write all of these things. Some of them, like half("hello") will error when they run, maybe breaking our program. Others, like double("22") will succeed but in surprising ways which may cause our program to give more subtly incorrect results later on.

In such a simple program as in 01-predict.py, it’s easy for us to run the program manually and see the errors (if we add enough logging). But as programs get bigger, these things get harder to spot, especially if there are branches and code only executes sometimes.

How many errors did you find in your testing? There is one big bug here which doesn’t always show. response.body is a stream not a string. So if a user ever tries to fetch a URL which returns a non-200 status code, our program will crash:

% node 02-playcomputer.js
What URL should we fetch?
> http://www.google.com/beepboop
file:///Users/dwh/tmp/jsplay/fetch.js:12
    if (response.body.toLowerCase().includes("permission")) {
                      ^

TypeError: response.body.toLowerCase is not a function
    at file:///Users/dwh/tmp/jsplay/fetch.js:12:23
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)

Node.js v22.11.0

How easy was it to spot this bug in your testing?

The code in this file was wrong. It could never have been correct. After a fetch, response.body.toLowerCase() never makes sense. Ideally we shouldn’t have needed to wait until running the code, and using that exact input, to find this out.

Types

This is where types come in.

Imagine if we could analyse our code and find out “You’re calling double with a string, but double expects a number, you have a bug”. Or that “You’re calling response.body.toLowerCase() but response.body is a ReadableStream which doesn’t have a method toLowerCase, you have a bug”.

We wouldn’t need to keep executing our program with lots of different inputs every time we change it. The type analysis could tell us “You have a bug here, you should fix it”. Without having to run the program, and without having to think about different possible inputs.

Limits of type checking

Types can be really useful for detecting bugs. But there are limits to what kind of bugs type checking can detect.

Type checking can’t catch this type of bug - as long as you give it a number as input, it gives you a number as output. All of the types are correct. Not all bugs are type errors. But checking for type errors can get rid of a lot of them.

Support for type checking

Different languages have different levels of support for checking types.

Some languages, like Java, C++, Rust, and Go, require you to write what types you expect function parameters to have.

Other languages, like JavaScript and Python, don’t require this but they have tools which allow you to add this information by using a tool like mypy or JSDoc.

Some very low level machine languages like assembly don’t have any typing at all.

Languages with optional type checking perform good checks when you add this type information through annotations. Annotations are a way of hinting at a type, for example that a variable contains a certain type, or that a function returns a certain type.

For example, imagine we had a variable price and we want to annotate its type. It could be a formatted string like "£3.50". A float like 3.5. If we annotated it as price : int it would be good hint that it should only contains whole numbers, so lets the programmer and the language know the code expects a value like 350.

If you don’t add type annotations in your code, the language will perform fewer checks. Sometimes they will infer the correct types based on what you have annotated. Other times they will just ignore code with no annotations and not give you errors about it even if it’s wrong.

Trying out Mypy

Mypy is a tool which enables type checking in Python code. It works best if you can build up a habit of integrating it into your workflow.

We’ve already seen that objects can group together related, named data. We can write:

imran = {
  "name": "Imran",
  "age": 22,
  "preferred_operating_system": "Ubuntu",
}

eliza = {
  "name": "Eliza",
  "age": 34,
  "preferred_operating_system": "Arch Linux",
}

This allows us to pass around the values of imran or eliza, and access all of the related information while we do.

We now know that typing can tell us if we make errors like calling .lower() on the numeric value 2.

It would be useful for a type checker to tell us if we try to access a property of an object that that object doesn’t have. Can mypy help us here?

The code in the above exercise doesn’t work, but mypy can’t tell us this. Remember how we said that type checking has its limits? As far as mypy is concerned, a dictionary is a dictionary - it could contain any keys!

Instead, we can use a class🧶🧶 classA class is a template for an object. It lets us say what properties (and methods) all instances of that class will contain. .

class Person:
    def __init__(self, name: str, age: int, preferred_operating_system: str):
        self.name = name
        self.age = age
        self.preferred_operating_system = preferred_operating_system

imran = Person("Imran", 22, "Ubuntu")
print(imran.name)
print(imran.address)

eliza = Person("Eliza", 34, "Arch Linux")
print(eliza.name)
print(eliza.address)

This code is saying: “There’s a category of object called Person. Every instance of Person has a name, an age, and a preferred_operating_system”. It then makes two instances of Person, and uses them.

The method called __init__ is called a constructor - it is what is called when we construct a new instance of the class.

You can use the names of classes in type annotations just like you can use types like str or int:

def is_adult(person: Person) -> bool:
    return person.age >= 18

print(is_adult(imran))

We’ve seen that we can take instances of classes as function parameters:

def is_adult(person: Person) -> bool:
    return person.age >= 18

We’ve also seen types that have methods on them, e.g. "abc".upper(). This looks a bit different from functions we define ourselves, e.g. upper("abc").

Methods are just like functions, but they are attached to a class.

We could rewrite our is_adult function as a method on Person:

class Person:
    def __init__(self, name: str, age: int, preferred_operating_system: str):
        self.name = name
        self.age = age
        self.preferred_operating_system = preferred_operating_system

    def is_adult(self):
        return self.age >= 18

imran = Person("Imran", 22, "Ubuntu")
print(imran.is_adult()) # True

This has a few advantages over free functions🧶🧶 Free functionA free function is a function that isn’t a method. It isn’t bound to a particular type (but may take parameters). .

Consider this free function called drivers_license_check which uses the Person class method is_adult outside of the class:

def drivers_license_check(person: Person):
  if person.is_adult():
    return 'Valid drivers license'

  return 'This person is underage!'

print(drivers_license_check(imran)) # returns 'Valid drivers license'

An advantage of classes over objects is encapsulation.

Imagine you have written your Person class that stores age information.

For privacy reasons, you don’t want to reveal the exact age of the person, only whether they are or are not over 18 years old.

This means we need to store the “age” value in a class, but somehow keep it private to that class. The only public information we want is whether or not they are over 18. How can we achieve this?

Look at the following code:

class Person:
    def __init__(self, name: str, age: int):
        self.name = name
        self.__age = age
        
    def is_adult(self):
      return self.__age >= 18

imran = Person("Imran", 22)
print(imran.name)
# print(imran.age) # fails
# print(imran.__age) # fails
print(imran.is_adult()) # works and prints True

eliza = Person("Eliza", 12)
print(eliza.name)
# print(eliza.age) # fails
# print(imran.__age) # fails
print(eliza.is_adult()) # works and prints False

In python, any field that begins with two underscores is considered private, i.e. it can only be used within that specific class instance.

You can now program classes to change behaviour based on the information stored within them. Compare this with objects, which can only ever store data, and behave the same every time.

Another benefit of encapsulation is letting you make “read only” properties. Think about the example above. Imagine you wanted to check if a Person class had a certain name using an equality test, but accidentally used a single = symbol:

imran.name = "Eliza"

Python allows you to update public fields whenever you want. If name were private, and the only way to access it was through a get_name() method that returns a string, it would be impossible to accidentally change the value. In this way, encapsulation can be used to prevent accidental errors in code.

Why encapsulate?

In your career you will rarely be building code used only once. It is likely the code you write will sit alongside code written by others as part of a large long-lived codebase. Classes and encapsulation are really important techniques as you move towards thinking about how others will use your code, and how you plan to make your code maintainable and reusable for future use.

Classes with encapsulation clearly define the outward-facing interface of what you are building. Think about the documentation you may have read for well-defined APIs like fetch or argparse. You don’t need to know how they work internally to make use of them, and the methods and their parameters are clearly stated. If argparse is updated, e.g. to make it more efficient, your code won’t break as the public interface won’t change. If fetch is changed, e.g. adding a new parameter, type checking will immediately highlight everywhere you need to update your code.

Encapsulation also makes it easy to swap different implementations. Imagine you started a big project with a python dict but later on needed to change it to an OrderedDict. The interfaces are almost exactly the same, so you wouldn’t need to change any of the method invocations, making the change much easier and safer.

Encapsulation also helps with testing. Only the public interface, methods and properties, need to be tested. You can write the test before you start using test-driven development, defining the public interface and behaviour. Then you can focus on the implementation inside, and when the test passes you know your class works. Testing a single class with a well defined interface is much easier than needing to test lots of interconnected separate free functions.

Until now you have been solving small coding challenges with the aim of solving the specific task. From now on you will start to think more about how you can build a solution that will adapt well to future changes. Well defined classes that encapsulate your implementations will be a big help.

We’ve seen that grouping together fields and methods into a class can help us encapsulate them. We can define a class whose purpose is just to group together related data, and provide access to it.

Our Person class is an example of this. We just store some data in it (and maybe add some methods that just read that data).

If a class is just a place to group related data, it is sometimes called a value object🧶🧶 Value objectA value object is an object which exists just to store data. They are normally immutable (never change). . We normally consider two value objects to be equal to each other if their fields contain the same values.

There are several methods we can implement on classes that have obvious implementations for value objects.

Equality is one: ideally two value objects are the same if their fields are the same. But this is not the case with objects by default:

class Person:
    def __init__(self, name: str, age: int, preferred_operating_system: str):
        self.name = name
        self.age = age
        self.preferred_operating_system = preferred_operating_system

imran = Person("Imran", 22, "Ubuntu")
imran2 = Person("Imran", 22, "Ubuntu")
print(imran == imran2)  # Prints False

Similarly, it’s useful when we print a value object to see its type and properties. But this is not the case with objects by default:

class Person:
    def __init__(self, name: str, age: int, preferred_operating_system: str):
        self.name = name
        self.age = age
        self.preferred_operating_system = preferred_operating_system

imran = Person("Imran", 22, "Ubuntu")
print(imran)  # Prints <__main__.Person object at 0x1048b5a90>

Python has a useful decorator🧶🧶 DecoratorA decorator is an annotation you can add to some Python code to give it extra behaviour. called dataclass which generates some of these methods for us. In fact, it even generates the constructor for us.

from dataclasses import dataclass

@dataclass(frozen=True)
class Animal:
    name: str
    species: str
    age: int
    noise: str

indigo = Animal("indigo", "cat", 2, "meow")  # We can call this constructor - @dataclass generated it for us.
print(indigo)  # Prints Animal(name='Indigo', species='cat', age=2, noise='meow')

indigo2 = Animal("indigo", "cat", 2, "meow")
print(indigo == indigo2)  # Prints True

The dataclass decorator generated a constructor, a __str__ method (which is called when string formatting the value), and a custom __eq__ method (which is called when comparing two values). This saves us having to write all of that code.

Other languages have a similar idea of a value type, and tools to help make them, such as Java’s record classes and C#’s’ structure types.

Limitations of Type Annotations

Sometimes we want to reason about more complicated type relationships than “this field is a string”. Lists and dictionaries are examples of this. We may want to reason that every value in a list is a certain type. We could model a family as being a list of members all of type Person, for instance.

In some languages, like Java, C#, Rust, or Go, type information is required - you can’t write code without it. This means those languages can do more checks, and give better error messages. We call these statically typed languages🧶🧶 Static typingA statically typed language is a language where every variable has a fixed type. It is an error to try to assign a value to a variable with a different type.

In other languages, like Python and JavaScript, type information is optional. Because of this, tools that check types are sometimes less strict. If they don’t know what type something has, they stop doing any checks.

That’s what’s happening in task 11. FamilyTree.members is a list, but mypy doesn’t know what type of thing is in the list. It doesn’t even know that everything in the list has the same type = ["hello", 7, True] is a legal list in Python. Many people would consider a pet to be a member of the family, so it seems correct, but due to the different types, this code breaks down and mypy can’t spot the problem.

Using Generics

We can use generics🧶🧶 Generic typesA list could store numbers, or strings. We use generic types to say which type a particular instance of a list stores. Even though we can have a list of strings, and a list of numbers, the code for finding the first element is the same. But knowing that a list only contains strings is useful. to tell mypy what type of thing is in the list. We could add an import and modify the FamilyTree class from task 11 as follows:

@dataclass(frozen=True)
class FamilyTree:
    parent: Person
    members: list[Person]

Try updating your code for task 11 with this change and see if mypy spots the problem

Now that we’ve told mypy FamilyTree.members is a list of type Person, it can identify that the child variable printed out must be of type Person. Because of this, it can tell us that child.age on doesn’t exist when the pet is accidentally included in the list.

Generic Functions

You have seen how we can use generics in type annotations. But what if we wanted to make the code we write support generics as well?

Think about a common task you may have done, getting the last element from a list. You can do this using indexing: [-1]. It should be pretty simple to turn this into a free function, but how do we annotate the types? This function needs to work with any typed list, not just lists of integers or strings. It would be too much work to write a separate function for every possible type of list.

Instead, we write a generic function. To make a generic function we pick a symbol, like the letter T, to represent a given type. After the function name, we add type parameters🧶🧶 Type ParametersJust like you can define parameters for the values passed to a function, you can define parameters that describe the generic types used within a function. These come before the main parameters and use square brackets: [T]. Some languages use angle brackets instead: <T>. the function using [T]. From then on, everywhere in the function that has a T becomes the given type.

Here is an example of a generic function:

def get_last[T](my_list: list[T]) -> T:
  return -1

get_last([0,1,2,3,4,5])
get_last(["a","b","c"])

This function contains a bug. Mypy can find the error that this function always returns an integer, rather than returning the same type as is held in the list. In this way we can write a generic free function that can be used with any type, and which works well with mypy.

Writing a Generic Class

We’re starting to move beyond writing free functions, though. How does this work with classes?

The kind of relationship structure we created with families and members, a tree🧶🧶 TreesA list can store a linear array of values. A tree stores values in a heirarchy, like a family tree. , is common across many types of data, for example how species of animal are related to each other, or how a dictionary might store words.

Thinking about keeping our code reusable, is there a way we could define such structures, and be able to force them to work with certain types, without needing to write a special class for each individual data type? Just like lists can take a generic to force them to be a certain type, we can write classes that accept generics.

Look at the following code:

from dataclasses import dataclass

@dataclass(frozen=True)
class Animal:
    name: str
    size: str

@dataclass(frozen=True)
class Person:
    name: str
    age: int

@dataclass(frozen=True)
class Tree[T]:
    parent: T
    children: list[T]

    def print_tree(self):
        print(self.parent)
        for child in self.children:
            print(child)

fatma = Person(name="Fatma", age=4)
aisha = Person(name="Aisha", age=6)
imran = Person(name="Imran", age=30)
family_tree = Tree[Person](parent=imran, children=[fatma, aisha])

cats = Animal(name="Cat", size="Small")
dogs = Animal(name="Dog", size="Medium")
mammals = Animal(name="Mammals", size="Variable")
species_tree = Tree[Animal](parent=mammals, children=[cats, dogs])

family_tree.print_tree()
species_tree.print_tree()

The Tree here has the type paramter T. Just like with a generic function, this tells python that every reference to T within the class becomes a given type.

See we then create two different trees: a Tree[Person] and a Tree[Animal]. In these trees, the parent and list of children must contain Person and Animal types respectively.

It also means instead of having to create a new method to print out every single tree type, we can create a single method - Tree.print_tree().

Imran (30 years old)
- Fatma (4 years old)
- Aisha (6 years old)
Mammals (Variable size)
- Cat (Small size)
- Dog (Medium size)

Using classes and objects can help us to understand and maintain codebases, particularly as they grow. The process of taking some old code, and updating it in a maintainable way is called “refactoring”.

We previously saw that using methods instead of free functions can help us to encapsulate information. But changing functions into methods, and modifying classes can be tricky, as it is easy to forget places where code needs to be updated during refactoring.

Type checking can help us with this. If you have some code which accesses imran.age, and we remove the age field, we can run mypy: It will tell us “Here are all of the places that reference age you also need to change your code”.

Look at file 13-refactor.py as an example. It is a program that works out what laptops could be allocated to what people based on their preferred operating system.

Let’s imagine we want to change our code. We don’t want to say “Every person has one preferred operating system” any more. We want to let people have a list of operating systems they prefer (in order). So we could say “Imran prefers Ubuntu most of all, and then Arch Linux, but will not use macOS”.

The bigger (and more complicated) our codebase is, the more useful it is that mypy tells us what code needs changing. This is even more useful when we start working with code we didn’t write ourselves, or we wrote long ago. Instead of needing to read all of the code and search around to try to work out where we need to change an age to date_of_birth, or how to access a single variable that has become a list of many, mypy can tell us “here are all of the places that are wrong”.

In the laptops example, we were using strings to store operating systems. Using strings is often problematic because they can take lots of different values. When we have a known set of possible values it is useful to ensure only those values can occur.

Some common problems with strings:

• Case sensitivity - are "macOS" and "MacOS" the same? Should they be?
• Spaces - are "ArchLinux" and "Arch Linux" the same? Should they be?
• Normalised values - are "Arch Linux" and "Arch" the same? Should they be?
• Typos - is "Arc Linux" meant to be "Arch Linux"? Or is it a separate operating system?

Did you spot in the bug in task 12? The laptop with id 3 was never put in anyone’s preferred list, because its operating system was spelled Ubuntu not ubuntu.

We can use enums to represent that only some values are allowed, and make sure we’re always using the same ones. This is similar to how in HTML we can use a <select> menu with <options> instead of an <input type="text"> to restrict what a user can enter into a form.

In Python, we can define an enum as a new type. This is like bool - bool is a type which has two possible values (True and False). We can make enums that have any number of possible values, and we can choose the values’ names.

from enum import Enum

class OperatingSystem(Enum):
    MACOS = "macOS"
    ARCH = "Arch Linux"
    UBUNTU = "Ubuntu"

This defines a new type called OperatingSystem which has three possible values - MACOS, ARCH, and UBUNTU. We can use this type in a type annotation to make sure that we’re only passed one of these values. If someone makes a typo in one of these values, mypy will catch it and tell us that UBUNT or macOS or NIX doesn’t exist.

We know that when we save data, transfer it across a network, or take user input, everything comes in as bytes. A typical pattern in software is to accept a string in the user input, and convert it to an enum before passing it into any other function. If the string wasn’t a valid operating system we know about, we will reject it and give an error when we first accept it. All of our other functions can take an OperatingSystem as a parameter, and know that any value it’s given must be a valid operating system. This restricts where we need to worry about incorrect input - once we’ve checked that the string was correct one time, the rest of our code doesn’t have to worry about incorrect strings.

In this prep we have seen how to add methods to classes to encapsulate functionality. We have seen how to use generics to force classes to work with certain types. Keeping code reusability and maintainability in mind, what if we wanted to add a new class that did mostly the same as an existing class, but with some slight changes?

Classes can extend other classes to share most of their functionality but add or replace some of it. A class carrying over something from another class is called inheritance.

Read the code in file 15-analyse.py.

We have two classes that behave the same. They both have a constructor, and four methods (first, last, largest, length). SortedImmutableNumberList also has an extra method: max_gap_between_values which ImmutableNumberList does not have. The method implementations are different for the two classes. They have different trade-offs to consider.

Many programming libraries will have different versions of classes optimised for different tasks, and even if the API to use them is the same, you should be careful considering which one is appropriate for your specific use case. As you develop your own code, you may find it beneficial to extend certain classes to assist with certain tasks, and this may help you maintain your code or make it more efficient. Inheritance is a great way of helping you achieve this.