Intermediate topics#

If you are already familiar with the Python basics, this notebook will introduce you to some more advanced topics. Neither of these topics is required to complete the exercises, but they may be useful to you in the future.

This notebook is not a complete introduction to these topics, but rather a quick overview of some of the most useful features. If you want to learn more, you can find many tutorials online.

We will cover:

  • Dict-comprehensions

  • Lambda functions

  • Type hints

  • Keyword arguments

  • *args and **kwargs

  • Dataclasses

  • Decorators

  • Utilities from the standard library

Dict-comprehension#

We previously introduced list-comprehension:

mylist = [2, 1, 3, 4, 5]
mysquares = [x**2 for x in mylist]
mysquares

[4, 1, 9, 16, 25]

There is also dict-comprehension, with various flavors:

mydict = {"a": 1, "b": 2, "c": 3}

# Iterate over dict items
mysquaresdict = {key: value**2 for (key, value) in mydict.items()}

# Iterate over dict items and modify keys
mydictwithnewkeys = {key.upper(): value**2 for (key, value) in mydict.items()}

# Iterate over a list or other iterable
mydictfromlist = {x: x**2 for x in mylist}

print(mysquaresdict)
print(mydictwithnewkeys)
print(mydictfromlist)

{'a': 1, 'b': 4, 'c': 9}
{'A': 1, 'B': 4, 'C': 9}
{2: 4, 1: 1, 3: 9, 4: 16, 5: 25}

Lambda functions#

Lambda functions are a way to define functions “inline”, without giving them a name. This can be useful when we deal with another function that takes a function as an argument. The syntax is:

lambda <arguments>: <expression>

The expression is evaluated and returned when the function is called. Example:

def process_list(values, function):
    return [function(x) for x in values]


mylist = [2, 1, 3, 4, 5]
print(process_list(mylist, lambda x: x + 2))
print(process_list(mylist, lambda x: x**2))

[4, 3, 5, 6, 7]
[4, 1, 9, 16, 25]

Type hints#

Python is a dynamically typed language, which means that the type of a variable is not known at compile time. This is in contrast to statically typed languages like C, where the type of a variable must be declared before it is used. As a relatively recent addition to the language, Python now supports type hints. These are not enforced by the interpreter, but can be used by external tools to check for type errors. Maybe more importantly, they can be used to document the expected types of arguments and return values, improving readability. Over the past years, type hints have become increasingly popular, and are now used in many popular libraries, to the point where they are almost expected.

Example:

def format_info(name: str, age: int) -> str:
    return f"{name} is {age} years old"

  • Type hints for function arguments are written after the argument name, separated by a colon:

    • name is a string

    • age is an integer

  • The return type is written after an arrow ->:

    • The function returns a string

Note that the type hints are literally hints and are not enforced by the interpreter, so the following code will run without errors:

format_info("John", 25.5)

'John is 25.5 years old'

format_info(11, [1, 2, 3])

'11 is [1, 2, 3] years old'

Apart from built-in types like int, float, str, list, dict, etc., we can also use custom types. In many cases, we will use the typing module, which provides a number of useful types.

Example:

from typing import List, Union
import math


def mysqrt(x: List[Union[int, float]]) -> List[float]:
    """Take a list of integers or floats and return a list of their square roots"""
    return [math.sqrt(y) for y in x]

Keyword arguments#

When calling a function, we can specify the arguments by position or by name. The latter is called keyword arguments. Keyword arguments can be used to make the code more readable, and to specify default values for arguments. Using keyword arguments for all arguments is a good practice (apart from cases such as function with a single arguments), as it makes the code more robust to changes in the function signature and improves readability:

def format_info(name: str, age: int) -> str:
    return f"{name} is {age} years old"


format_info(name="John", age=25)

'John is 25 years old'

*args and **kwargs#

Sometimes we want to write a function that takes an arbitrary number of arguments. This can be done using *args and **kwargs. The names args and kwargs are not special, but are commonly used. Example:

def average_age(*ages: int) -> float:
    return sum(ages) / len(ages)


def format_family_info(**ages: int) -> str:
    info = ", ".join([f"{name} is {age} years old" for name, age in ages.items()])
    return info + f". Their average age is {average_age(*ages.values())}."


format_family_info(John=25, Jane=24, Jack=17)

'John is 25 years old, Jane is 24 years old, Jack is 17 years old. Their average age is 22.0.'

We can also use * and ** when calling a function, to unpack a list or dictionary into arguments:

family = {"Jack": 17, "Jill": 15}
print(f"Average age: {average_age(*family.values())}")
print(format_family_info(**family))

Average age: 16.0
Jack is 17 years old, Jill is 15 years old. Their average age is 16.0.

Dataclasses#

Dataclasses are a convenient way to create classes that are mostly used to store data. Even if you are not familiar with classes in Python, you can probably understand the following example. For more details, see, for example, this video introduction: If you’re not using Python DATA CLASSES yet, you should.

from dataclasses import dataclass


@dataclass
class Experiment:
    name: str
    date: str
    temperature: float
    pressure: float

@dataclass is a so-called decorator that adds convenient functionality to the class. We can use our class Experiment as follows:

exp1 = Experiment(
    name="my first experiment", date="2020-01-01", temperature=20.0, pressure=1.0
)
exp2 = Experiment(
    name="my second experiment", date="2020-01-02", temperature=21.0, pressure=1.1
)
print(exp1)
print(exp2)
# Access fields
print(exp1.date)

Experiment(name='my first experiment', date='2020-01-01', temperature=20.0, pressure=1.0)
Experiment(name='my second experiment', date='2020-01-02', temperature=21.0, pressure=1.1)
2020-01-01

Dataclasses can also have default arguments. For mutable default argument such as a list, we need to use field(default_factory=list):

from dataclasses import dataclass, field


@dataclass
class Experiment:
    date: str
    location: str = "ESS"
    comments: list = field(default_factory=list)


# Default location and comments
print(Experiment(date="2020-01-01"))
# Override default location, keep default comments
print(Experiment(date="2020-01-01", location="ILL"))
# Default location, override default comments
print(Experiment(date="2020-01-01", comments=["test1", "test2"]))

Experiment(date='2020-01-01', location='ESS', comments=[])
Experiment(date='2020-01-01', location='ILL', comments=[])
Experiment(date='2020-01-01', location='ESS', comments=['test1', 'test2'])

Decorators#

Decorators are a way to modify the behavior of a function. They are written as functions that take a function as an argument, and return a new function. The syntax is:

def mydecorator(func):
    def wrapper(*args, **kwargs):
        # Do something before calling the function
        print(
            f"Your function {func.__name__} is about to be called with arguments {args} and {kwargs}"
        )
        # Call the function
        result = func(*args, **kwargs)
        # Do something after calling the function
        print(f"Your function {func.__name__} was called")
        return result

    return wrapper


@mydecorator
def myfunction():
    print("Hello world")


@mydecorator
def myfunction2(x):
    print(f"Hello {x}")


myfunction()
myfunction2("John")

Your function myfunction is about to be called with arguments () and {}
Hello world
Your function myfunction was called
Your function myfunction2 is about to be called with arguments ('John',) and {}
Hello John
Your function myfunction2 was called

Utilities from the standard library#

When you are writing Python code, you should always check if there is a utility in the standard library that does what you want. More often than not, there is. Here are some examples: