obestwalter
Python is made of star-stuff

hubble telescope picture of messier9 star cluster

Carl Sagan wrote in Cosmos that we are all made of “star-stuff”. The elements that make life on earth possible were formed in stars a long time ago. It puts things into perspective to remember that we still have no clue how the first life on earth came to be and why we run around on this rock hurtling through space, but at least we know that we are all made from stars :). You ask yourself what this has to do with Python? Right. Nothing. But obvioulsy the humans involved in the development of Python are also made of star-stuff and I like to believe that their deep sense of wonder about the miracle of life has inspired them to put so darn many stars into its syntax.

So, this is all about the innocent little (star or asterisk) as a versatile syntax element in Python. Depending on the context, it fulfills quite a few different roles. Here is a piece of code that uses all of them (as far as I know).

# Yes. The code makes no sense. Thanks for pointing it out.

from os import *


def append(*, end=linesep):
    def _append(function):
        def star_reporter(*args, **kwargs):
            print(*args, **kwargs, end=end)
            return function(*args, **kwargs)

        return star_reporter

    return _append


@append(end=" ❇❇❇" + linesep)
def wrapped(stars, bars):
    first, *middle, last = stars
    for elem in [*middle, last, *bars]:
        first *= 2 ** elem
    print(f"answer: {first} (don't know the question though)")

Simple things first: ✱ and ✱✱ operators 🔗

One of the first things a new Python disciple might learn is how to use Python as a calculator - like in many other languages ✱ is the multiplication operator and ✱✱ is used for exponentiation - e.g.:

2 * 2
[result]
4
2 ** 4
[result]
16
"spam" * 3
[result]
'spamspamspam'

Although it is not even trying to make sense, the star-spangled code example further up actually works. If you look at the wrapped function, ✱ is used as a boring old mathematical operator here: first *= 2 ** elem (which is using an augmented assignment and is the same as first = first * 2 ** elem).

If we run wrappped, we won’t get a useful result but at least we see that the code executes:

wrapped([1, 2, 3, 4], (23, 42))
[stdout]
[1, 2, 3, 4] (23, 42) ❇❇❇
answer: 18889465931478580854784 (don't know the question though)

Enforcing the API of a function 🔗

If your function needs 23 arguments, you have a big problem anyway but you can at least alleviate it a bit by making calls to that function more readable. Passing some or all arguments as keyword arguments usually helps. Problem is: the caller normally has the choice how to pass the arguments. You can even call a “keyword only” function like this:

def kw_only_you_wish(spam=None, eggs=None, lobster=None):
    return spam + eggs * lobster


kw_only_you_wish(2, 3, 4)
[result]
14

With Python 3.0 a new syntax was introduced to make enforcement of so called “keyword-only arguments” possible. This is used in the definition of the append function above. When using this, everything after the [, ]*, has to be passed as keyword argument or you get into trouble.

Trying to decorate a function with append and not passing end as a keyword parameter results in a friendly TypeError exception:

@append("❈❈❈")
def badly_wrapped():
    pass
[TypeError]
append() takes 0 positional arguments but 1 was given

Pack and unpack arguments 🔗

This goes back to at least Python 2.0. In this case ✱ and ✱✱ are syntax elements to be used as prefix, when defining or calling functions. The idea is usually that you want to pass through parameters to an underlying function without having to care about what or even how many they are. In this example we have a function that is just passing through arguments without needing to now anything about them:

def passing_things_through_function(*args, **kwargs):
    print(f"passing through {args=} and {kwargs=}")
    the_actual_function(*args, **kwargs)


def the_actual_function(a, b, c=None, d=None):
    print(f"passed arguments: {a=}, {b=}, {c=}, {d=}")


passing_things_through_function(*[1, 2], **dict(c=3, d=4))
[stdout]
passing through args=(1, 2) and kwargs={'c': 3, 'd': 4}
passed arguments: a=1, b=2, c=3, d=4

A case where this is particularly useful is when creating decorators that are not opinionated about the kind of function they decorate (like append). They just need to pass through whatever the decorated function needs to be called with.

There is even more to unpack 🔗

In pre Python3 days so-called tuple unpacking was already supported. Here is the classic example of swapping assignments between two names:

a = 1
b = 2
print(f"before: {a=}, {b=}")
a, b = b, a
print(f"after:  {a=}, {b=}")
[stdout]
before: a=1, b=2
after:  a=2, b=1

PEP 3132 - extended iterable unpacking brought the star into the “classic” tuple unpacking (which was never restricted to tuples but that name somehow stuck):

for iterable in [
    "egg",
    [1, 2, 3],
    (1, 2, 3),
    {1, 2, 3},
    {1: "a", 2: "b", 3: "c"},
]:
    print(f"{iterable} ({type(iterable)}):")
    a, b, c = iterable
    print(f"a, b, c        -> {a} {b} {c}")
    *a, b = iterable
    print(f"*a, b = iterable -> {a} {b}")
    a, *b = iterable
    print(f"a, *b = iterable -> {a} {b}\n")
[stdout]
egg (<class 'str'>):
a, b, c        -> e g g
*a, b = iterable -> ['e', 'g'] g
a, *b = iterable -> e ['g', 'g']

[1, 2, 3] (<class 'list'>):
a, b, c        -> 1 2 3
*a, b = iterable -> [1, 2] 3
a, *b = iterable -> 1 [2, 3]

(1, 2, 3) (<class 'tuple'>):
a, b, c        -> 1 2 3
*a, b = iterable -> [1, 2] 3
a, *b = iterable -> 1 [2, 3]

{1, 2, 3} (<class 'set'>):
a, b, c        -> 1 2 3
*a, b = iterable -> [1, 2] 3
a, *b = iterable -> 1 [2, 3]

{1: 'a', 2: 'b', 3: 'c'} (<class 'dict'>):
a, b, c        -> 1 2 3
*a, b = iterable -> [1, 2] 3
a, *b = iterable -> 1 [2, 3]

There is also more to pack 🔗

Pretty much analogous to how ✱ and ✱✱ are used in function calls they can be used in literals to create new iterables or mappings:

This syntax to merge iterables was implemented via PEP 448 (additional unpacking generalizations) in Python 3.5. [1]

a, b = [1, 2, 3], [4, 5, 6]
[*a, *b]
[result]
[1, 2, 3, 4, 5, 6]
a, b = {1: 2, 2: 3, 3: 4}, {1: 4, 4: 5, 5: 6}
{**a, **b}
[result]
{1: 4, 2: 3, 3: 4, 4: 5, 5: 6}
a, b = {1, 2, 3}, {3, 4, 5}
{*a, *b}
[result]
{1, 2, 3, 4, 5}

This is the more “natural” approach for sets though (union):

a | b
[result]
{1, 2, 3, 4, 5}

As the underlying functionality only cares about whether something is iterable, you can mix and match. This creates a tuple from a list and a set:

(*[1, 2, 3], *{3, 4, 5})
[result]
(1, 2, 3, 3, 4, 5)

Be aware though that merging maps like this is not recursive. Later keys overwrite earlier ones. Here foo will contain the second dict after merging:

a = {"a": 1, "foo": {"a": 1}}
b = {"a": 1, "foo": {"b": 2, "c": 3}}
{**a, **b}
[result]
{'a': 1, 'foo': {'b': 2, 'c': 3}}

Import all the things 🔗

The last star shines a bit dimly as this is usually an antipattern and it looks like this:

from os import *

this is usually not a good idea because:

If a package (or module) [2] is explicitly designed to be imported like this, this is usually documented and the authors defined the special module attribute __all__ that explicitly lists the names that should be imported when using from <module or package> import *

★ ★ ★ The end ★ ★ ★ 🔗

That’s all the stars I can think of for now. If you know any more: please let me know.

  1. For the historically interested: discussion on the mailing list part I and part II. [⤴]

  2. I’m either not seeing it or the Python documentation is omitting that __all__ also works for modules. It does though … I tried it. [⤴]

python fundamentals little-things [last update: 2019-11-13]