Skip to content

python

compile

Install dependancies (Debian)

# apt-get install build-essential gdb lcov pkg-config \
      libbz2-dev libffi-dev libgdbm-dev libgdbm-compat-dev liblzma-dev \
      libncurses5-dev libreadline6-dev libsqlite3-dev libssl-dev \
      lzma lzma-dev tk-dev uuid-dev zlib1g-dev
$ wget https://www.python.org/ftp/python/x.y.z/Python-x.y.z.tar.xz
$ tar xvf Python-x.y.z.tar.xz
$ cd Python-x.y.z
$ CFLAGS="$CFLAGS -fPIC"    #if compiling uwsgi python plugin later
$ echo $CFLAGS              #check
$ ./configure --enable-optimizations CFLAGS=$CFLAGS
$ make
$ make test                 #takes 30min
# make altinstall           #altinstall not to override distribution setup

interpreter binary should now be avail in /usr/bin/pythonX.Y

project setup

# pip3 install virtualenv
$ virtualenv -p pythonX.Y <PROJECT>
$ cd <PROJECT>
$ source bin/activate
(env) $ pip3 install ...
...
(env) $ deactivate

libs

email-validator

pip3 install email-validator

jinja2

Jinja cheatsheet

msgspec

msgspec is fast (Cython) vs pydantic v2, but:

  • no validation on class instantiation (only when converting from dict/JSON with msgspec.convert())

  • in class declaration, need to respect non-optional fields before optional fields

# patterns
# class declaration
class Test(msgspec.Struct, as_dict=True)
# constructor, to enforce type validation
return msgspec.convert(kwargs, cls)
# database (for insert/update queries)
msgspec.to_builtins(self)

pydantic

data validation (user input)
v1 is slow, validation part written in Cython
v2 is faster, rewrite in Rust (via py3o) 

from pydantic import BaseModel

class User(BaseModel)
    kind: ClassVar[str] = 'Humain'
    id: str | None = None

nobody = User()
migration 1.x to 2.x

Most common changes are:

.dict(), .json() 

# 1.x
model.dict()
model.json()

# 2.x
model.model_dump()
model.model_dump_json()

Validators:

# 1.x
@validator("field")
def validate_field(cls, v):
    return v

# 2.x
def field_validator("field")
def validate_field(cls, v):
    return v

python-dateutil

date manipulation

ulid-py

ULID generation
pros: time-incrementing and clearer than uuid
cons: no support in postgresql, need to cast to str 

import ulid

random_id = ulid.new().str

uuid

is part of std lib since 3.7

for uuid7, if python < 3.14, use uuid7

utils

formatting

import

first import block with standard lib imports then with third-parties libs, then with project libs

function default arguments

Without type-hints, no space around '=' PEP-8
With type-hints, spaces around '=' PEP-3107

patterns & anti-patterns

python-patterns.guide

type hints and circular imports

type hints increase occurence of circular imports.
typing.TYPE_CHECKING, True with type-checkers, False at runtime, it solves this issue.

from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from App import App
def func(app: 'App'):
    ...

Note we must now convert type-hint in string. PEP-563
With from __future__ import annotations it wouldn't be necessary. 3.10 then 3.11 were planned to default it, but postponed.

do not use mutable in default argument

Do not use [], list(), {}, dict() as default value for function argument.
Use None and test for None within function.

type-hints, list or typing.List

Before python 3.9, tuple, list could not be used as generic types in type-hints. Using typing was required. From 3.9, using tuple, list directly is the prefered way.

typing.TypedDict vs pydantic class

dict[str, Union[str, float]] allow to type a dict keys/values but granularity is low and can raise warnings from static analysis.
TypedDict improves granularity at key level. But has no impact on runtime.
pydantic will provide runtime type-validation on top of static analysis.

Enum vs StrEnum

Enum are not JSON serializable, but typed enums such as StrEnum/IntEnum/etc. are.

API hooks

@www.route(path, method)
def index():
    result = Result()
    try:
        do_stuff()
    except Exception as exc:
        result.error_msg = handle_exc(exc)

    return result.to_response(response)

debugging

use @__debug__ (from 3.12) or if __debug__: (before) to exclude function calls from production runtime (-O or -OO).

testing

pytest

(env) $ pip3 install pytest pytest-cov
(env) $ APP_ENV=dev python3 -m pytest -v tests/test_file.py
(env) $ APP_ENV=dev python3 -m pytest --cov=<MODULE> -v tests/

coverage cannot test a single file
good practice is to create tests/unit/ tests/api/...

mocking

To simplify db mocking, put all db-related functions in the same file, so it's easy to patch the db object in one single file.

Libs: Mimesis better than Faker

package management

Info

Prefer uv over poetry. Simplier and faster.

uv

docs

(py_uv) $ pip3 install uv hatchling
(py_uv) $ uv init lib1
(py_uv) lib1/ $ ln -s /path/to/lib1/ lib1    # ln to src dir
# edit pyproject.toml
(py_uv) uv build
pyproject.toml
[project]
name = "lib1"
version = "1.0.0"
description = "lib one"
authors = [ { name='user', email='user@test.com' } ]
requires-python = ">=3.11"
dependencies = []

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

poetry

docs

(env) $ pip3 install poetry
(env) $ poetry new <package>

Creates all files for new package.
Edit pyproject.toml, in particular version.

(env) /package $ poetry build

package is built and available in package/dist

pip install from local directory

No need to setup a repo. To install latest release of a given package: 

(env) $ pip install <package> --no-index --find-links ~/py_packages/<package>/dist

Note that more intuitive pip install c0lib ~/code/py_packages/c0lib/dist fails.

misc

virtual env flavors

pipenv: pip + pipfile + virtualenv
virtualenv: a PyPy package (out of Python standard distribution), use it
pyvenv: shipped with python by default, do not use
venv: similar to virtualenv (without python interpreter duplication), do not use

reinstall pip & setuptools

With fresh virtualenv setup, sometimes pip3 is not found

(env) $ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
(env) $ python3 get-pip.py --force-reinstall
(env) $ pip3 -V
(env) $ pip3 install --upgrade setuptools

pandas visualized