Skip to content

python

compile

Install dependancies (Debian) 

# apt-get install build-essential gdb lcov pkg-config \
      libbz2-dev libffi-dev libgdbm-dev libgdbm-compat-dev liblzma-dev \
      libncurses5-dev libreadline6-dev libsqlite3-dev libssl-dev \
      lzma lzma-dev tk-dev uuid-dev zlib1g-dev
$ wget https://www.python.org/ftp/python/x.y.z/Python-x.y.z.tar.xz
$ tar xvf Python-x.y.z.tar.xz
$ cd Python-x.y.z
$ CFLAGS="$CFLAGS -fPIC"    #if compiling uwsgi python plugin later
$ echo $CFLAGS              #check
$ ./configure --enable-optimizations CFLAGS=$CFLAGS
$ make
$ make test                 #takes 30min
# make altinstall           #altinstall not to override distribution setup
interpreter binary should now be avail in /usr/bin/pythonX.Y

project setup

# pip3 install virtualenv
$ virtualenv -p pythonX.Y <PROJECT>
$ cd <PROJECT>
$ source bin/activate
(env) $ pip3 install ...
...
(env) $ deactivate

libs

email-validator

pip3 install email-validator

pydantic

data validation (user input)
v1 is fast, validation part written in Cython
v2 will be faster, rewrite in Rust (via py3o)

from pydantic import BaseModel

class User(BaseModel)
    kind: ClassVar[str] = 'Humain'
    id: str | None = None

nobody = User()

migration 1.x to 2.x

Most common changes are:

.dict(), .json()

# 1.x
model.dict()
model.json()

# 2.x
model.model_dump()
model.model_dump_json()

Validators: 

# 1.x
@validator("field")
def validate_field(cls, v):
    return v

# 2.x
def field_validator("field")
def validate_field(cls, v):
    return v

python-dateutil

date manipulation

ulid-py

ULID generation
pros: time-incrementing and clearer than uuid
cons: no support in postgresql, need to cast to str

import ulid

random_id = ulid.new().str

uuid

is part of std lib since 3.7

utils

formatting

import

first import block with standard lib imports then with third-parties libs, then with project libs

function default arguments

Without type-hints, no space around '=' PEP-8
With type-hints, spaces around '=' PEP-3107

patterns & anti-patterns

python-patterns.guide

type hints and circular imports

type hints increase occurence of circular imports.
typing.TYPE_CHECKING, True with type-checkers, False at runtime, it solves this issue. 

from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from App import App
def func(app: 'App'):
    ...
Note we must now convert type-hint in string. PEP-563
With from __future__ import annotations it wouldn't be necessary. 3.10 then 3.11 were planned to default it, but postponed.

do not use mutable in default argument

Do not use [], list(), {}, dict() as default value for function argument.
Use None and test for None within function.

type-hints, list or typing.List

Before python 3.9, tuple, list could not be used as generic types in type-hints. Using typing was required. From 3.9, using tuple, list directly is the prefered way.

typing.TypedDict vs pydantic class

dict[str, Union[str, float]] allow to type a dict keys/values but granularity is low and can raise warnings from static analysis.
TypedDict improves granularity at key level. But has no impact on runtime.
pydantic will provide runtime type-validation on top of static analysis.

Enum vs StrEnum

Enum are not JSON serializable, but typed enums such as StrEnum/IntEnum/etc. are.

API hooks

@www.route(path, method)
def index():
    result = Result()
    try:
        do_stuff()
    except Exception as exc:
        result.error_msg = handle_exc(exc)

    return result.to_response(response)

debugging

use @__debug__ (from 3.12) or if __debug__: (before) to exclude function calls from production runtime (-O or -OO).

testing

pytest

(env) $ pip3 install pytest pytest-cov
(env) $ APP_ENV=dev python3 -m pytest -v tests/test_file.py
(env) $ APP_ENV=dev python3 -m pytest --cov=<MODULE> -v tests/
coverage cannot test a single file
good practice is to create tests/unit/ tests/api/...

mocking

To simplify db mocking, put all db-related functions in the same file, so it's easy to patch the db object in one single file.

package management

Info

Prefer uv over poetry. Simplier and faster.

uv

docs 

(py_uv) $ pip3 install uv hatchling
(py_uv) $ uv init lib1
(py_uv) lib1/ $ ln -s /path/to/lib1/ lib1    # ln to src dir
# edit pyproject.toml
(py_uv) uv build

pyproject.toml
[project]
name = "lib1"
version = "1.0.0"
description = "lib one"
authors = [ { name='user', email='user@test.com' } ]
requires-python = ">=3.11"
dependencies = []

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

poetry

docs 

(env) $ pip3 install poetry
(env) $ poetry new <package>
Creates all files for new package.
Edit pyproject.toml, in particular version. 
(env) /package $ poetry build
package is built and available in package/dist

pip install from local directory

No need to setup a repo. To install latest release of a given package:

(env) $ pip install <package> --no-index --find-links ~/py_packages/<package>/dist
Note that more intuitive pip install c0lib ~/code/py_packages/c0lib/dist fails.

misc

virtual env flavors

pipenv: pip + pipfile + virtualenv
virtualenv: a PyPy package (out of Python standard distribution), use it
pyvenv: shipped with python by default, do not use
venv: similar to virtualenv (without python interpreter duplication), do not use

reinstall pip & setuptools

With fresh virtualenv setup, sometimes pip3 is not found 

(env) $ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
(env) $ python3 get-pip.py --force-reinstall
(env) $ pip3 -V
(env) $ pip3 install --upgrade setuptools

pandas visualized