C interoperability¶
A TyPy program calls C libraries through the standard ctypes module, so
the same program runs under CPython, and typy build --lib builds a module
as a C library that C, C++, Rust and Python programs call as they call any
C library (FFI-01 to FFI-05, CLI-09).
What TyPy accepts¶
| Item | Rule |
|---|---|
typy build --lib static FILE writes libNAME.a, and --lib shared writes libNAME.so, beside the file or at -o PATH, for the module NAME |
CLI-09 |
A library with a version is named as Linux names libraries, such as libNAME.so.5.0 with links |
CLI-13 |
The functions the module's __all__ lists are exported under their own names, and the build writes the C header NAME.h beside the library |
FFI-05 |
The build also writes NAME_binding.py, a module that loads the library with ctypes and defines a typed function for each export, and its stub NAME_binding.pyi |
FFI-08 |
int, float and bool are int64_t, double and bool in C, and a result of None is void |
FFI-05 |
An exception never reaches C: a SystemExit ends the process with its status, and any other exception is printed and the process aborts |
EXC-05 |
NAME = ctypes.CDLL("library") or ctypes.cdll.LoadLibrary("library") at module level declares a library, which the build links |
FFI-01 |
A C function whose argtypes and restype are assigned at module level is called directly, with the conversions ctypes defines |
FFI-02, FFI-03 |
ctypes objects, pointers and arrays hold C memory as in CPython, and a C function writes them through byref and POINTER parameters |
FFI-03 |
ctypes.Structure subclasses with a literal _fields_ list are laid out as C lays out a struct, and pass to C by pointer or by value |
FFI-03 |
typy.toml says whether each library is linked statically or dynamically, and where the linker looks |
FFI-04 |
Building a library¶
A library is a module whose __all__ lists the functions it exports.
Save this as geometry.py:
__all__ = ["area", "is_square"]
calls = 0
def area(width: float, height: float) -> float:
global calls
calls += 1
return width * height
def is_square(width: int, height: int) -> bool:
return width == height
typy build --lib static geometry.py writes libgeometry.a and
geometry.h, which declares:
double area(double width, double height);
bool is_square(int64_t width, int64_t height);
A C program includes the header and links the library and the C maths library, which the runtime in the library calls:
cc -o shapes shapes.c -L. -lgeometry -lm -Wl,--gc-sections
--gc-sections leaves out the parts of the runtime that no export
reaches.
Calling a library from Python¶
typy build --lib shared geometry.py writes libgeometry.so, and beside
it geometry_binding.py, a module that loads the library with ctypes
and defines a function of the same name and types for each export, and
its stub geometry_binding.pyi, which type checkers read. A Python
program imports the binding:
>>> import geometry_binding as geometry
>>> geometry.area(2.0, 3.5)
7.0
>>> geometry.is_square(3, 3)
True
The binding loads the library by its soname, as a program loads any C
library, so the dynamic loader finds it where the library is installed,
or in a directory LD_LIBRARY_PATH names:
LD_LIBRARY_PATH=$PWD python3 shapes.py
A library built with --lib static gets the same binding, which loads
the shared library of the same release, since CPython loads only shared
libraries.
Naming a library by its version¶
A library's files carry the release of its version: the module's
__version__, or else version in typy.toml, for a module that has
none. The release is the numbers the version starts with, so
__version__ = "5.0rc1" names the files after 5.0:
| Kind | Library | Links | Soname |
|---|---|---|---|
--lib shared |
libgeometry.so.5.0 |
libgeometry.so.5 and libgeometry.so |
libgeometry.so.5 |
--lib static |
libgeometry.5.0.a |
libgeometry.a |
none |
-lgeometry finds the library through its link, and a program linked
with a shared library loads it by its soname, so it keeps working when
release 5.1 replaces 5.0. Without a version, the files are
libgeometry.so and libgeometry.a. With -o PATH, the library is
written at that path alone. A __version__ that does not start with a
number, or that differs from version in typy.toml, is E0429.
How a library runs¶
The module runs once, when the library is loaded: when the program starts,
for a library it links, or when ctypes.CDLL loads it. That is when CPython
runs a module's body, on import, so the module's variables hold their
values before the first call. The module's __name__ is its own name, so
code under if __name__ == "__main__": does not run, and sys.argv is
['']. What the module and the exports print is written before each call
returns, so it keeps its order with what the program prints, as long as the
program writes its own output before it calls the library, as
fflush(stdout) or python3 -u does.
An export's name is its C symbol. Its parameters and result are int,
float and bool, and it may return None. A function that takes or
returns other values needs an export of these types that calls it. An
int result outside int64_t raises OverflowError, which ends the
process as any exception does. The runtime is not safe to call from two
threads at once, so a program calls a library's exports from one thread at
a time; each call may come from a different thread. Each library holds its
own copy of the runtime, so a program may link several TyPy libraries.
An export may not take a name the runtime calls in the C library, such as
read, free or sqrt, or one C keeps for itself, such as __init; the
build reports E0429 and names the function. Building with --lib checks
the names: typy check --lib static geometry.py reports the same
diagnostics without writing a library.
Calling C through ctypes¶
A program declares a library and each function it calls, as a program for CPython does, and TyPy compiles each call into a direct call of the C function, with the library linked when the program is built (ADR-0047):
import ctypes
libm = ctypes.CDLL("libm.so.6")
libm.cbrt.argtypes = [ctypes.c_double]
libm.cbrt.restype = ctypes.c_double
def side(volume: float) -> float:
return libm.cbrt(volume)
print(side(27.0), side(1000.0))
The library is named by a string literal, and both argtypes and
restype are assigned at module level, once, before the function is
called at module level; a declaration that breaks one of these is E0430.
An argument of a type the declaration does not take is E0404 when the
program is compiled. Each value converts as ctypes converts it:
| Type | Argument | Result |
|---|---|---|
c_int8 to c_uint64, c_byte, c_short, c_int, c_long, c_longlong and their unsigned forms, c_size_t, c_ssize_t |
An int or a bool, wrapped to the type's width |
An int |
c_float, c_double |
An int, a float or a bool |
A float |
c_bool |
Any value, by its truth | A bool |
c_char_p |
bytes, whose own buffer C reads, or None |
bytes, or None for null |
c_void_p |
An int, bytes or None |
An int, or None for null |
An int too large for a float raises ctypes.ArgumentError, as in
CPython. A library or a function the linker cannot find stops the build
with E0702, where CPython raises OSError or AttributeError when the
program runs.
Objects, pointers and arrays¶
A ctypes object holds a C value in memory a C function can write. A
parameter declared as ctypes.POINTER(T) takes the object's address,
whether the program passes ctypes.byref(x), the object itself, an array
of T, a pointer from ctypes.pointer(x), or None for null (ADR-0048):
import ctypes
libm = ctypes.CDLL("libm.so.6")
libm.frexp.argtypes = [ctypes.c_double, ctypes.POINTER(ctypes.c_int)]
libm.frexp.restype = ctypes.c_double
exponent = ctypes.c_int()
mantissa = libm.frexp(48.0, ctypes.byref(exponent))
print(mantissa, exponent.value, exponent)
Numbers = ctypes.c_int * 4
numbers = Numbers(1, 2, 3)
numbers[3] = numbers[0] + len(numbers)
print(list(numbers), sum(numbers), ctypes.sizeof(numbers))
pointer = ctypes.pointer(exponent)
pointer[0] += 1
print(exponent.value, pointer.contents)
| Operation | Gives |
|---|---|
T(value), T() |
An object of a scalar type holding the value converted as an argument is, or zero, None or False |
x.value, x.value = v |
The value as a result of type T is, and a new value converted and wrapped to the type's width |
ctypes.pointer(x), POINTER(T)(x), POINTER(T)() |
A pointer that keeps x alive, or a null pointer |
p.contents, p[i], p[i] = v |
The object pointed to, and the value of an item; a null pointer raises ValueError |
T * n, (T * n)(items) |
An array class of n items, n an integer literal, and an array, the rest zero |
a[i], a[i] = v, len(a), for v in a, v in a |
Items counted from the end when negative; one outside raises IndexError |
ctypes.sizeof(T), ctypes.sizeof(x) |
The bytes of the class on 64-bit Linux |
repr(x), bool(x), x == y |
As in CPython: c_long(7) for a c_int64, the value's truth, and identity |
A POINTER(T) result is a new pointer over memory the C library owns.
A class is bound to a name, called, multiplied into an array class or
named in a declaration; using it any other way is E0201 under FFI-10.
Structures¶
A class whose one base is ctypes.Structure, with a _fields_ list
written in the class, is a C struct: each field sits where a C compiler
puts it, and a C function takes the structure by pointer or by value
(ADR-0049):
import ctypes
libc = ctypes.CDLL("libc.so.6")
class Division(ctypes.Structure):
_fields_ = [("quotient", ctypes.c_int), ("remainder", ctypes.c_int)]
class Segment(ctypes.Structure):
_fields_ = [("start", Division), ("ends", ctypes.c_int * 2)]
libc.div.argtypes = [ctypes.c_int, ctypes.c_int]
libc.div.restype = Division
result = libc.div(17, 5)
print(result.quotient, result.remainder)
segment = Segment(result, ends=(ctypes.c_int * 2)(1, 2))
segment.start.quotient += 10
print(segment.start.quotient, list(segment.ends), ctypes.sizeof(Segment))
S(...) takes the fields in order, then by name; too many values, a name
that is not a field, or a field given twice is E0404. A scalar field reads
and assigns as value does. A structure, array or pointer field reads as
an object over the field's memory, so a change through it changes the
structure, and assigning one copies it. The calling convention is that of
x86-64 Linux.
Linking a library¶
A library is linked dynamically unless typy.toml says otherwise, and the
system loads it when the program starts. A project that ships its own
libraries names where the linker looks, and which to link into the
program:
library-paths = ["lib"]
[libraries."libtools.so.2"]
link = "static"
A static library is the archive named like the shared one up to .so, so
libtools.so.2 links lib/libtools.a. A TyPy library's binding,
tools_binding.py, loads the library by its soname, so a TyPy program that
imports the binding calls the library directly, linked as typy.toml
says, and CPython runs the same program through ctypes.
Planned¶
Planned for 1.0 are methods and class attributes other than _fields_
in a structure, _fields_ assigned after the class statement, bit
fields, a subclass of a structure, and ctypes.Union (FFI-11); and
c_char, c_wchar, c_wchar_p, create_string_buffer, cast and
addressof, pointers to and arrays of pointers and arrays, arrays whose
length is not an integer literal, array slices, byref other than as a C
function's argument, assigning a pointer's contents, and ctypes
classes as values, in annotations, isinstance and type (FFI-10), so a
function takes and returns the values of an object rather than the object.
A library or a C function used as a value, stored, passed or printed, is
planned for 1.0 too (FFI-09):
import ctypes
libc = ctypes.CDLL("libc.so.6")
print(libc)