TyPy detailed design¶
| Field | Value |
|---|---|
| Document | TyPy detailed design |
| Version | 2.23 |
| Status | Approved |
| Owner | typy-lang contributors |
| Last updated | 2026-11-17 |
1. Purpose and scope¶
This document specifies the code of TyPy: the package layout, the layering rules, and each type with its fields, behaviour and errors, under the names the code uses. It builds on the approved software architecture, the domain model and the decision records ADR-0001 to ADR-0009.
It covers release 0.1, every requirement of the specification whose release is 0.1, and the parts of release 0.2 written so far. Each later release extends this document in the same change as its code. Where a type gains fields or variants in a later release, the text says so with the release number.
2. Workspace layout¶
Each crate is laid out by the rules of section 3.1: one module per kind of
object, a name.rs file for a module and a name/ folder for its children.
Cargo.toml workspace, shared lint settings
Cargo.lock committed
rust-toolchain.toml Rust 1.99.1
clippy.toml project names allowed in documentation
crates/
typy-diagnostics/src/ span, code, rule, registry, diagnostic,
diagnostics, render
typy-source/src/ settings, module, source_file, line_index,
int_literal, project,
syntax/ (tree, statement, expression,
operator, node_kind, dump),
parse/ (statement, expression, operator)
typy-check/src/ types, builtin, method, unprovided, check/
(definition, annotation, module_scope,
definition_order, visit), subset/ (feature,
statement, expression), body/ (program,
flow, locals, statement, expression, name,
operator, call, builtin_call, iterable_call,
list, slice, loop_source), typed/
(symbol, constant, signature, function,
program, statement, expression)
typy-check/tests/ checker.rs, through the public API
typy-tyir/src/ module, function, block, instruction,
types, verify, print
typy-lower/src/ program, function, ownership, invoke,
statement, augmented, expression,
arithmetic, comparison, conversion, builtin,
iterable_builtin, tuple, list, list_method,
slice, loop_source, call_graph, mangle,
strings, types
typy-codegen/src/ target, optimization, layout, statics, abi,
runtime, function, operation, constant,
descriptor, bits, tuple, arithmetic,
comparison, call, terminator, emit
typy-build/src/ request, mode, emit_stage, runtime_archive,
linker, artifact
typy-driver/src/ pipeline, emission, outcome, failure,
sources
typy-cli/src/ main, arguments, command/ (check, build,
run), cache, output, exit_status, guard
typy-runtime/src/ entry, cell, header, allocator, libc, libm,
refcount, word, compare, int, float,
string, list, tuple, slice, zip, message,
io, exception, range, stack, set, dict,
hash, type_object, instance
typy-conformance/src/ expectation, observation, oracle, subject,
suite, test_case, verdict
tests/
conformance/<area>/ <ID>_<behaviour>.py, one rule per file
golden/<stage>/ expected --emit output
e2e/ tests that run the typy executable
The crates from typy-tyir on are listed with their planned modules; each
is laid out by the same rules when it is written.
3. Layering and code rules¶
The dependencies between crates are those of architecture section 4.2, and
a CI job checks them against cargo metadata.
3.1 Modules¶
The code is organised so that it stays easy to change as it grows:
- One module per kind of object or per responsibility:
SpanandPositioninspan, theDiagnosticscollection apart from the singleDiagnostic, operator typing apart from name resolution. - A module with children is a
name.rsfile and aname/folder, withoutmod.rs. A type's methods may be spread over child modules by responsibility, asBodyChecker's are underbody/. - Modules are private. The crate root re-exports the public API with
pub use, so a type moves between modules without breaking a caller. - Items are as private as they can be:
pub(super)for a module family,pub(crate)for the crate,pubonly for the API. - A source file stays under about 400 lines. A longer file is split by responsibility before more is added to it.
- Unit tests sit with their module, in
#[cfg(test)] mod tests, or inname/tests.rswhen they are long. Tests that need only the public API are integration tests in the crate'stests/folder.
3.2 Code rules¶
Within the code:
- Every compiler crate starts with
#![forbid(unsafe_code)]. Onlytypy-runtimecontainsunsafe, and each block carries a comment that starts withSAFETY:and says why it is sound. - Clippy runs with
-D warningsand thepedanticgroup. Each allowed lint is listed inCargo.tomlwith its reason. - Identifiers of compiler objects are newtypes over
u32:FileId,SymbolId,ScopeId,FunctionId,BlockId,ValueId. They index a vector owned by the aggregate that created them and are never mixed. - Maps whose order can reach any output are
BTreeMapor sortedVec(architecture section 7.2).HashMapis used only for lookups whose order never escapes. - Functions that can fail return
Result. Compiler code does not callunwrap,expectorpanic!on a path that input can reach; a broken internal invariant returns an internalDiagnosticwith codeE9001. - There is no process-wide mutable state in the compiler. Everything a
command needs travels in the
BuildRequestand the objects the stages return. - Line and branch coverage is measured with
cargo-llvm-covover the whole workspace, the runtime included, and counts thetypyexecutable when tests run it as a subprocess. CI fails below 95 percent (NFR-11).
4. typy-diagnostics¶
4.1 Types¶
TextOffset
| Field | Type | Default | Rule |
|---|---|---|---|
0 |
u32 |
none | A byte offset into a SourceFile; LEX-09 keeps it below 2^24 |
Span
| Field | Type | Default | Rule |
|---|---|---|---|
file |
FileId |
none | Names a file of the current Project |
start |
TextOffset |
none | On a UTF-8 character boundary |
end |
TextOffset |
none | Not before start; on a character boundary |
Span::new(file, start, end) returns None when end is before start.
Position
| Field | Type | Default | Rule |
|---|---|---|---|
line |
u32 |
none | From 1 |
column |
u32 |
none | From 1, counted in code points (DIA-01) |
DiagnosticCode
| Field | Type | Default | Rule |
|---|---|---|---|
severity |
Severity |
none | Error for E codes, Warning for W codes |
number |
u16 |
none | 0 to 9999; its first two digits give the range of section 4.3 |
DiagnosticCode::parse("E0401") returns Err(CodeError::Malformed) for
any text that is not a letter E or W followed by four digits.
Display writes the same form back.
RuleId wraps a &'static str such as "TYP-03". Only the constants in
rules.rs create one, and a unit test checks that each constant is an ID
in docs/requirements/srs.md.
Diagnostic
| Field | Type | Default | Rule |
|---|---|---|---|
code |
DiagnosticCode |
none | Listed in the registry (section 4.3) |
span |
Option<Span> |
none | The primary location; None for a diagnostic about no place in the source, such as a failed link |
message |
String |
none | One sentence in lower case with no final period (DIA-01) |
labels |
Vec<Label> |
empty | Secondary spans, each with a short text |
help |
Option<String> |
None |
A hint, shown after help:, when no fix is certain |
fix |
Option<Fix> |
None |
Set only when the fix is certain (DIA-02) |
rule |
Option<RuleId> |
None |
Required when the diagnostic rejects valid Python (DIA-03) |
Diagnostic::new(code, span, message) makes a located diagnostic and
Diagnostic::without_location(code, message) one without a span.
Fix holds a Span and the replacement: String for it. Label holds a
Span and a text: String.
Diagnostics collects diagnostics for one command.
| Method | Behaviour |
|---|---|
push(diagnostic) |
Adds it, unless one with the same code and span is already present |
has_errors() |
True when any diagnostic has severity Error |
iter() |
The diagnostics in the order they were added |
into_sorted(lookup) |
Returns them sorted by file path, line, column and code; those without a location come first |
SourceLookup is the trait the renderers use to turn a Span into text.
typy-source::Project implements it.
| Method | Returns |
|---|---|
path(file) |
The file's path relative to the project root |
position(file, offset) |
The Position of the offset |
line_text(file, line) |
The text of one line, without its line ending |
4.2 Rendering¶
render_text(&Diagnostic, &dyn SourceLookup) -> String writes the form
below. The source line is shown with a marker under the span, and the fix,
when present, as the corrected line.
error[E0402]: `total` was declared as int and is assigned a str
--> app.py:3:13
|
3 | total = "done"
| ^^^^^^
= rule: TYP-03
help: assign an int, or use a new variable
The gutter is as wide as the line number. A span that runs past its first
line is marked to the end of that line. Each label is written as
= LINE:COLUMN: text. A certain fix follows as help: write it as and the
corrected line.
A diagnostic without a location is written as its header, then its rule and its help:
error[E0701]: the C toolchain program `cc` was not found
= rule: A-01
help: install `cc`, which TyPy runs to link the program
render_json writes the same diagnostic as one JSON object on one line
(DIA-05). Every key is always present, in this order, with null where
the diagnostic has nothing, so a tool reads each line the same way:
| Key | Value |
|---|---|
code |
The code, such as "E0402" |
severity |
"error" or "warning" |
message |
The one sentence of DIA-01 |
file |
The path from the project root, or null |
line, column, end_line, end_column |
The span from 1, columns in code points, or null |
rule |
The requirement ID, such as "TYP-03", or null |
help |
The help line, or null |
labels |
An array of {"line", "column", "text"} |
fix |
{"line", "column", "end_line", "end_column", "replacement"}, or null |
A string escapes a quote, a backslash and each control character, and keeps every other character as UTF-8.
4.3 Diagnostic codes¶
The registry in registry.rs lists every code with its summary, the rule
it enforces and its status. typy explain reads the same table.
| Code | Summary | Rule |
|---|---|---|
| E0101 | Source file larger than 16 MiB | LEX-09 |
| E0102 | Source file is not valid UTF-8 | LEX-01 |
| E0103 | Python syntax error, syntax newer than the Python floor, or return, break or continue out of place |
STD-01 |
| E0104 | Imaginary literal | LEX-08 |
| E0105 | Source file cannot be read | None |
| E0106 | Output file cannot be written | None |
| E0107 | typy.toml is not valid |
CLI-07 |
| E0201 | Feature planned for a later TyPy release | The feature's own rule |
| E0202 | Feature TyPy cannot compile with CPython's meaning | STD-04 |
| E0203 | Parameter without a type annotation | TYP-01 |
| E0204 | Function without a return annotation | TYP-01 |
| E0205 | Name is not defined | STD-04 |
| E0206 | Import of a module TyPy does not provide | LIB-08 |
| E0301 | is used on values other than None, bool values or class instances |
EXP-08 |
| E0302 | ** whose CPython result could be a float for int ** int, or complex |
EXP-04 |
| E0303 | and or or with operands of different types |
EXP-09 |
| E0304 | Truth test on a value of an unsupported type | EXP-10 |
| E0305 | Conditional expression with branches of different types | EXP-11 |
| E0306 | Variable read before it is assigned on every path | TYP-12 |
| E0307 | Tuple indexed other than by an integer literal inside it | DAT-08 |
| E0308 | Unpacking into a different number of targets | SMT-02 |
| E0309 | Instance used, or an attribute read, before __init__ assigns every attribute; __init__ that can end without assigning one |
CLS-01 |
| E0310 | Attribute whose type is not known yet, because the __init__ that assigns it is checked later |
CLS-01 |
| E0401 | Operator not defined for these operand types | EXP-02 |
| E0402 | Assignment changes the type of a variable | TYP-03 |
| E0403 | Any, cast or an untyped value |
TYP-10 |
| E0404 | Argument of the wrong type | TYP-01 |
| E0405 | Returned value of the wrong type | TYP-01 |
| E0406 | Arithmetic on bool |
DAT-03 |
| E0407 | Comparison between unsupported types | EXP-06 |
| E0408 | tuple, list, set or dict annotation without its item types |
TYP-06 |
| E0409 | List, set or dict display with items, keys or values of different types | DAT-07, DAT-09, DAT-10 |
| E0410 | Empty list or dict display, list(), tuple(), set() or dict() without an annotation |
TYP-05 |
| E0411 | for over a value that cannot be iterated |
CST-03 |
| E0412 | Method a built-in type does not have | LIB-03 |
| E0413 | Set item or dict key of a type without a hash: a list, a set, a dict, a view, or a tuple that holds one | DAT-09, DAT-10 |
| E0414 | Attribute or method a class does not have | CLS-02 |
| E0415 | raise or from of a value that is not an exception |
SMT-06 |
| E0416 | Bare raise outside an except clause |
EXC-07 |
| E0417 | Call of an exception class TyPy does not let a program build | EXC-12 |
| E0418 | except clause that names something other than exception classes |
EXC-09 |
| E0419 | return, break or continue that leaves a finally block |
EXC-06 |
| E0420 | with on a value whose class has no __enter__ and __exit__ |
CST-05 |
| E0421 | del of something other than a list item, a list slice or a dict key |
SMT-05 |
| E0423 | global of a name the module does not bind |
SMT-08 |
| E0424 | Module variable an __init__ reads before the module's code gives it a type |
SMT-08 |
| E0425 | Import of a name its module does not define | SMT-07 |
| E0426 | Import inside __init__ of a module that the imported module imports |
MOD-15 |
| E0427 | % format that does not fit its values |
LIB-10 |
| E0428 | Copies of a generic function without end | GEN-03 |
| E0429 | Module or export that a C library cannot have | FFI-05, CLI-09 |
| E0430 | ctypes declaration TyPy cannot compile |
FFI-01, FFI-02, FFI-03 |
| E0701 | C toolchain program not found | A-01 |
| E0702 | Link failed | A-01 |
| E0703 | Runtime archive missing, or from another TyPy version | None |
| E9001 | Internal compiler error | NFR-09 |
| W0101 | Invalid escape sequence | DIA-08 |
| W0301 | Assertion that is always true, of a tuple with items | DIA-08 |
| W0401 | match that leaves members of the subject's type unmatched |
CST-07 |
5. typy-source¶
5.1 Types¶
ProjectSettings
| Field | Type | Default | Rule |
|---|---|---|---|
root |
Utf8PathBuf |
Directory of the main file | Absolute; every module path is relative to it |
release |
Release |
Release::V0_1 |
The release whose feature table the subset checker uses |
python_floor |
PythonFloor |
PythonFloor::V3_12 |
One of 3.12 to 3.15; the parser's target version and the lowest oracle version (ADR-0008). Read from python-floor in typy.toml (CLI-12) |
module_paths |
Vec<Utf8PathBuf> |
empty | Directories relative to root, each made only of normal components and existing; the loader looks for a module in root, then in each, in order (MOD-01) |
library |
bool |
false |
Set by --lib through SettingOverrides::library: the main file is a module named after its file, and the checker reads its exports from __all__ (CLI-09, ADR-0044) |
version |
Option<String> |
None |
version in typy.toml, a string whose release version_release finds; the version of a library whose module has no __version__ (CLI-13) |
library_paths |
Vec<Utf8PathBuf> |
empty | library-paths: directories under root, checked as module_paths are, where the linker looks for C libraries first (FFI-04) |
libraries |
BTreeMap<String, LibraryLink> |
empty | [libraries."NAME"], each a table of one link, "static" or "dynamic", for a name of letters, digits and _.+- that does not start with -; LibraryLink::linker_name gives :NAME or :BASE.a (FFI-04) |
allocator |
Option<Allocator> |
None |
The table [allocator]: the C functions allocate, reallocate and free, each an identifier of at most 255 characters outside the prefixes typy_, __typy and _TY, all three different, and an optional library, a name the linker takes as -lname that starts with a letter, a digit or _ (MEM-08, ADR-0043) |
ProjectSettings::read(root, overrides) builds the settings once, when a
command starts: the defaults, then root/typy.toml when it exists, then
SettingOverrides, the flags of the command line (CLI-07). The file is
parsed with toml_edit, whose spans place each E0107 on its line; every
wrong setting is reported, and the command stops with exit status 2 before
any module is read (UC-11). The file becomes SourceFile 0 of the
LoadFailure, so the diagnostics render with its text. The table
[allocator] is read by settings_file::allocator, whose problems name
MEM-08. Library links from typy.toml are planned for 0.5 (FFI-04).
Project::load names the main module __main__, or for a library build
after its file without .py, so tools.py is the module tools. The name
also names the library and its C header, so a file name that is not an
ASCII identifier is E0429 under CLI-09, before any file is read.
SourceFile
| Field | Type | Default | Rule |
|---|---|---|---|
id |
FileId |
none | Unique in its Project |
path |
Utf8PathBuf |
none | Relative to ProjectSettings::root, or <stdlib>/name.py for embedded modules |
text |
Arc<str> |
none | Valid UTF-8, at most 16 MiB, without a leading byte order mark |
lines |
LineIndex |
none | Byte offset of each line start, built when the file is read |
SourceFile::read(id, root, path) refuses a file over 16 MiB with E0101
before reading it, and a file that is not UTF-8 with E0102 at the offset of
the first invalid byte. A file that cannot be read gives E0105 with the
operating system's reason. In each case the file is still registered, with
empty text, so the diagnostic has a location. A leading byte order mark is
removed, as CPython accepts it. LineIndex treats LF, CRLF and a lone CR
as line endings, as LR 2.1.2 does.
ModuleName wraps a String of dotted identifiers, such as __main__ or
builtins. ModuleName::new refuses an empty name and any part that is
not an identifier.
Module
| Field | Type | Default | Rule |
|---|---|---|---|
name |
ModuleName |
none | __main__ for the main file (STD-06) |
file |
FileId |
none | Its SourceFile |
syntax |
SyntaxTree |
none | Built by parse |
is_package |
bool |
none | Read from an __init__.py (MOD-02); package() gives the package a relative import starts from |
Project
| Field | Type | Default | Rule |
|---|---|---|---|
settings |
ProjectSettings |
none | Immutable after load |
main |
ModuleName |
__main__ |
A key of modules |
modules |
BTreeMap<ModuleName, Module> |
none | The main module and every module its imports reach |
files |
Vec<SourceFile> |
none | Indexed by FileId; a file imported under two names, as the main file by its own name, is read once |
imports |
BTreeMap<ModuleName, Vec<Import>> |
none | Each module's imports, with the packages above each module they name |
order |
Vec<ModuleName> |
none | Each module after the modules its module-level imports need, in name order otherwise |
standard |
BTreeSet<ModuleName> |
none | The standard library modules imported, which no file under the root provides |
Project::load(main_path, settings) -> Result<Project, LoadFailure> reads
and parses the main file and every module its imports reach, and returns
every diagnostic of every file together. LoadFailure holds the
Diagnostics and the files they point into, and implements
SourceLookup, so a failed load can still be rendered. The checker reads
the declarations of builtins from typy-stdlib (section 6).
loader.rs finds the imports of each module at any depth, in function and
class bodies too. An Import names the module, its span, whether it runs
while the module initialises, which an import in a function does not, and
whether it is optional: from a import b needs the submodule a.b only
when it exists. import a.b needs a, then a.b. absolute_name turns a
relative import into an absolute name from the importing module's
package, with CPython's messages for a module outside a package and for
too many dots. locate finds a.b as the package a/b/__init__.py, or
else a/b.py, under the main file's directory, as CPython's file finder
prefers a package. A module no file provides is a standard library module
when its top-level name is in STDLIB_NAMES, CPython 3.12 to 3.15's
sys.stdlib_module_names written by scripts/generate-stdlib-names.py;
the checker says whether TyPy provides it. Any other is E0206, no module
named 'x', with ; 'a' is not a package when a is a module.
initialisation_order follows the module-level imports depth first; an
import that leads back to a module on the path is E0422, with each import
of the cycle shown, MOD-05 (ADR-0015).
5.2 The syntax tree¶
SyntaxTree is TyPy's own copy of the parsed module. Its node types follow
the names of CPython's ast module for Python 3.12, such as FunctionDef,
AnnAssign, BinOp and Compare, so the Language Reference and the ast
documentation describe them. Every node carries a Span.
A node the current release compiles is converted with all its parts. Any
other node becomes Stmt::Other or Expr::Other, holding its NodeKind,
such as ClassDef or ListComp, and its span, which is what the subset
checker needs to reject it with E0201 or E0202. Each release converts the
nodes of the features it adds, as 0.2 does for Expr::Set, a set
display, Expr::Dict, a dict display whose keys hold None for a **
item, as CPython's ast.Dict does, and Expr::ListComp, SetComp and
DictComp, whose clauses are Comprehension values as CPython's
ast.comprehension.
elif chains become nested If nodes, as
in CPython's ast. An integer literal keeps its value as decimal digits,
so literals of any size survive conversion (DAT-01).
dump(tree, lines) -> String writes the tree as indented text with the
LINE:COLUMN of each node, for --emit=ast and golden tests. No type from
ruff_python_parser or ruff_python_ast appears outside parse.rs
(ADR-0005).
parse(file: &SourceFile, floor: PythonFloor) -> Result<SyntaxTree,
Vec<Diagnostic>> calls
ruff_python_parser::parse_unchecked with
ParseOptions::from(Mode::Module) and the program's Python floor as the
target version, 3.12 in 0.1. Each entry of Parsed::errors() and
Parsed::unsupported_syntax_errors() becomes an E0103 diagnostic, so
syntax that the floor version does not accept is refused (STD-01). The
parser reads PEP 810 lazy imports and marks them is_lazy, which the
syntax tree keeps (MOD-06). typy_source::lazy_modules reads a module's
__lazy_modules__ list, and typy_source::is_lazy says whether an import
statement is lazy by its keyword or by that list (MOD-07); the loader
orders the modules of lazy imports as it orders imports inside a
function, so they close no cycle. The tree is converted only when there
are no errors.
Before parsing, parse/limits.rs reads the tokens and refuses the 201st
nested bracket with "too many nested parentheses" and the 100th nested
block with "too many levels of indentation", CPython's tokenizer limits
(LEX-11, ADR-0020). The parser runs with its stacker feature, which
grows its stack on the heap. Converting the tree counts the nesting of
expressions and statements, an elif counting as the if it becomes,
and stops past MAX_DEPTH, 25000, with E0103; so every later pass
recurses at most that deep, and the typy command runs the compiler on
a thread with a 512 MiB stack, which holds it.
The parser gives a literal's value as UTF-8 text, which writes U+FFFD
for each escape of a surrogate. parse/surrogate.rs reads the literal's
source in order: the n-th U+FFFD of the value comes from the n-th item
that gives one, U+FFFD itself, \ufffd, \U0000fffd,
\N{REPLACEMENT CHARACTER} or a surrogate escape, so each surrogate goes
back in its place whatever newlines the parser normalised (LEX-10). A
literal that holds one becomes Constant::StrPoints, its code points, and
so does its implicit concatenation; literal text of an f-string that
holds one becomes a replacement field of that constant, which formats as
the string itself. A raw literal has no escapes. The constant goes on as
TypedConstant::StrPoints and as TyIR's Constant::StrPoints, an index
into TyirModule::str_points, and codegen emits it as any string
constant, at the width of its widest code point. Compile-time readers of
literal text, such as open modes and % formats, take it as a value
that is not a literal.
6. typy-stdlib¶
typy-stdlib holds the Python sources of the standard library modules
TyPy implements (ADR-0007), embedded in the typy binary with
include_str!, and the declarations of the private _typy module. It is
created with the first module written in Python, sys and math in 0.3.
math is not written in Python either (ADR-0018): its functions take an
int or a float and some any number of arguments, which the 0.3
subset cannot declare. crates/typy-check/src/math.rs lists each name:
a constant with its value, or a function with its MathShape, which
gives the types of its arguments and its result, and the Python version
that added it, so a name newer than the program's floor is E0425.
import math binds a built-in module object, which runs no code and
prints as <module 'math' (built-in)>; TypedProgram::builtin_modules
lists the ones a program imports, after its own modules. A call is
TypedExprKind::MathCall, checked by body/math_call.rs with CPython's
messages for an argument of a type it refuses for every value, and
lowered by lower/math.rs, which converts each argument as CPython does,
an int to a float for a real-number parameter, and calls the runtime
function (section 14). fsum, dist, prod and sumprod (LIB-12) are
checked by body/math_iterable.rs, which turns each iterable into a list
with TypedExprKind::ListOf and takes int, bool or float items, or a
union of them (LIB-15); prod of other items, or with another start, is
E0201, planned for 1.0 (LIB-21). lower/math_iterable.rs gives prod and
sumprod whose type is int | float an int when
typy_rt_math_has_float finds no float item, and otherwise the float
the runtime computes; prod with a bool start gives the start for no
items, and otherwise the product of the start as an int, so its type is
bool | int or bool | int | float.
sys is built the same way. crates/typy-check/src/sys.rs sorts every
public name of CPython 3.12 to 3.15's sys into a SysName: a
SysValue (argv, stdout, stderr, stdin, version_info), exit, a
switch of lazy imports (E0202, MOD-09), a SysConstant folded into the
program, getrecursionlimit and setrecursionlimit, which
TypedExprKind::RecursionLimit reads and sets in the runtime, or a name TyPy
does not provide (E0202). body/version_info.rs folds sys.version_info
written out into constants where it is compared with a tuple of ints,
indexed or read by field (ADR-0029). Held as a value it is
Type::VersionInfo, the runtime's one immortal object of type
sys.version_info, type ID 20, which typy_rt_sys_version_info gives and
writes as CPython 3.15.0 writes it (LIB-16). An index, a slice, a
comparison other than is, len, a loop, + and unpacking read it through
version_tuple, which makes TypedExprKind::VersionRead: the value is
evaluated, then the constant tuple (3, 15, 0, 'final', 0) is what the
operation reads, so the tuple's own checks apply; a comparison with a tuple
of int literals is made when compiling, its result the read's constant,
and a field is item i of the tuple. body/builtin_module.rs holds what the two
modules share: BUILTIN_MODULES, the ModuleFunction and
BuiltinValue a name gives, and Program::builtin_lookup, whose
refusals point at the name. A value of sys is TypedExprKind::SysValue,
lowered by lower/sys.rs to a runtime call whose result is borrowed,
since the runtime keeps the list and the two files for the whole run.
sys.exit(code) is TypedExprKind::SysExit, lowered to raising
SystemExit built with the code as its argument. sys.stdout and
sys.stderr have the type Type::TextFile, shown as TextIOWrapper,
whose write and flush are methods of the method table; print
takes one as file, and file=None is no file.
open is a builtin checked by body/open_call.rs (ADR-0019). The mode
must be a string literal, which crates/typy-check/src/open.rs reads as
CPython's _io.open does: open_access gives an OpenAccess (Read,
Write, Append, Create) or a ModeRefusal, either Fails with
CPython's ValueError text (E0404), and an OpenMode of the access and
whether the mode is binary; a mode with + is one of ReadPlus,
WritePlus, AppendPlus and CreatePlus, numbered with the bit of +
above the access. A binary mode gives Type::BinaryFile with the
BinaryKind of CPython's class, Raw for FileIO with the literal
buffering=0 (ADR-0031); its mode is the one CPython 3.15's FileIO
shows, such as rb+. binary_file refuses encoding, errors and
newline with CPython's ValueError text, buffering=1, and a
buffering that is not a literal (E0201, LIB-19). The methods of a text
file serve a binary one, typed by Type::file_line: bytes in place of
str. A mode that is not a literal is a str in OpenOptions::mode, and
the file is a TextFile; typy_rt_open reads it as open_access does,
with the same ValueError texts, and raises TypeError for one with b
(ADR-0031). A file argument of type
int is a descriptor: the file is Type::DescriptorFile, text or of a
BinaryKind, whose name is FileAttribute::Descriptor, an int, and
it lowers to typy_rt_open_descriptor. closefd other than the literal
True is OpenOptions::closefd; the literal False with a path is
E0404, and any other value sets a bit of the word typy_rt_open reads,
tested when the program runs. A bool file is E0404, since CPython 3.13
and later warn for it.
body/open_options.rs checks the arguments after the mode into an
OpenOptions: buffering, an int kept unless it is the
literal -1; encoding and errors, string literals resolved by
codec_argument as for str.encode, so a file has the codecs utf-8,
ascii and latin-1 and the six handlers, numbered as the runtime takes
them, and the encoding as written for the file's repr; newline, a
str | None, whose literal CPython refuses is E0404. closefd other than
True is E0201 (LIB-14), and opener other than None is E0201 for 1.0
(LIB-18). The call is TypedExprKind::Open { path, mode, access, options }
of type Type::TextFile, the
type of sys.stdout, whose methods add read, readline, readlines,
writelines, close, and the methods of LIB-14: fileno, isatty,
readable, writable, seekable, seek(cookie, whence=0), tell and
truncate(pos=None), whose pos is ParameterType::IntOrNone, an int
or the literal None. name, mode and closed are
TypedExprKind::FileAttribute; calling one is E0401, and another name of
a text file is E0412. A text file is iterable: Type::iterable_item
gives str, a for loop and iter() make a runtime iterator over it,
and next(file) is next(iter(file)). TypedStmt::With carries a
WithProtocol: Methods { enter, exit } for a class, or TextFile,
which enters with typy_rt_text_enter and leaves with
typy_rt_text_close. An OSError the runtime raises for a file opened from
a descriptor holds the descriptor as its third argument, CPython's
filename for it (ADR-0035).
The builtins (LIB-01, LIB-02) are not written in Python. print, min
and max take any number of arguments, which the 0.1 subset cannot declare
(EXP-17), and sorted, sum, any and all take any iterable, which
needs generics (0.4), so ADR-0007's "written around it" applies: each
builtin is a Builtin value that typy-check types from its own table and
typy-lower lowers to runtime calls (section 7.3). A builtin moves to
builtins.py when the features it needs arrive.
7. typy-check¶
7.1 Types¶
Type is an enum. Its variants are Int, Float, Bool, Str,
NoneType, Tuple(Vec<Type>) and List(Box<Type>) from 0.2, and Error.
Error is the type of an expression that already has a diagnostic; every
check accepts it silently, so one cause gives one error (DIA-04). A type
built from other types holds them, so Type is Clone and not Copy, and
two types are equal when their structure is. Dict, Set, Bytes and
Class follow in 0.2; Union, TypeParameter and Callable in 0.4 and
1.0.
| Method | Behaviour |
|---|---|
is_assignable_to(&self, target) |
True when equal, or either side is Error; a tuple item by item. An Int is not assignable to Float (TYP-09) |
can_test_equality(&self, other) |
True for the same type, two numbers, or tuples of one length whose items can (EXP-06) |
can_order(&self, other) |
The same, except that None cannot be ordered; inside a tuple a pair of None items is accepted, because CPython orders tuples by the first pair of items that differ and two None items never do |
has_error(&self) |
True when the type is or holds Error |
Display |
The Python spelling, such as int or None |
with_article(&self) |
The spelling with its article, for messages: an int, a str, None |
Symbol
| Field | Type | Default | Rule |
|---|---|---|---|
id |
SymbolId |
none | Unique in the SymbolTable |
name |
Name |
none | A Python identifier |
kind |
SymbolKind |
none | Local, Parameter, ModuleVariable, Function or Builtin |
ty |
Type |
none | Never changes after the symbol is created (TYP-03) |
span |
Span |
none | Where the name is first bound |
Scope
| Field | Type | Default | Rule |
|---|---|---|---|
id |
ScopeId |
none | Unique in the SymbolTable |
kind |
ScopeKind |
none | Module or Function in 0.1 |
parent |
Option<ScopeId> |
None for a module |
A function scope's parent is its module |
names |
BTreeMap<Name, SymbolId> |
empty | One symbol per name |
FunctionSignature
| Field | Type | Default | Rule |
|---|---|---|---|
parameters |
Vec<Parameter> |
empty | Each with name, ty and default: Option<Constant> |
returns |
Type |
none | NoneType for -> None |
A default value is a constant in 0.1, which makes "evaluated once, when the function is defined" (EXP-16) hold trivially.
TypedProgram
| Field | Type | Default | Rule |
|---|---|---|---|
symbols |
SymbolTable |
none | All scopes and symbols |
functions |
Vec<TypedFunction> |
none | Sorted by qualified name; the module body is one of them |
entry |
FunctionId |
none | The body of __main__ |
A TypedFunction holds its FunctionSignature and its body as typed
statements, in which every expression is a TypedExpr with a kind, a
ty and a span.
Tuples (DAT-08, SMT-02, TYP-06) add these forms. TypedExprKind::Tuple
holds a display's items. TypedExprKind::TupleItem { tuple, index } reads
an item at a position known when compiling: the index is an integer
literal, or a negated one counted from the end, and E0307 reports any other
index or one outside the tuple, a program CPython would stop with
IndexError. TypedStmt::Assign stores into AssignTargets, each a
Name(SymbolId) or a Tuple of targets; unpacking needs a tuple with one
item for each target, or E0308. An annotation tuple[A, B] names each item
and tuple[()] the empty tuple; a bare tuple is E0408. + joins two
tuples, len gives a constant, and ==, !=, <, <=, > and >=
compare tuples whose items compare.
Type::Bytes is bytes (DAT-06): a literal such as b"a\x00", an
item read at a position, an int from 0 to 255, slices, len, +, *,
comparisons, in of an int or of bytes, for over its int items,
and str and repr, which write b'...'. bytes() is empty, bytes(n)
holds n zero bytes, and bytes(values) takes any iterable of int;
bytes of a str without an encoding is E0404, and bytes(text,
encoding, errors) is text.encode(encoding, errors).
str.encode and bytes.decode take the encoding and the errors as string
literals, by position or keyword, so the checker resolves them when
compiling, as codecs.lookup would, and passes their numbers: the codecs
utf-8, ascii and latin-1 under every alias CPython gives them, and
the error handlers strict, ignore, replace, backslashreplace,
xmlcharrefreplace for encoding only, and surrogateescape. A name TyPy
does not provide, or one that is not a literal, is E0404, so a program
never meets a LookupError that CPython would not raise.
Type::TupleOf(T) is tuple[T, ...] (DAT-14), an immutable sequence of any
length, written tuple[int, ...]; ... anywhere else in a tuple annotation
is E0408. It is a list object at run time, under the tuple type ID, so it
reads as a list does: items at computed positions, slices, len, in,
+, *, comparisons, for, and the builtins of one iterable, and it has
the methods count and index, Method::TupleCount and
Method::TupleIndex. Type::sequence_item gives the item type of either.
A fixed tuple whose items all have type T is accepted where a
tuple[T, ...] is expected: BodyChecker::converted wraps it in
TypedExprKind::TupleOf, at an annotated or typed assignment, an argument,
a return, an item of a display, and beside a tuple[T, ...] in a
comparison, in or +, so t += (3,) gives a tuple[T, ...]. A for
over a fixed tuple reads it the same way when its items have one type, and
is E0411 otherwise, because the loop variable would need a union.
tuple(source) is TupleOf(ListOf(source)), and tuple() the empty
tuple[()], or an empty tuple[T, ...] where that is expected.
Lists (DAT-07, DAT-11, TYP-05, EXP-07, CST-03) add TypedExprKind::List,
ListItem { list, index } and Method { receiver, method, arguments }, the
target AssignTarget::Item { list, index }, and TypedStmt::ForEach. Every
item of a display has the first item's type, or E0409. An empty display has
a type only where the context gives one, through
expression_expecting(expr, expected): an annotated or already typed
variable, a return, or an argument; anywhere else it is E0410 (TYP-05).
Method::of(receiver, name) lists the methods of the built-in types; any
other name is E0412 (LIB-03). On a list they are append, pop, insert,
extend, remove, index, count, clear, copy, reverse and sort;
on a tuple[T, ...], count and index. On a str they are the methods
that read or rebuild text without Unicode tables: find, rfind, index,
rindex, count, startswith, endswith, split, rsplit,
splitlines, partition, rpartition, join, strip, lstrip,
rstrip, removeprefix, removesuffix, replace, center, ljust,
rjust, zfill, expandtabs and isascii, the case methods upper,
lower, casefold, swapcase, capitalize and title, and the tests
isalpha, isalnum, isdecimal, isdigit, isnumeric, isspace,
isupper, islower, istitle, isidentifier and isprintable. On a
bytes they are the same methods with bytes in place of str, except
casefold, isdecimal, isnumeric, isidentifier, isprintable and
encode, with decode and hex added; what a search looks for is a
bytes, where CPython also takes an int.
Method::signature(receiver) gives a MethodSignature: each Parameter
has CPython's name, a ParameterKind, Positional, Either, Keyword
or Rest, which takes every remaining positional argument, none or many,
whether a call must give it, and a ParameterType: an exact type,
StrOrNone for a str or None, Affixes for a str or a tuple of
them, Strings for any iterable of str, or Items(T) for any iterable
of T. method_call binds the positional arguments in turn, then each
keyword by name, and keeps them in source order as Argument { parameter,
value }; a keyword for a
positional-only parameter, a name the method does not have, a parameter
given twice, or a missing one is E0404. for over a
call of the builtin range stays ForRange; any other for is ForEach
over a LoopSource, binding a target that may unpack. A source is
Items of a list, a str, a set or a dict, Reversed of one,
Range { start, stop, step, reversed } for range(...) and
reversed(range(...)), Enumerate { source, start }, or Zip { sources,
strict }, and enumerate and zip take sources, so enumerate(zip(a, b))
nests. A range source has one to three int arguments, E0404 otherwise,
and is read wherever a source is: a comprehension, list, set, dict,
sorted, sum and the other builtins of one iterable. A range(...)
anywhere else is a range value, Type::Range, also written as an
annotation (DAT-17): len, indexing and slicing give an int and a
range, in takes an int, == and != compare two ranges, and for,
reversed and iter read it. Ranges are not ordered, E0407. A for over a
range value keeps the range loop's counter, limit and step, read from the
object by typy_rt_range_bound. called_builtin names a
builtin only when no local or program function of the same name hides it,
as a call resolves it, so a program's own range is called by for. Over a
fixed tuple whose items differ, and over any other value, a for is E0411.
enumerate, zip and reversed outside a for, a comprehension or an
iterable argument are E0201, planned for after 1.0 (CST-09). x in
xs needs an x of the item type; on a str it is a substring test and
needs a str; on a tuple it compares with each item, which must all compare
with x for equality, or E0407, and two lists compare with == when
their item types are equal, because the runtime compares items by the
list's own item kind. A list holds items of any type: int, float,
bool, None, str, lists and tuples of these.
In a comparison chain, an empty display takes its type from a neighbouring
operand, so xs == [] and x in [] need no annotation; [] == [] is still
E0410. + joins two lists of one type, and * repeats a list by an int on
either side; lists of different item types are E0401, because the runtime
stores items by the kind of the item type. Two lists order with <, <=,
> and >= like two tuples, item by item.
The builtins of one iterable (LIB-02), sorted, sum, any, all, and
min and max of one argument, add TypedExprKind::IterableBuiltin {
builtin, iterable, option }. The option is the one optional value each
takes: reverse, a bool, for sorted; start, by position or keyword,
for sum; default, of the item type, for min and max. key needs a
function value and is E0201, planned for 1.0 (FUN-05); any other keyword is
E0404. iterated_item gives the item type of the iterable, which for also
uses: a list's item type, E0201 for a str or a tuple until they can be
iterated later in 0.2, and E0411 for anything else. sorted, min and
max need items that order. sum adds int or float items; a
list[float] needs a float start, E0404, because CPython returns the
int 0 for an empty list without one, a type the call could not have. A
list of bool is E0406, as all arithmetic on bool is (DAT-03).
round(number, ndigits) gives an int for an int, and for a float
an int without ndigits and a float with it, as CPython does;
ndigits is an int, by position or keyword, and round of a bool is
E0406. list() has the expected type, as [] does, and list(source) is
TypedExprKind::ListOf(source), a new list of every value the loop source
gives. The builtins of one iterable take any loop source the same way:
iterable_argument passes a list on as it is and wraps any other source,
a str or enumerate, zip or reversed, in ListOf, as CPython's
sorted builds a list. list.sort takes reverse by keyword only, which
MethodSignature::keyword names. Inside a display without an expected type,
an empty display takes the type of the first other item, so [[1], []]
needs no annotation.
Slices (DAT-12) add TypedExprKind::Slice { value, lower, upper, step }, each
bound optional and an int, or E0404. A str or a list can be sliced, and
the slice has the sliced type; any other value is E0401. A tuple is sliced
with integer literals, or missing bounds, E0307 otherwise, because each item
can have its own type: TypedExprKind::TupleSlice { tuple, positions }
holds the positions, worked out as CPython's PySlice_AdjustIndices does,
and a zero step is E0307, since the slice would always raise. A slice
inside a tuple index, grid[1:2, 0], is E0401. Assigning to a slice is
E0201, planned for 0.3 (DAT-13).
Sets (DAT-10, DAT-15, TYP-05, EXP-07, CST-03) add Type::Set(T), written
set[T], and TypedExprKind::Set(items) for a display and SetOf(source)
for set(source). An item of a set needs a hash: Type::is_hashable is
false for a list, a set, and a tuple that holds one, and such an item type,
in a display, in set(source) or in a set[...] annotation, is E0413.
Every item of a display has the first item's type, or the expected item
type, or E0409. A display always has items, since {} is a dict; set()
has the expected type, from an annotation, a return, an argument, or a
neighbouring operand of a comparison as [] has, and is E0410 elsewhere.
set(source) takes any loop source, as list(source) does. |, &, -
and ^ combine two sets of one type, and the augmented forms change the
set in place; sets of different item types are E0401. == and != compare
sets of one type, and <, <=, > and >= test proper and plain subsets
and supersets. x in s needs an x of the item type. for, len,
bool, print, str, repr and the builtins of one iterable read a set
in its table's order; reversed of a set is E0411, because a set has no
order to reverse. On a set the methods are add, remove, discard,
pop, clear, copy, union, intersection, difference,
symmetric_difference, update, intersection_update,
difference_update, symmetric_difference_update, issubset,
issuperset and isdisjoint. union, intersection, difference,
update, intersection_update and difference_update take any number of
arguments, a Rest parameter, and the other methods that combine sets one;
each argument is Items(T) of the set's item type (DAT-15).
items_argument checks it: a display, [] or set() takes the wanted
type, a set, a list or a tuple[T, ...] is passed as it is, and any other
loop source, such as a str or zip, becomes ListOf(source), whose
items the runtime reads in the order CPython reads the iterable. A source
whose items have another type is E0404.
Dicts (DAT-09, TYP-05) add Type::Dict(K, V), written dict[K, V], and
Type::DictView(View, K, V) for keys(), values() and items(), written
dict_keys[K, V] and so on, as TypedExprKind::Dict { keys, values },
DictOf(source) and DictItem { dict, key }, and the target
AssignTarget::Entry { dict, key }. A key needs a hash, E0413. The keys of
a display have one type and the values one type, or E0409; {} and
dict() take the expected type as [] does, also beside a dict in a
comparison, and are E0410 elsewhere. dict(source) takes a dict, which is
merged, or any loop source of two-item tuples, whose pairs are set in
order; any other item is E0404. Keyword arguments, with or without a
source, give TypedExprKind::DictWith { source, names, values }, a dict of
str keys whose values have the source's value type, the expected one or
one type, E0409 otherwise; a name given twice is E0404, as CPython's
keyword argument repeated. dict.fromkeys(source, value) gives
DictFromKeys, of the source's items and the value's type, or None
without a value; it takes no keywords. Both are checked in
body/dict_call.rs. Lowering in dict_call.rs keeps CPython's order:
every argument is evaluated, then the source is read through
each_value_after, which calls a class's __iter__ only then, and each
keyword is set last, as dict_vectorcall does. d[k] and d[k] = v
check the key against the key type and the value against the value type,
E0404 and E0402, and
d[k] op= v reads and writes the one entry. in finds a key in a dict, a
key, a value or a (key, value) pair in a view; for, len, bool,
print and the builtins of one iterable read a dict's keys or a view's
items in insertion order. | merges two dicts of one type and |= updates
in place; == and != compare dicts of one type, and dicts do not order,
E0407. A view of keys or items acts as a set of its items, a key or a
(key, value) tuple (DAT-16): |, &, - and ^ with a set, a view or a
list of the same items give set[T], E0413 for items without a hash and
E0401 for other items, and each comparison with a set or such a view gives
bool. Views of values compare only with == and !=, by identity. Both
are checked in body/view_operator.rs and lowered in view_set.rs.
reversed of a dict or a view reads it from its last entry: lowering
starts at typy_rt_dict_last_entry and steps with typy_rt_dict_previous,
which raises RuntimeError when the size changed, as CPython's
dictreviter_iternext does; it does not notice keys changed at the same
size. The methods are get and
setdefault with a default, pop with or without one, popitem, keys,
values, items, update, clear and copy. get and setdefault
without a default would give V | None, E0201 until TYP-06 brings it.
update takes Pairs(K, V): a dict, which is passed as it is, or any
source of pairs of the dict's types. A dict passed to a set method or to
set() stays a dict, so the runtime takes CPython's dict steps.
Generic functions (GEN-01 to GEN-03, ADR-0023) keep their syntax as
TypeParameters on Stmt::FunctionDef. check::definition::collect_functions
resolves each into a generic::TypeParameter, with its bound or its
constraints, owned by the definition's span, and checks the signature with each
name in AnnotationScope::parameters as Type::Parameter, an opaque type
assignable only to itself and, through through_union_base, offering its
bound's attributes and methods. check::generic_body checks the body with each
unconstrained parameter opaque and once for each combination of the constraints
of the others, then keeps the function as a generic with no body, which lowering
compiles to an empty stub. body::generic_call types the arguments once with
what that look reports dropped, solves the parameters with generic::infer,
joining two solutions into a union, checks bounds and constraints, and asks
Program::request_copy for the copy of the arguments, which
check::generic_copies::check checks at the end of the module as an ordinary
function with Program::type_arguments naming the arguments. A copy made inside
64 others is E0428. A copy of another module's generic, a function or a method
of a generic class, is checked in that module's names, which
Program::context_of kept when the module ended, from the definitions
ModuleDefinitions holds for it, so its body reads its own module's functions,
classes and variables (GEN-07). The module body's first look gives back the
copies it made, with the function IDs they took. Generic classes (ADR-0028) are
noted before any signature is read, by
check::generic_class::note_generic_classes, in body::ClassCopies
behind a RefCell the annotation scope shares, so Box[int] in a signature
asks for a copy, which takes a class ID after the module's classes.
ClassCopies::applied gives the generic class's own instances for its own
parameters and otherwise the copy's, an InstanceClass whose generic names
the class and the type arguments. The generic class's methods are generic over
its parameters; Program::make_class_copies adds each copy's ClassInfo, with
its attribute types substituted, which Program::attribute_ty also reads from
the generic class while the copy's are unknown, and a copy of each method,
checked as function copies are. Program::substitute maps a copy to the copy
for the substituted arguments, so a recursive class such as Node[T] refers to
its own copies. A call of a generic class solves its type arguments from
__init__ in BodyChecker::class_to_call. A static method that names no type
parameter of its class, in its signature or in its body, is left out of
finish_generic_class's copies, so it is compiled once; any other static or
class method called through the generic class takes the type arguments its
arguments give, solved by BodyChecker::solved_arguments, and a class method
then runs the copy's method with the copy as cls. Class[arguments].name(...)
calls the method of the copy BodyChecker::applied_class names. A class
variable or a default of a generic class is one module variable, Class.name,
which a copy's ClassInfo, cloned from the generic class's, names as the
generic class does, so every copy shares it as every instance shares it in
CPython. One whose annotation names a type parameter of the class, which
check::class::refuse_shared_parameter finds, is E0201, planned for 1.0
(GEN-11), and class_values makes no module variable for it. A generic class
may derive from a class of the program (GEN-08, ADR-0040): its
GenericClass::instance holds the chain class_types gives it, and
ClassCopies::applied gives each copy that chain through copy_ancestors,
each copy in it made again for the copy's arguments by
ClassCopies::substitute, which Program::substitute now calls. A copy's
ClassInfo keeps the generic class's base, so its layout and method table
extend the base's. isinstance with a generic class does not narrow a
variable, since Program::is_opaque_class holds for the generic class
(GEN-14); for the same reason @dataclass refuses the comparison methods of
a generic dataclass derived from a dataclass, and a generic exception class or
protocol is E0201 from refuse_generic_kind (GEN-12, GEN-13). A base that
names a copy, Box[int] or Box[T], is read after the module's generic classes
are noted, by check::generic_base::derive_from_copies, which goes over the
classes in source order, gives each class whose base is a copy, and each class
derived from one, the chain of the copy, and gives a generic class its new
GenericClass::instance; class_types passes over such a base, and a bare
generic class as a base is E0408. Collecting the members reads a copy base
through Classes::get, which gives the generic class's ClassInfo, its
attributes taking the copy's arguments through Classes::as_member_of, and an
override is checked against the base method's signature with the arguments of
each copy inherited_method passed through. make_class_copy gives a copy of
a generic class derived from a copy the copy of that base for its own
arguments, and make_class_copies adds the bridges of the copies it made once
all are, since a base copy may be made after the copy that names it. A class
derived from a copy whose generic class has class methods, and a dataclass
derived from a copy of a generic dataclass, are E0201, planned for 1.0
(GEN-15). Since a class may derive from a copy with a higher ID,
lower::methods::MethodTables::new lays out the tables with each base first,
and the TyIR verifier walks a chain of bases as far as there are classes,
refusing one that comes back to its class. A copy whose arguments hold a
parameter, and the generic class itself, are opaque: lowering lays them out
empty and their methods are stubs. A copy's TyirClass::public names the
generic class, which codegen writes as the type object's base and as its
public field, which typy_rt_object_type returns.
A method with type parameters of its own (GEN-01) is a generic function owned
by its class: check::class::method_definition resolves its parameters in a
scope that adds them to the class's, and notes its name in
ClassInfo::generic_methods. BodyChecker::method_target sends a call of such
a method, through an instance, its class or super(), to
BodyChecker::call_target, which solves the parameters from the arguments
after the first skip and returns the copy and its signature; the call does not
dispatch. A generic method may override one with as many type parameters, in
classes that are not generic: check_override compares it with its own
parameters named as the base's, as_parameters_of, and when the copies of a
module are checked, Program::make_override_copies gives each copy of an
overridden generic method, and the copy of each override for the same type
arguments, a place in their classes' methods under the key name[arguments],
which no Python name can be, so lowering's method tables dispatch a call through
the base to the object's class's copy; the generic stand-ins themselves stay out
of the tables. Other overrides of or by a generic method are E0201, planned for
1.0 (GEN-16). A class method's own type parameters are resolved for each class's
copy in ClassMethodCopies::make, which also lets a protocol parameter make it
generic; a call through a class goes to call_target_with with the arguments of
a copy of a generic class first, copy_arguments_of. Such a class method has no
bridge, so a call through an instance is E0201, and so are type parameters of a
special method or a property, from later_generic_method, all planned for 1.0
(GEN-16). A method of a generic class with type parameters of its own is generic
over the class's first and its own after them, two lists with two owners that
Program::substitute_each replaces in turn; finish_generic_class keeps it out
of the copies' methods, and a call takes the class's arguments from the copy
that holds the method, holder_arguments, solves the method's own by
solve_each, and runs the copy call_target_with makes for the copy of the
class. A static method that names none of the class's type parameters is generic
over its own alone. A generic type alias (SMT-10) is expanded by
check::type_alias before any annotation is read: Pair[int] becomes the
alias's value with int in place of T, and a type parameter of an enclosing
function or class hides an alias of its name.
A protocol (GEN-04, ADR-0023) is a class whose base names typing.Protocol;
check::class::class_types marks its InstanceClass::protocol. It has no
instances: Program::is_opaque_class holds for it, its methods are stubs whose
bodies, often ..., are not checked, and calling it is E0401.
check::protocol::protocol_parameters gives each parameter whose annotation
names a protocol a type parameter of its own, bounded by the protocol and named
Shape of shape, a name no source can write, so def total(shape: Shape) is
def total[S: Shape](shape: S) and compiles to a copy per class. When a type
argument's bound is a protocol, BodyChecker::unsatisfied checks it by its
members: each method, not static, taking the protocol method's parameters by
name with types that accept the protocol's and a default where it has one, then
only parameters with defaults, and returning a type the protocol's accepts; and
each attribute with the same type. A union satisfies a protocol when each member
does. isinstance with a protocol is E0401 (GEN-05).
A protocol as the type of a variable, an attribute, a result or a container
item, and of a parameter of a special method or a property, which
protocol_parameters leaves as it is for such a method, is a protocol value
(GEN-10, ADR-0041), which lowering holds as TyirType::UNION. When a module's
classes are collected, and again after its __init__ methods are checked,
Program::note_protocol_fits notes in ProtocolValues::fits each class that
satisfies each protocol, by Program::class_unsatisfied, which skips a pair
whose attribute types are not all known yet; it does so for each copy of a
generic class make_class_copies makes too. It notes each pair in the table of
protocol_fits, which a FitsScope opens for the check and clears when it
ends, and InstanceClass::derives_from holds for a class and a protocol it
satisfies, so every conversion that takes an instance of a base takes one of a
satisfying class as an Upcast. begin_module gives a protocol no bridges and
makes its dispatchers, <protocol Protocol.name>, one for each method, with
the method's signature, and a getter for each attribute; Program::method of a
protocol gives the dispatcher, so a call through a protocol value, by a method
call, an operator or a builtin such as len, reaches it, and an attribute read
calls the getter through protocol_attribute. Once every module is checked,
write_protocol_dispatchers gives each its body: an EachMember over the
satisfying classes, the most derived first, each arm calling the class's method
through its table with the arguments and the result converted, or reading its
attribute. == and != of protocol values call typy_rt_objects_equal, which
the equal slot of the class that decides answers. A mismatch where a protocol
is wanted takes the reason as its help, from protocol_help. Assigning an
attribute through a protocol value, a class derived from a protocol, a
protocol's properties, static and class methods, class variables and defaults,
a protocol in its own methods' signatures, and a value of a built-in type held
as a protocol are E0201, planned for 1.0 (GEN-17).
Unions (TYP-14, ADR-0027) add Type::Union(members), which Type::union builds
in normal form in union_type: nested unions flatten, None makes the union
optional, a member another member accepts is dropped, the members are sorted by
their debug text so A | B is B | A, and one member is that member. A member
is assignable to its union, and a union to any type each member is assignable
to, through Upcast. Lowering holds a union as its members are when all are
held alike, as the class every member derives from when they are instances that
share one (Type::union_base), and otherwise as TypeId::UNION, an object
whose header says what it is, which the verifier lets any object become and
become again. body::union::through_union_base gives the receiver of an
attribute, a method or raise the shared class, or the common exception class
of a union of exceptions, which an except (A, B) as name binds. str, repr,
type() and a truth test of a TypeId::UNION value call typy_rt_object_str,
typy_rt_object_repr, typy_rt_object_type and typy_rt_object_is_true, which
read the object's type slots; str has a str slot that returns the object
itself. IsInstanceTest::otherwise gives the members no tested class takes in
every case, which the flow state holds in the branch where the test fails, and a
narrowed T | None that a test showed is not None reads as T. A truth test
of a union held as a base whose members test themselves otherwise, or of a
member class whose truth comes from __len__, is E0201, planned for 0.5.
body::each_member checks an attribute, a method call, an operator, an item, a
comparison other than is, or a builtin call other than print, str, repr,
type, isinstance and bool, whose operand is a variable that holds a union
of members with no shared class, once for each member with Fact::Narrowed set
to it, and joins the results with Type::union. TypedExprKind::EachMember
keeps the variable's read and, for each member, a MemberTest with its checked
expression. The test is the member's class, or for an iterator, a traceback, a
module, a file, a type alias, sys.version_info and NotImplemented the
RuntimeKind of the runtime's object. Lowering tests the object with
typy_rt_is_instance against a class, or with typy_rt_object_has_type_id
against the type ID of a kind, for each member but the last, whose branch runs
when none holds; the branches pass their result to one block. Two members that
share a kind, such as two iterators, cannot be told apart at run time, so
member_test refuses the operation with E0201, planned for 1.0 (TYP-19).
A loop, a comprehension or a builtin that reads an iterable over a
variable that holds a union of fixed tuples goes over
body::union_loop::union_tuple_items: an EachMember whose arm for each
member is a list display of that member's items, each converted to the union
of every member's item types, so the loop reads one list (ADR-0034). A
literal index outside some of the tuple members gives those members'
arms TypedExprKind::Raise of IndexError("tuple index out of range"),
which lowering throws in place of a value; an index outside every member is
E0307. Equality
and order refuse a union that is not a variable, which no split reaches. A
container of a union of objects stores each as an object item, and the runtime
compares two objects of different types as CPython does: a built-in type is
equal only to its own objects, and a class's slot, which tests
the other object's class, decides for its instances. A union with a member that
is not an object (Type::is_object) is held as TupleTypes::intern_union, a
tuple of an int tag and a slot for each member, which the descriptor marks
with item kind 11; every slot but the tag's holds the value zero_value gives,
which counting passes over, so owning the tuple is owning its member.
lower::tagged_union tags a member in widened_to_union, reads a member's
slot, or retags a narrower union, in narrowed_union, both through an optional,
and switches on the tag in tag_switch for str, repr, truth, type(),
isinstance, whose IsInstanceTest::members give each member's outcome, and an
operation checked for each member. The runtime's ItemKind::Union holds a union
item as the member its tag names, and values_equal compares two members as
CPython does: numbers by value, exactly, tuples item by item, and other types
never equal. An expected union converts the value given for a method's argument,
an item, an entry and a needle of in. A single target's value is checked as
the target's type already; one value assigned to several targets, or a tuple
unpacked into targets, may reach a target of another form, a union or an
optional of the value's type. converted_target then gives the target
AssignTarget::Converted: lowering stores the value in a hidden variable,
<converted N>, and assigns the target that variable's read converted to the
target's type, as converted builds it for a single value (TYP-16). A
variable, an instance attribute, a list item and a dict entry take it.
Optional values (TYP-06, TYP-13, ADR-0011) add Type::Optional(T), written T |
None or None | T, which Type::optional builds so that T is never None
or optional itself. is_assignable_to accepts None and a T where T | None
is declared, and matches compares types item by item for everything a
container stores, so list[int] is not a list[int | None]: a stored item is
never converted. BodyChecker::converted makes the conversion explicit as
TypedExprKind::Wrap(value) wherever a value meets a declared type, at an
assignment, an argument, a return, an item of a display and a branch of a
conditional expression, and converts a tuple display item by item. An instance,
or an optional one, that meets the type of a class it derives from becomes
TypedExprKind::Upcast(value), and so does the receiver of an inherited method
(CLS-04). A display, and a conditional expression with a None branch, join
their items' types, so [1, None] is list[int | None]. An optional value is
compared with None by is, tested for truth and written as text. == and
!= take it with None, with another optional value or with a value of its
base: None equals only None, and two values compare as their base does,
which lowering's optional_equality tests one side at a time (EXP-06).
Ordering, operators and methods on it are E0407, E0401 and E0412, with a help
that says how to narrow it. True, False and None are each the only object
of its value, so two bool or bool | None values also compare by is:
lowering compiles it as == of two bool values, after testing an optional one
for None, which is never a bool (EXP-08). The flow state of definite
assignment holds Fact::Assigned(symbol) and Fact::NotNone(symbol), joined by
intersection, so a fact holds only where it holds on every path. test_facts
gives the facts a test shows when it holds and when it fails: x is not None
and a truth test of x show x is not None when they hold, x is None when
it fails, not exchanges them, and and and or combine their operands'. An
if and a while add them to each branch, the operands of and and or, the
branches of a conditional expression and the clauses of a comprehension after a
condition see what the test before showed, and an early return, break or
continue leaves what the other branch showed. Assigning a value that is not
None to an optional variable shows it is not None, and assigning None or
an optional value forgets it. A loop forgets the facts of every variable its
body assigns before it reads its test or binds its target, since the next pass
starts from what the body left. A name whose fact holds reads as
TypedExprKind::Unwrap(local) of type T. dict.get(key) without a default
gives V | None, or V when V is optional, as get(key, None);
setdefault(key) without a default needs an optional V, E0404 otherwise.
Comprehensions (EXP-12) add TypedExprKind::Comprehension { element, value,
generators }, a value only for a dict, and each Generator { target,
source, conditions }. A clause's source is checked before its target is
bound, so the first is read in the enclosing scope and each later one sees
the targets before it, as CPython reads them. Each target name gets a new
variable of the comprehension, and the scope's names and definite
assignment are restored after it, so a comprehension variable is not seen
after it, E0205, and leaves an outer variable of the same name as it was.
The conditions are truth tests. A list comprehension makes list[T] of
its element's type, a set comprehension set[T], whose item needs a hash,
E0413, and a dict comprehension dict[K, V]; an expected type gives the
element, as it gives a display's items, so [[] for _ in xs] takes an
annotation. An async for clause is E0201, as async is.
An assignment expression (EXP-13) is Expr::Named { target, value } in
the syntax tree and TypedExprKind::Named { target, value } after
checking. collect_locals and module_variables bind its name through
visit_statement, which reaches every expression of a statement, so a
:= inside a comprehension assigns the variable of the function or module
around it, as CPython scopes it. body/named.rs checks the value against
the variable's declared type as an assignment does, and the expression
gives the value itself: a value wrapped into a T | None is unwrapped, and
assign_name wraps it again as it stores it. narrowable_variable sees
through it, so (m := f()) is not None narrows m. Each function lists
its targets in TypedFunction::named_targets, which lowering adds to the
assigned symbols, since no statement shows them, so a parameter assigned
by := is not borrowed. The subset walker keeps the names enclosing
comprehensions bind and whether a comprehension's source is being walked,
and reports CPython's SyntaxErrors as E0103: rebinding a comprehension
variable, := in a source, and __debug__ as the target.
A match statement (CST-06) is Stmt::Match { subject, cases }, each
MatchCase holding a Pattern, a guard and a body, as CPython's ast
module has them. body/matching.rs writes each pattern as the test it
stands for and checks that test as an expression: a value pattern is
subject == value, a singleton subject is None, a capture or an as
name an assignment of the subject, and p | q the alternatives, whose
assigned variables meet. The facts a test gives hold in its guard and
body, and the facts of its failure hold in the later cases when a case is
one test without a guard, so case None: narrows an optional subject in
the cases after it. A local subject is tested as it is; any other subject
is first stored in a hidden local named <match N>, which no program can
write. The result is TypedStmt::Match with a TypedCase per case, whose
TypedPattern is Test, Bind, All or Any. A sequence or mapping
pattern (body/pattern_sequence.rs, body/pattern_mapping.rs) first
tests the subject with isinstance_test against list, tuple and
range, or dict, which narrows a union or an optional subject; a type
no class takes is E0407. A sequence pattern then tests the length, with
the built-in len called directly so a function of that name in the
program does not change it, or compares a fixed tuple's length when it is
checked, and matches item i as subject[i], counting from the end after
a starred name. The starred name binds a list: a slice, list() of a
slice, or for a fixed tuple a list display whose type is the union of the
items it takes. A mapping pattern tests key in subject and matches
subject[key]; **rest binds a dict comprehension over the items whose
key is not one of the pattern's. A class pattern (body/pattern_class.rs)
resolves its class as isinstance does, tests the subject the same way,
then matches each sub-pattern against subject.attribute. A positional
sub-pattern takes its attribute from ClassInfo::match_args, which class
collection reads from a __match_args__ = (...) of str literals in the
class body, the nearest base's applying when a class has none; for the
nine built-in classes CPython names, one positional sub-pattern matches
the subject itself. More positional sub-patterns than the class takes,
and two for one attribute, are E0401, since CPython raises TypeError
for them on every instance. @dataclass writes __match_args__ from the
fields unless match_args=False is given or the class assigns its own,
and DataclassInfo::match_args carries the names to the class's
information. A part that a nested pattern tests is stored
first in its own hidden local, so it is read once and narrows as a
variable does. Lowering gives each case a block its pattern jumps to when
it fails, so the cases are tried in order and a subject no case matches
goes on after the statement. body/pattern_coverage.rs splits the
subject's type into members, each member of a union, None of an
optional and True and False of a bool, and finds the members no
case without a guard covers, reading the patterns without their values
and resolving a class pattern's class with type_object_named, which
reports nothing. When none is left, the match cannot end without
running a case, so no assigned state falls through it and a function
returning from each case does not reach its end; otherwise a subject of
two or more members gives W0401 naming the members left, unless checking
the match reported an error (CST-07, ADR-0033). The subset walker
reports CPython's compile-time SyntaxErrors for patterns as E0103: a
capture or wildcard before the last case or alternative, alternatives that
bind different names, a name bound twice, __debug__
as a target, an f-string as a value, two starred names in one sequence
pattern, a keyword repeated in a class pattern, and a literal key repeated
in a mapping pattern, where 1,
1.0 and True are one key (subset/pattern_key.rs).
Classes (CLS-01, CLS-02, CLS-08, ADR-0012) are collected before any
signature: collect_classes gives each class statement at the top of the
module a ClassId in source order, and an annotation names a class through
an AnnotationScope. A bare class name may be used only where the class
statement has already run when CPython evaluates the annotation: in a
signature after it, never in its own class. Any other use is E0306, with a
help that quotes the name. A quoted annotation, "Node | None", which the
parser reads into Expr::Quoted, may name any class, and a program whose
floor is 3.14 or later may name any class bare, since those versions
evaluate annotations only when they are read. collect_members gives each
class its ClassInfo: the attributes the class body declares with an
annotation, then those __init__ assigns on its first parameter, in source
order, and its methods. A method is a Definition whose first parameter is
the instance, Type::Instance(InstanceClass), without an annotation, and
whose qualified name is Class.method. __init__ may leave out its return
annotation and returns None, E0405 otherwise; __str__ and __repr__
take only self and return a str, E0404 otherwise; any other name with
two leading and trailing underscores is E0201, planned for 0.4 (CLS-07).
A second class, attribute or method of one name is E0202, as is a class
whose name is also a function or a variable.
Each __init__ is checked first, so the attributes it assigns without a
declaration take the type of their first value; a first value of None
gives none, E0410. An attribute whose type is still unknown where another
__init__ reads it is E0310, with a help that declares it. Inside
__init__ the flow state also holds Fact::Attribute(index) for each
attribute assigned on every path, and the instance is used only to assign
and read assigned attributes until every attribute holds a value: a read
before the assignment, any other use of self, and a return or the end
of __init__ with an attribute unassigned are E0309. So no program reads
an attribute that holds no value. Calling a class checks the arguments of
its __init__ after the instance, or none when it has no __init__, and
gives TypedExprKind::New { class, init, arguments }. object.name reads
TypedExprKind::Attribute { object, index }, object.method(arguments)
is a Call of the method with the instance as its first argument, and an
assignment to an attribute is AssignTarget::Attribute; a name the class
does not have is E0414 (CLS-02). An instance is equal only to itself, has a
hash and no order, and is true; is compares two instances, or optional
instances.
An f-string (LEX-07) is Expr::FString in the syntax tree: its literal
text and ReplacementFields, each with its value, its debug text for
{x = }, its conversion and its format specification, itself parts. The
checker makes TypedExprKind::FString of FStringParts, each a literal or a
FormattedValue; {x = } becomes its text and a field with the conversion
!r unless the field has a conversion or a specification, as CPython does.
A field formats a value of any type.
Type objects (CLS-12) are Type::TypeObject(Some(TypeObjectOf)), written
type[int], or Type::TypeObject(None), written type, when the type is
known only when the program runs. type(x) is TypedExprKind::TypeOf(x).
The type object of an instance of a class that no class of the program
derives from is the class's, and that of an optional one NoneType or
its base's; an instance of a class with a subclass, and an exception, give
the class the object holds when the program runs. A class name,
or one of int, float, bool, str, bytes, list, tuple, set,
dict, object and type read as a value, is TypedExprKind::TypeObject,
and __name__ of a type object is TypedExprKind::TypeName; any other
attribute is E0412. Two type objects compare with is, is not, == and
!=, have a hash and no order, and are true; a display of different type
objects holds type. The annotation type holds any type object and
type[X] the one of X. type with three arguments makes a class while
the program runs, E0202.
Exceptions (ADR-0013) are Type::Exception(ExceptionClass), written with the
class name. ExceptionClass::Builtin holds a BuiltinException, one of the
classes of EXC-02 and EXC-10, each with its CPython base.
ExceptionClass::Class holds a shared ProgramException: a class of the
program, its ancestors that are classes of the program, nearest first, and the
built-in class its chain reaches. derives_from compares the IDs of the
program's classes and walks the built-in chain, so an exception is assignable
where its class or any base is declared without the class table. A class's base
is resolved in source order by class_types, which gives every class the type
of its instances: object, a built-in exception class, or a class of the
program defined before it, named by its name or as module.Class (CLS-03,
CLS-04). A class derived from an exception class is an exception type; any
other is Type::Instance(InstanceClass), which holds the class, its name and
its ancestors that are classes of the program, nearest first, so
is_assignable_to accepts an instance where its class or any base is declared
without the class table, as for exceptions, and two instances of related
classes compare with == and is. A base defined after the class, or the
class itself, is E0306, UnicodeDecodeError and UnicodeEncodeError are
E0417, any other base, such as int or list, is E0201 (CLS-04), and several
bases are E0201, planned for after 1.0. A class whose base is the program's
holds its base's attributes first, counted in ClassInfo::inherited, and the
type of an inherited attribute is read from the class that declares it; a
subclass may declare an inherited attribute again only with the type its base
declares, E0202 otherwise and when the base assigns it without declaring it
(ADR-0021). A method is looked up in the class, then in each base. A method
that a base defines too is checked against it by check::overriding::Override,
except __init__: the same number of parameters with the same names, each of a
type that accepts the base's and with a default where the base has one, and a
result the base's result type accepts, E0404 otherwise (CLS-04, CLS-13). A
call through the method table that leaves out a parameter whose default an
override changes takes the default of the object's class: lowering tests the
instance against each override's class, the nearest first
(MethodTables::overrides, dispatched_default). An override whose parameter
or result is held otherwise than the base's, such as int | None for int
(overriding::Fit::Adapted), gets <adapter name>, which
Program::add_override_adapters writes with the base's signature: it widens
each argument, calls the override through the table, and widens the result.
MethodTables puts the adapter in the base method's place and gives the
override a place of its own, under which calls typed at its class find it; such
a parameter with a default of its own is E0201, planned for 1.0 (CLS-34). A
call of a method on an instance is TypedExprKind::Call with dispatch set,
and a call through super() has it clear. Calling an
exception class without
an __init__ of the program's takes positional arguments of any type; with one,
its arguments are bound as for a call and TypedExprKind::NewException carries
an InitCall with them and the count of those written by position, which become
the exception's arguments; an OSError starts with none, as OSError.__new__
leaves them to OSError.__init__ for a class with an __init__ of its own.
body::super_call checks super().name(...), and
super(Class, self) naming the enclosing class or one of its bases and the
method's instance, in any method (CLS-14, CLS-26): a Call of the method of
the nearest base of that class on the instance with dispatch clear, E0414
when no base of the program's has the method or outside a method, E0404 for
a class outside the chain, and E0201, planned for 1.0, for another instance
(CLS-29). In __init__,
super().__init__(...) calls the nearest base's __init__; with none, an
exception class gets TypedExprKind::SetArguments and any other class nothing,
as object.__init__() does, E0404 with arguments. After it the inherited
attributes count as assigned. ClassInfo::init_uses_instance notes an
__init__ that uses the instance other than through its attributes, directly
or through its base's; super().__init__(...) that runs such an __init__
before the class's own attributes are assigned is E0309, since an override may
read them. Calling a class whose attributes the __init__ that runs, its own
or its nearest base's, does not all assign is E0309, so no instance holds an
attribute without a value (CLS-01). body::isinstance checks
isinstance(value, classes) with classes named in the call (CLS-09). For each
class it decides, from the value's static type, whether every value is an
instance, none is, or the runtime must check: a class derived from the static
class, or any class for a value held as a generic object. The result is
TypedExprKind::IsInstance(IsInstanceTest), which lowers to a constant or to
is_instance calls, which walk the object's type up its bases; None gives
the checker's answer for NoneType and object. When the value reads a
variable, the type a true test gives it, the common base of the tested
classes, takes a place in the function's table of narrowed types, and
flow::test_facts gives Fact::Narrowed(variable, place). Reading the
variable where the fact holds gives TypedExprKind::Downcast, which lowers to
Operation::Downcast, verified like Upcast the other way and free when the
program runs; assigning the variable ends the narrowing (TYP-07). is
compares two exceptions. FunctionParts::method_kind reads a method's
decorator (CLS-06). A static method's definition takes no instance, is recorded
in ClassInfo::static_methods, and is called directly as Class.name(...) or
through an instance; an instance that is not a variable is evaluated first, in a
TypedExprKind::Sequence (CLS-25). Any other method called as
Class.name(instance, ...) is BodyChecker::unbound_call: a direct call of the
method Class holds or inherits, its first parameter taking the instance; for
a generic class, the method of the copy the instance belongs to. In __init__,
Base.__init__(self, ...) on the instance being built goes to base_init, which
super().__init__(...) also reaches, and counts Base's attributes as
assigned. Program::add_static_bridges gives each
static method <static name>, which takes the instance first and calls it; the
method tables hold the bridge in the static method's place, so when a class
derived from the instance's static class defines the name again, as a static
method or not, a call through the instance calls the bridge through the table
(through_static_bridge). An override compares a static method as one that
takes the instance; one held otherwise is E0201, planned for 1.0 (CLS-35). A
property's getter is a method of its name, recorded in ClassInfo::properties,
and its setter the method name.setter, so both override and dispatch as
methods do. body::accessor reads instance.name of a
property as a call of the getter with dispatch set, and checks
object.name = value as the target AssignTarget::Property, which lowering
gives the setter after it evaluates the value and the object; object.name op=
value gives the setter the getter's value combined with the value, the
object held in a hidden variable. A property without a setter is E0414, and
calling a property E0201. A class method follows
ADR-0025: ClassMethodCopies, kept across modules, makes, for each class, a
copy of each class method it defines or inherits, whose cls has the type of
that class's type object, recorded in ClassInfo::class_methods. A copy of a
class method of another module reads its signature in that module's names,
which ClassMethodCopies keeps, and its Definition::home names the module, so
check_bodies gives it to a Deferred checked once every module is, in the
ModuleContext Program::end_module kept for that module (ADR-0037).
Program::add_class_method_bridges gives each copy of a class that is not
generic a bridge, a function of the class method's name in
ClassInfo::methods that takes the instance and the copy's other parameters
and calls the copy with the class's type object, so a call through an
instance dispatches to the bridge of the instance's class; a generic class's
copies have none, and such a call is E0201, planned for 1.0 (CLS-30). An
inherited copy's Definition::copy_of
names the copy it repeats, and the checker drops the diagnostics of an
inherited copy that the original reported. Class.name(...) calls the class's
copy with the class's type object first; calling a variable of type[C], such
as cls, makes an instance of C. Before any module is checked,
check::subclass_variables reads every module for Class.name = value of a
class that only inherits name, following the bases by name, and notes the
class in SubclassVariables::own and the class whose body gives name in
shared (ADR-0037). Each such class holds a module variable Class.name of its
own; a read through a class is TypedExprKind::ClassVariable of the holders
from that class up, which lowering reads as the first one bound, and a read
through an instance of a shared class's chain calls the getter
name.class, which Program::add_class_variable_getters gives each class of
the chain once the first look has typed the variables, through the method
table. An assignment through a variable that holds a class, such as cls, of
a variable a base gives is E0201, planned for 1.0 (CLS-31). Comparison methods follow
ADR-0026: class_types records in InstanceClass::operators which of __eq__,
the ordering methods and __hash__ a class defines or inherits, so
Type::can_order and Type::is_hashable answer for instances, and a class
with __eq__ and no __hash__ has no hash. A comparison method takes self
and a value of a type that accepts its class, E0404 otherwise.
body::operator_method makes a single comparison of instances whose class
defines the method TypedExprKind::OperatorCall, with the left class's method,
the right class's reflected one, the left class's reflected one, and its
OperatorKind: Arithmetic, Ordering or Equality, with the symbol a
TypeError names; != without __ne__ negates __eq__. Lowering
(operator_call) builds the steps CPython takes (ADR-0038). A comparison
calls the right object's reflected method first when its class derives from
the left object's and is another class, and the other method after it. An
arithmetic operator does so only when the right object's class also has
another method at the reflected method's table slot, which
Operation::SameMethod compares, and calls a reflected method of two
instances only when their classes differ. A method whose result may be
NotImplemented is followed by a test of the result; when every method gave
way, equality compares identities, and operands_unsupported or
ordering_unsupported raises CPython's TypeError with the operands'
classes. check::not_implemented reads every module, through
check::class_index, before any is checked, and each method of an
arithmetic, reflected, in-place or comparison name in a chain of bases where
one says return NotImplemented returns T | NotImplementedType
(Type::may_give_way); the call's type is the methods' result without it
(Type::implemented). Such a method that also returns None is E0201,
planned for 1.0 (CLS-33). TypedExprKind::NotImplemented reads the
runtime's one object of the type, and is tests a result that may hold it.
TyirClass holds equal, order and hash, and codegen fills the type
object's slots with functions that call them; the equal slot compares by
identity an object its __eq__ does not take. Program::add_comparison_bridges
writes, for each comparison method of a class's own, a bridge such as
<equal> or <less> that returns self op other, and the slot calls the
bridge in the method's place when the method may give way or the class is one of
a chain of bases, whose objects a container may hold side by side. A slot method
that may raise follows ADR-0039: codegen's slot function gives the exception to
typy_rt_slot_failed and returns its default, every slot of the program first
asks typy_rt_slot_pending, and lowering follows each runtime call whose
RuntimeFunction::may_call_slots holds with slot_raised, when
Calls::raising_slots says a slot method may raise. body::comparison_chain
checks a chain such as a < b < c of such instances as its comparisons
joined with and, each middle operand held in a hidden <chained N>
variable through TypedExprKind::Named (CLS-22). Arithmetic methods
reuse the same call, and when the right operand's class has no reflected
method, OperatorCall::reflected_below lists the classes derived from it that
define one, which lowering tests the right object against in the reflected
method's place. Arithmetic methods: operator_binary makes a op b an
OperatorCall with __add__ and its reflected __radd__, typed by the
method's result, joined
with the reflected method's when it may run first or after the method gives
way (CLS-22); operator_unary calls
__neg__, __pos__, __invert__ and, for abs, __abs__. An augmented
assignment on an instance calls the class's in-place method, such as
__iadd__, with dispatch set, and the binary method when the class has
none, and assigns the result to the target (CLS-20); an in-place method that
may give way is the first step of an OperatorCall whose symbol is +=.
body::in_place::pinned_augmented first holds the object and the key of an
item or an attribute of an expression in hidden variables, so the target is
evaluated once; an in-place method that only a class derived from the
target's defines is E0201, planned for 1.0 (CLS-32). body::container_method calls
__len__ for len, wrapped in TypedExprKind::CheckedLength, which lowers to
int_length and raises CPython's ValueError or OverflowError;
__getitem__ for a subscript; __setitem__ for variable[key] = value, as a
statement that evaluates the value first; __delitem__ for del; and
__contains__ for in and not in. Without __contains__, in on an
instance whose class has __iter__ is TypedExprKind::IterContains: the
value and each item are held in hidden variables, and lowering iterates the
instance as for does until item is value or item == value holds
(CLS-23); a class with __getitem__ alone is E0201, planned for 1.0
(CST-11). body::slice_object gives Type::Slice, a runtime object of type ID
22 that owns its three bounds, each an int word or 0 for None:
x[a:b:c] on an instance passes Builtin::Slice of the bounds to the
container method (index_value), a slice's attributes are
TypedExprKind::SliceBound, indices is Method::SliceIndices, three
calls of slice_index, which computes with integers of any size when the
length is past an inline int, as CPython does, and seq[s] of a built-in
sequence is TypedExprKind::Slice of the bounds, the slice held in a hidden
variable (object_bounds), also as an assignment target. A slice bound
may be int | None: lowering passes its word, and the runtime's
slice::select takes the word 0 as a missing bound (DAT-12). An
augmented assignment to an item of an instance (is_instance_item)
pins the container and the index, then gives __setitem__ the item
__getitem__ gave, combined with the value (instance_item_augmented).
Lowering tests an instance's truth with
__bool__, or with its checked __len__ against zero, and TyirClass::truth
puts __bool__, or <truth>, which Program::add_truth_bridge writes as
len(self) != 0 for a class whose truth comes from __len__, in the type
object's truth slot for any and all. When a class derived from the
static class may decide otherwise, lowering asks the object's type object
through object_is_true (CLS-24). A class body's values follow ADR-0024.
check::dataclass expands each @dataclass before the
subset check reads the module: it finds the fields, its own after a dataclass
base's, writes __init__, __repr__, __eq__ and, with order=True, the
ordering methods as Python source, parses it, moves every node to the
decorator with Stmt::relocate, and gives __init__ the fields' own
annotations and defaults; the methods take the first dataclass of the chain,
and test the other object's class, as CPython does. init=False and
repr=False leave out those methods, and slots=True keeps no class
attribute of a default. __hash__ follows CPython's table of actions
(CLS-16): unsafe_hash=True, or eq=True with frozen=True, writes one
that returns TypedExprKind::Hash of the tuple of the fields, the hash a
set takes through typy_rt_hash_value, unless a field's annotation names a
list, dict or set or a dataclass that has none, and unsafe_hash=True
with the class's own __hash__ is E0404. A field with
field(default_factory=f) becomes a module-level function
<default module:Class.field> that returns f(), written before the class,
and __init__ takes its call as the field's default, which
SignatureParameter::default_call holds, so a call that leaves the field out
calls it each time (CLS-15). Other options of field and @dataclass, and
a default_factory in a generic dataclass, are E0201, planned for 1.0
(CLS-28). A frozen dataclass's
ClassInfo::frozen makes assigning a field E0414 outside the __init__ the
expansion wrote. check::type_alias expands the module's type statements
after the dataclass expansion and before the subset check: it expands each
alias's value through the aliases it names, reports an alias that reaches
itself and one defined twice, and replaces each alias name in an annotation,
except before a subscript, with its value as an Expr::Quoted, which is read
late, so an alias may name a class defined below it (SMT-10). A type
statement inside a function, a class or a block adds its alias to the
scope of that block, which Expansion::scopes keeps innermost last, so the
alias stands for its value in the statements after it and the blocks inside
them (SMT-12). A bound or constraints on an alias's parameter are read as
CPython reads them, which checks no argument against them; the expanded
alias is checked as the type it stands for. expand returns the module's
aliases, and Program::export_aliases keeps each as an ExportedAlias:
its parameters, its value, and the module's class names and
ImportedNames, which ModuleScope::alias_values holds when the module
ends. An import of an alias binds Imported::TypeAlias, which
ImportedNames::aliases and Program::imported_aliases keep, and a module
bound by name carries its aliases in module_aliases; annotation_type
reads Pair[int], shapes.Pair[int] or a plain alias of another module by
reading the alias's value in a scope made from its own module's names, with
each type argument, read where it is written, bound to its parameter. The
subset walker lets a subscript of a name the module imports, or of a name
read through an imported module, reach the checker. An alias read as a value
is TypedExprKind::TypeAliasObject, an index into TypedProgram::type_aliases
that Program::type_alias_object fills with the alias's name and module the
first time it is read; its type is Type::TypeAlias, held as
TyirType::TYPE_ALIAS, type ID 19. Lowering gives Constant::TypeAlias, and
codegen emits one immortal static object per alias, a header then the name
and the module's name as str objects, as it emits module objects, so is
compares two reads. The runtime's typing.TypeAliasType writes an alias as
its name and hashes it by identity, and typy_rt_type_alias_name and
typy_rt_type_alias_module give __name__ and __module__
(TypedExprKind::TypeAliasPart). Calling an alias and isinstance with
one are E0401, as CPython raises TypeError; an alias a function or a
class defines, an alias given its type arguments, and __value__ and
__type_params__ read as values are E0201, planned for 1.0 (TYP-18)
(SMT-12). Final follows TYP-08:
body::locals::declare records each Final variable in
Program::final_variables with the name its declaration assigns, and assign
and module_attribute_target refuse any other with E0414; a class body's
Final with a value is a class variable, whose module variable Class.name is
declared the same way; and note_init records each self.name: Final statement
of __init__ in ClassInfo::final_attributes before any method is checked, so
attribute_target refuses every other assignment whichever method comes
first. Class bodies' values follow ADR-0024:
check::class_values turns name = value and name: ClassVar[T] = value into
an assignment to the module variable Class.name, and name: T = value into
one that holds the default of the attribute name. These module variables are
declared with the module's others, bound by collect_locals and assigned
where the class statement stands in the module body, and every function may
assign them, so a test of one narrows nothing. ClassInfo::class_variables
and ClassInfo::defaults record them, and body::class_value reads
Class.name and instance.name of a class variable as that module variable,
through module_attribute for a class of another module, and checks
Class.name = value and Class.name op= value as its assignment. Assigning a
class variable through an instance, and a default through the class, are
E0414. An exception class's defaults are stored once the runtime makes the
exception, before its __init__ runs (store_defaults).
TypedClass::defaults gives, for each attribute with a default, the variable
of the nearest class that gives one; __init__ starts with those attributes
assigned, and lowering stores each default into a new instance after
NewInstance. A class name
read as a value is TypeObjectOf::Exception, and in an annotation it names the
exception type. Calling a class takes positional arguments of any type and gives
TypedExprKind::NewException { class, arguments }; a keyword is E0404, and a
call of UnicodeDecodeError or UnicodeEncodeError is E0417 (EXC-12). The type
object of an exception is read when the program runs, since its class may derive
from the static one, so type(e) has the type type. raise gives
TypedStmt::Raise { exception, cause }: the exception is an exception, or a
class, which becomes a NewException with no arguments; anything else is E0415.
from None is RaiseCause::None, from x is RaiseCause::Exception under the
same rule, and no from is RaiseCause::Absent. A bare raise is accepted
only inside an except clause, E0416 (EXC-07). Like return, raise makes the
point after it unreachable for the flow state, so a function that ends with it
needs no return.
body/exception_part.rs recognises name.args of an exception and name.code
of a SystemExit where print, str, repr, bool, len of args, an
f-string field, a condition, not, == or != reads them, before the operand
is checked, and gives TypedExprKind::ExceptionPart { exception, part, operation
} with a PartOperation (EXC-11, EXC-13, ADR-0030). Any other use reads them
as values (EXC-15, ADR-0034). exception_arguments::Creations records each
creation of an exception: exception_with_arguments and a call of a class with
its own __init__ record the class with the tuple of the arguments given by
position, super().__init__ records the initializing class with its
arguments, sys.exit records SystemExit, and a dict lookup, dict.pop,
set.remove, an entry target and a % format of a dict record the key type.
Creations::arguments_of(class) gives the type of args: the tuples of every
creation of the class and of the classes derived from it, of every
initialization of a related class, and of the runtime's own raises (a message,
tuple[str]; the key of a KeyError; (int, str) of an OSError; () of
StopIteration; and the five items of a UnicodeEncodeError or a
UnicodeDecodeError only when the program encodes, decodes, opens a file or
makes a str with a surrogate standard output cannot write, so args of a
ValueError stays narrow in most programs). A call of OSError itself with two
to five arguments becomes the subclass for its errno, so it counts for every
related class, as an initialization does. Tuples of one layout, as
many items with one type or two object types at each position, merge into one
member, so the layout of an exception's arguments names its member; the
members are not given to Type::union, which would drop tuple[int] for
tuple[float]. name.args becomes TypedExprKind::ExceptionArguments, and
name.code reads it and takes, for each member, None, its one item, or the
tuple. each_member::Split lets an operation be checked for each member of a
union of tuples that name.args is, as for a variable. TypeChecker::check
checks the program once; when a body read args or code as a value, the
first check typed them as errors, so it checks again with the creations found,
and again while a check finds a creation the ones before did not, up to eight
checks, after which the read is E0201 (EXC-17). Only the errors of the check
whose creations settled are reported, since an earlier check may type args
without a creation it later finds. Lowering reads the arguments
with typy_rt_exception_arguments_address and, for a union, tests each member
but the last with typy_rt_exception_arguments_match, which compares the
layouts field by field, the runtime's text standing for one str.
body/os_error.rs accepts an OSError of arguments of any type, as CPython
does (EXC-14, EXC-16, ADR-0035), except a number as the third of three to five
arguments of OSError or BlockingIOError, which a BlockingIOError reads as
characters_written: E0201, planned for 1.0 (EXC-18). errno, strerror,
filename and filename2 of an OSError whose class declares no attribute of
that name are read from the arguments the exception holds, file names
included: Creations::raw_arguments_of gives their type, a union of tuples,
and for each member the attribute is the item its position names (the first
two of two to five, the third of three to five, the fifth of five when the
third is not None, through a TypedExprKind::IfExp when it may be), or
None. The read is a TypedExprKind::EachMember over
TypedExprKind::ExceptionArguments { raw: true }, whose arms read the
exception again, held first in a hidden variable unless it is one; like args,
it makes the program checked again. Creations keeps each creation's raw
tuple and gives args from it by CPython's rule, a file name that is not
None leaving errno and strerror alone. Lowering calls os_error_new in
place of exception_new for an OSError of two to five arguments, and gives a
class derived from OSError the str of typy_rt_os_error_str through
TyirClass::os_error.
assert test, message gives TypedStmt::Assert { test, exception }, whose
exception is an AssertionError of the message, and the facts the test gives
hold after it (SMT-04). Where T | None is expected, a display, a comprehension
or a collection call is checked against T, then wrapped like any other value.
Iterators (ADR-0014) are Type::Iterator(T), written Iterator[T]. A
class may define __iter__ and __next__, each taking only the instance.
for over an instance of a class with __iter__ is
LoopSource::Protocol { iterable, iter, next, iterator, item }, where
next is the __next__ of the class __iter__ returns and item is
what it returns; a class without __iter__, or an __iter__ whose
result has no __next__, is E0411 (CST-08). iter(value) of a built-in
iterable is TypedExprKind::Iter, of an iterator the iterator itself, and
of an instance a Call of its __iter__. next(iterator) and
next(iterator, default) are TypedExprKind::Next { iterator, method,
default }, with method the __next__ of an instance's class; with a
default the result has the item type, or is optional when the default is
None, and any other default is E0404 (CST-09). Iterators compare with
is only.
typing and collections.abc give Iterator and Iterable, and types
gives TracebackType and ModuleType, as AnnotationNames that
resolve_import binds for annotations only (TYP-15): a name imported from
them, or a name bound to one of the modules, such as typing.Iterable,
enters AnnotationScope::names or standard, and its use as a value is
E0201, planned for 1.0 (TYP-18). Iterator[T] is Type::Iterator(T),
TracebackType is Type::Traceback and ModuleType is Type::AnyModule,
a module whose attributes are unknown, E0412. Iterable[T] is
Type::Iterable(T): any built-in iterable or iterator whose items have
type T, by Type::iterable_item, a fixed tuple of one item type first
becoming a tuple[T, ...]. converted wraps such a value in
TypedExprKind::AsIterable, which keeps the object, so an iterable list
reads twice; an empty display where an iterable is expected is a list or a
set of its items. Its truth and length depend on its collection, so a
condition, not, bool and len of it are E0304 and E0404, and it
neither compares nor hashes. Lowering holds it as TyirType::ITERABLE, an
object of any of those types under the type ID 16, and a loop or iter()
over it calls typy_rt_iterable_iterator, which makes the iterator its
header calls for, reading a view through its dict, or gives an iterator
itself.
typing also gives Optional, Union, and List, Dict, Set,
Tuple and Type as AnnotationName::Builtin (LIB-17). In an annotation
Optional[T] is Type::union of T and None, Union[A, B] the union
of its members, and List[int] is read as list[int] by rewriting the
name; the subset walker lets these subscripts through by name, as it does
Iterator[T]. As the class of isinstance or of a class pattern,
body/typing_class.rs reads these names before the class is checked as a
value (TYP-17): List and the others give the built-in class,
Optional[T] and Union[A, B] the classes of their members, and
Iterable and Iterator give TypeObjectOf::Abstract, which lists the
classes of the program that have __iter__, and __next__ for
Iterator, each with its bases. abstract_outcome answers each member of
the value's type: the built-in collections are Iterable, their iterators
and files are both, and a class of the program is one when it or a base is
listed, may be one when a listed class derives from it, and is not
otherwise. isinstance_test replaces the abstract class in the checks it
leaves for the program's run with the listed classes, so lowering never
meets it. A bare Optional, ClassVar or Final, and a subscripted
generic, are E0401, as CPython raises TypeError; a bare Union is
E0202, since the oracle versions differ; and an Iterable[T] value tested
against Iterator is E0201, planned for 1.0 (LIB-20), since a built-in
iterator it may hold has no class of the program.
A class may define __enter__, taking only the instance, and __exit__,
taking the instance and three parameters that may be left without
annotations: they are then type | None, BaseException | None and
TracebackType | None, and any annotation must accept those types;
__exit__ returns None, a bool or bool | None, or it is E0404
(CST-10). with a as x, b as y: gives nested TypedStmt::With { manager,
enter, exit, suppresses, target, body }; a manager whose class lacks
either method is E0420 (CST-05). suppresses is true when __exit__ may
return a true value, and then the point after the with meets the state
before its body with the state at its end, since a suppressed exception
may stop the body anywhere. Type::Traceback is the traceback, which
compares with is only.
del target, ... gives TypedStmt::Delete { targets }, each target the checked
subscript: a ListItem or a Slice of a list, or a DictItem; a tuple or list
of targets is flattened in order. A name, an attribute, or part of a tuple,
str or bytes is E0421 (SMT-05).
list[lower:upper:step] = items gives AssignTarget::Slice { list, lower,
upper, step }: the target takes a list of the same type, E0402
otherwise, and a slice of a tuple, str or bytes is E0401 (DAT-13). The
value is checked against the list's type, so xs[:] = [] types its empty
list. An augmented assignment to a slice is E0201.
try gives TypedStmt::Try { body, handlers, orelse }, with one
TypedHandler { classes, name, body } per except clause. A clause names
an exception class or a tuple of them, and a bare except catches
BaseException; anything else is E0418 (EXC-09). The name after as is a
new symbol of the clause alone, whose type is the nearest common base of
the classes, so two clauses may bind one name to different classes; when
the clause ends the name is unbound again, and a variable of that name
outside the clause counts as unassigned after it, as CPython deletes the
name. The body may stop at any statement, so each clause starts from the
flow state before the try, less the NotNone facts of variables the
body assigns; else continues from the end of the body, and the state
after the try is the meet of the ends of else and of every clause.
A bare raise is accepted inside a clause. finally runs on every way
out, so it is checked from the flow state before the try, and what it
assigns holds after the try in addition to the meet of the other ends;
a finally that cannot complete leaves the point after the try
unreachable. A return, or a break or continue of a loop outside the
block, inside finally is E0419 (EXC-06).
TypedStmt::AugAssign takes an AssignTarget: a name, or an item of a list,
xs[i] += 1. An item of a str or a tuple cannot be assigned, E0401, and the
result must have the item type, E0402.
7.2 Behaviour¶
TypeChecker::check(project) -> Result<TypedProgram, Diagnostics> checks
every module into one Program, each after the modules its module-level
imports need, in Project::order. A module's functions and classes are
numbered after those of the modules before it, and begin_module gives the
bodies its own tables of names, so TypedProgram::modules lists each
module with the function of its body and its path. Before a module's
functions are collected, module_imports resolves its module-level
imports with Program::resolve_import: a function or a class imported by
name enters the module's tables, so calls and annotations find it, and a
name bound to a module or to a module variable's value becomes a module
variable of the importer, typed Type::Module or with the source's type.
A call or an annotation reads such a name when compiling.
ModuleImports::report_clashes gives E0202 for a name an import binds that
the module also defines with def or class or assigns as a variable
(STD-04). A name imports bind to more than one function, class or function
of a built-in module, or that an import inside a block binds, is a
body::rebound::Rebound (MOD-12, ADR-0036): add_binding_variables adds
the module variable <bound name>, an int whose symbol is named name so
its NameError names it, and takes a name with several targets out of the
tables of functions and classes and into the annotation names as
AnnotationName::Rebound, which an annotation reports as E0201 (MOD-14).
The module body starts by storing 0 in each, and each import statement that
binds the name stores its number, from 1 in source order, as an
ImportBinding::Expression; a lazy import does it in an Import with no
modules that follows its LazyImport, and notes its function in
Program::rebound_reify. rebound_call checks a call once for each
binding and chains TypedExprKind::IfExps on the number, each arm the call
of that import's target, wrapped in Lazy for a lazy one, and a last arm
of Raise of a NameError; rebound_read guards a class read as a value
the same way. A function's own import of the name hides it. Another module
that reads a name with several targets gets E0201 through
Program::rebound_elsewhere.
A function's own imports enter the tables while it is checked. An import
statement is TypedStmt::Import { modules, bindings }: the modules to
initialise, each package before its submodules, and each ImportBinding,
a module object or a copied value. module_of sees through a name of type
Type::Module, so m.x is ModuleAttribute, m.f() a call of m's
function, m.C() an instance, m.x = v an assignment of m's variable,
pkg.sub a Submodule, and m.C an annotation. A name a module does not
define is E0425. A function whose body imports a module not checked yet,
which only an import that closes a cycle can make, waits: check_module
gives it to a Deferred with a copy of the module's ModuleContext, and
once every module is checked Deferred::check swaps that context in,
checks the body and the copies of generics it needs, and swaps back
(MOD-13, OI-22). An __init__ is checked before the rest of its module,
since it gives the attributes their types, so such an import there is
E0426, planned for 1.0 (MOD-15).
A standard library module gives E0201 when a later release provides it,
and E0206 otherwise (LIB-08). For each module it runs:
SubsetChecker::checkwalks the syntax tree and looks up each node kind in the feature table ofProjectSettings::release. A node planned for a later release gives E0201, naming the release and the rule, such as "dicts are planned for TyPy 0.2 (DAT-09)". A node that is excluded from TyPy gives E0202 with rule STD-04.- The resolver binds names. A name assigned anywhere in a function is a
local of that function, as LR 4.2.2 defines. A name the module binds
at module level is a module variable,
SymbolKind::ModuleVariable, whose symbolProgram::declare_module_variablesmakes before any body is checked. A function reads it asTypedExprKind::ModuleVariable, which no test narrows, and afterglobalassigns it (SMT-08); aglobalof a name the module does not bind is E0423. In the module body it is read with definite assignment, and narrowed unless some function declares itglobal. The subset walker reports CPython'sSyntaxErrors forglobal: a name used or assigned before it, a parameter, or an annotated name, E0103. A lazy import at module level becomesTypedStmt::LazyImport. Each name it binds gets a function,Program::add_lazy_import, whose body is the eagerTypedStmt::Importof that name alone, so assigning one name cancels only its own import. A call of a function or a class imported lazily, a class read as a value, andm.f()orm.xthrough a module bound lazily becomeTypedExprKind::Lazy, which names that function.Program::lazy_namesmaps each module and name to it; its IDs are the same in the first look and the final check of a module body, since only module bodies add functions while they are checked. A function's own import of the name hides it. Another module's use of such a name, asm.nameorm.f(), isLazywith that module's function, and itsfrom m import namelists the function in the import'spending, run after the modules and before the names are bound.m.x = vthrough a module bound lazily isAssignTarget::Lazy. A variable bound by several lazy imports keeps its first function inlazy_names; lowering reads every function that binds the variable, and eachLazyImportarms its own and disarms the others, so at most one waits. A function or a class bound by two is called through its binding variable (MOD-11, MOD-12, ADR-0036). The subset walker reports CPython'sSyntaxErrors forlazyinside a function or atry, and with*or__future__, as E0103; a class body holds only methods, so one there is E0202.format % valueswith a string literal on the left is checked bybody/percent.rs(LIB-10).percent_format::parsereads the format with CPython's grammar andValueErrormessages; each conversion then takes its value from the right operand: an item of a tuple, the operand itself when it is not a tuple, or a key of adict[str, V], and*takes anintitem. A conversion that cannot format its value's type, or values that are too few or too many, is E0427 with CPython's message, since CPython raises for every run of such a program. The result isTypedExprKind::PercentFormat, the right operand with aPercentPartfor each piece. A format that is not a literal, and atuple[T, ...]operand, are E0201 (LIB-11). - Inference gives each local the type of its first assignment (TYP-02) and refuses a later assignment of another type with E0402.
- The definite assignment pass follows every path through each function
and the module body, and refuses a read that some path reaches with the
variable unassigned, with E0306 (TYP-12). A loop body may run zero
times,
while Trueends only throughbreak, andelseruns only when nobreakdoes. A function whose return type is notNoneand whose end can be reached gets E0405. - Expressions are checked against the rules EXP-01 to EXP-11. An
intwhere afloatis declared is refused (TYP-09), with the certain fix1.0for a literal. The arguments ofminandmaxmust have one type, because CPython returns one of them unchanged.
Two checks look at the module as a whole. A name may be a function or a
module variable, not both, because CPython would rebind it. And a
module-level statement may only reach, through any chain of calls,
functions whose def has already run, and functions and classes that an
import outside any block binds only once that import has run
(ModuleImports::bound_at_top); otherwise CPython raises NameError, so
TyPy refuses it with E0306.
The subset walker also reports return outside a function and break or
continue outside a loop as E0103, because CPython's compiler rejects them
but the parser does not.
Functions are checked one after another, and their diagnostics are
sorted before they are returned. Each __init__ comes first, since it
gives the attributes it assigns their types, then the module body, which
gives the module variables theirs, then the other functions. An
__init__ reads a module variable with the type a first check of the
module body gives it. That check runs on the program itself, and
Program::snapshot and keep_module_types undo all it did but those
types, dropping its diagnostics, so a module costs the same however many
modules come before it; one whose type needs an attribute is E0424. Parallel checking
(architecture section 7.5) is added when the build-time benchmarks of
NFR-04 call for it.
The 0.1 feature table accepts: module-level def without decorators or
type parameters; positional and keyword parameters with constant defaults;
return, pass, break, continue; assignment, annotated assignment and
augmented assignment to a name; if, while and for with else, where
for iterates over a call to range with one to three int arguments;
expression statements; the operators of EXP-02 to EXP-11; calls; names;
constants of type int, float, str, bool and None; and indexing a
str with an int.
dump(program, lookup) -> String writes the typed program for
--emit=typed-ast: a Program line naming the entry function, then each
function with its module, parameters and return type, its locals, and its
body, one node per line, indented two spaces per level. Each statement and
expression line ends with @LINE:COLUMN, and each expression line also
gives its type after ::
Function fn0 __main__.twice(n: int) -> int @1:1
locals total: int
body
Assign total: int @2:5
Local n : int @2:13
7.3 Exports of a library¶
When ProjectSettings::library is set, TypeChecker::check gives
TypedProgram::exports the Export of each name the main module's
__all__ lists, in order: the name and the FunctionId (FFI-05,
ADR-0032). exports::listed_names reads __all__ from the syntax, the
one module-level assignment, annotated or not, of a list or tuple of string
literals. Each name is checked in turn, and each problem is E0429 under
FFI-05:
| Check | Refused |
|---|---|
| The name | Listed twice; not ASCII; starting with __, _ and a capital letter, or typy_; one of RUNTIME_C_FUNCTIONS, the C library functions the runtime calls, and main; one of the allocator's functions (MEM-08) |
| The function | No function of the module defined at its top level has the name; it is generic |
| Its types | A parameter or the result other than int, float or bool; a result of None is accepted |
An executable has no exports, and exports is None.
library_version::library_version gives TypedProgram::library_version,
the release its files carry (CLI-13, ADR-0045): of __version__ when the
module binds it once to a string literal, else of
ProjectSettings::version, else None. typy_source::version_release
finds the release, the numbers and dots a version starts with after an
optional v. A __version__ bound otherwise, one without a release, and
one that differs from version are E0429 under CLI-13, reported with the
problems of the exports.
7.4 ctypes declarations¶
check::ctypes_declarations::read runs on each module after its type
aliases, before checking (ADR-0047). It finds the names the module's
imports give ctypes and its members, then reads its statements at
module level: NAME = ctypes.CDLL("library") or
ctypes.cdll.LoadLibrary("library") declares a DeclaredLibrary, kept
by the span of its call, and NAME.function.argtypes = [...] and
NAME.function.restype = ... declare a DeclaredFunction, whose
declared is the end of the later of the two. The assignments to
argtypes and restype then leave the module. Each type is a CData
(section 7.5): CType::from_name maps each scalar name of ctypes to a
CType, with the sizes of 64-bit Linux, so c_long is Int64;
POINTER(T) of a scalar is CData::Pointer; and a name an earlier
module-level assignment binds to either stands for it, while the
assignment stays in the module. A library named other than by one
literal, a value that is not a list or tuple of types or a type or
None, a type ctypes has but TyPy does not compile, and a declaration
made twice are E0430; a pointer to a pointer is E0201 under FFI-10.
A class statement whose one base is ctypes.Structure is read by
structure::fields (ADR-0049): a docstring, pass and one _fields_
list of (name, type) pairs, whose types Types::declared reads with
arrays allowed and with the structures of this module and of the modules
it imports, by from or by import. StructureClass::new places each
field at the next offset its alignment allows and rounds the size up to
the largest alignment. The structure joins the program's table, and the
statement becomes NAME = <structure N>, a hidden name the body checker
reads as the class. Any other statement in the body, a bit field, a
decorator, a Union base, other bases, and a subclass of a structure are
E0201 under FFI-11; a field listed twice, _fields_ other than a literal
list, and an item other than a pair of a string literal and a type are
E0430.
The body checker's Foreign holds the declarations and the
ForeignLibrary and ForeignFunction tables TypedProgram carries. A
call whose span a declaration names is TypedExprKind::ForeignLibrary of
type Type::ForeignLibrary, before its callee is checked. A method call
whose receiver is a module variable a library is bound to, which no local
name hides, is TypedExprKind::ForeignCall: E0430 without both
declarations, or at module level before them; E0404 for a keyword, a wrong
count, or an argument CData::accepts refuses, and a truth test for a
c_bool. An argument ctypes.byref(x) is checked where the call is, as
CDataExpr::Reference of x with x's type. The call's type is
CData::result_type. A library read as a value is E0201 under FFI-09.
7.5 ctypes objects¶
CData is a ctypes class (ADR-0048): Scalar(CType), Pointer of a
class, or Array of a class and a length. class_name gives CPython's
name, size the bytes of an object's buffer on 64-bit Linux, and
accepts what a C function's argument of the class takes. A class read as
a value, such as ctypes.c_int, POINTER(T) or T * 3, is
CDataExpr::Class of type Type::CDataClass, which lowers to nothing.
BodyChecker::expression refuses such a value with E0201 under FFI-10,
unless class_allowed is set, which an assignment to names sets for its
value; a call's callee, the operands of *, and the argument of
POINTER and sizeof are checked with expression_or_class, which takes
one.
An object has type Type::CData, which lowers to the runtime's CDATA
objects; it has identity, no hash and a truth. TypedExprKind::CData
holds each CDataExpr:
CDataExpr |
Source | Checks |
|---|---|---|
New(arguments) |
T(...) |
A scalar takes 0 or 1 value it stores, an array its items, a pointer class 0 or 1 object of its target; E0404 for a keyword, a count or a type |
Pointer(target) |
pointer(x), POINTER(T)(x) |
A scalar object; a pointer or an array is E0201 under FFI-10 |
Value(object) |
x.value |
A scalar; its type is CType::result_type |
Contents(pointer) |
p.contents |
A pointer; E0412 for any other attribute |
Item { object, index } |
p[i], a[i] |
A pointer or an array of scalars and an int index; a slice is E0201 under FFI-10 |
Length(array) |
len(a) |
An array |
Size(object) |
sizeof(x) |
An object; sizeof(T) is a constant |
NewStructure(fields) |
S(...) |
Positional values in field order, keywords by name, each field once; E0404 otherwise |
Field { object, field } |
s.name |
A field of the structure: a scalar's value, or a new object over the field |
Reference(object) |
byref(x) |
Only as a C function's argument; elsewhere E0201 under FFI-10 |
AssignTarget::CData writes CDataPlace::Value, Item(index) or
Field(position) of an object, after check_value accepts the value's
type, E0402 otherwise: a scalar takes what CType::stores accepts and a
c_bool any value with a truth, a structure or an array an object of its
class, and a pointer one of its class or None. An augmented assignment to a value
or an item is checked as the assignment of the combined value, with the
object held in a hidden variable as for a property. A for loop, a
comprehension, in and the iterable built-ins read an array's items as
LoopSource::Items, each of its item's value type.
8. typy-tyir¶
8.1 Types¶
TyirType is Int, Float, Bool, Unit, Object(TypeId) or, from
0.2, Tuple(TupleTypeId) and Optional(OptionalBase), an int, float,
bool, object or tuple that holds None in a value it never takes
(ADR-0011): the word 0, the signalling NaN 0x7ff4000000000001, the byte 2,
the null address, or for a tuple a two-item tuple of a presence flag and the
tuple, which TupleTypes interns. Wrap { value, ty }, Unwrap(value)
and IsNone(value) make, read and test one, and Constant::NoneOf(base)
is its None. An instance of class n of the module is
Object(TypeId::class(n)), and TyirModule::classes holds each
TyirClass: its name, its module, the tuple type that lays out its
attributes, its __repr__ and __str__, its own or a base's, and its base
when that is a class of the module. Upcast { value, ty } is an instance,
or an optional one, as an instance of a class it derives from, which the
verifier checks by the bases; it generates no code, since a subclass's
attributes follow its base's (ADR-0021). A call the lowering makes of an
inherited method, such as __init__, __iter__ or __repr__, upcasts the
instance the same way. str, repr and print of an instance of a class
that has a subclass call object_str or object_repr, which call the
methods the object's class holds; of any other class, they call its
methods directly.
NewInstance { class } makes an instance
with every attribute zero, Field { object, index } borrows an attribute,
SetField { object, index, value } stores one, taking over the value's
reference without releasing the old value, and Identical { left, right }
compares two objects by address. A type object is TyirType::TYPE, and
Constant::TypeObject is the address of a runtime type object, by the
symbol of its RuntimeType, or of a class's. Constant::NoInstance(id)
is the null address, the first value of a slot of a class or exception
type. TypeObjectRef::Exception(name) is a runtime exception class, by
the symbol typy_rt_exception_ and its name. An
optional bool is a byte; every other optional has its base's layout, and
is counted as its base is. TyirType is Copy: a tuple type is a number
in the module's TupleTypes, which interns each distinct list of item
types and says whether a type is counted. A tuple is a value
with its items in place, not an object, as ADR-0001 decides, so is cannot
observe a copy (EXP-08), and it is counted when an item is.
methods::MethodTables lays out each class's method table (ADR-0021).
A class starts from a copy of its base's table, puts its own method in place
of each one it overrides, and appends each new method that a class derived
from it defines again; __init__ has no entry, since it runs only on the
class being made. TyirClass::methods holds the table, and a program
without overrides has empty tables. call_method lowers a call with
dispatch set, and the calls of __iter__, __next__, __enter__ and
__exit__: when a class derived from the method's class defines the method
again, the call is Operation::CallMethod { class, slot, arguments }, or an
invoke of Callee::Method { class, slot }, whose parameters and result are
those of the entry of class's table; otherwise it is a direct Call. The
methods at one position of the tables form a family, and when one of them
may raise, each is lowered as one that may, so every entry returns the same
way. The call graph of FUN-02 counts each override of a called method as a
callee. The verifier checks that a class's table is at least as long as its
base's and that the entries at one position take the same number of
arguments, may raise alike, and have equal types or two instance types, which
are both addresses. Codegen emits a class's type object followed by its
table in one global, so the runtime reads the type object as before; a call
loads the class from the instance, at offset 8 in an instance and an
exception alike, then the entry, and calls it indirectly. The type object of
a class derived from a class of the program has that class as its base.
A list is TyirType::LIST, Object(TypeId::LIST). TyirType::Bits is the
eight bytes that stand for a list item at the runtime's interface, never
counted: ToBits(value) makes them from an int, float, bool or
object, 0 from None, and for a tuple the address of a stack slot made at
the start of the function that holds it, which the runtime copies from
before the call returns. FromBits { bits, ty } reads them back as a value
borrowed from the list, for a tuple by loading it from the address the
runtime gives (MEM-02). Constant::PlaceholderList is the runtime's
immortal empty list, the first value of a list slot.
Constant::TupleDescriptor(id) is the address of the descriptor of a tuple
type, which ListNew receives for a list of tuples, and zero bits for any
other list. An Int
is a 64-bit word: an inline value has its lowest bit set and holds the
number shifted left by one, so it covers -2^62 to 2^62 - 1; any other value
is a pointer to a BigInt (ADR-0001, DAT-01). Str values are
Object(TypeId::STR) and exceptions Object(TypeId::EXCEPTION). Int and
Object values are counted; the others are not.
ValueInfo holds a value's ty: TyirType and its
representation: ValueRepresentation: Inline, Stack or Heap. In 0.1
Int, Float, Bool and Unit are Inline and objects are Heap;
from 0.5 escape analysis marks Stack each new instance its function holds
in its frame (section 8.3, MEM-03).
A function's locals are Slots, stack variables read with load and
written with store, as LLVM's alloca is. LLVM's mem2reg pass turns
them into registers, so lowering does not have to build SSA form for
locals itself. A counted slot starts empty and owns the reference it
holds.
TyirFunction
| Field | Type | Default | Rule |
|---|---|---|---|
id |
FunctionId |
none | Unique in the module |
symbol |
String |
none | The mangled name of section 10.3 |
name |
String |
none | The name tracebacks show, such as fib or <module> |
parameters |
Vec<ValueId> |
empty | The entry block's parameters |
return_type |
TyirType |
none | Unit for -> None |
may_raise |
bool |
false |
Set by RaiseAnalysis |
blocks |
Vec<BasicBlock> |
none | The first is the entry block |
values |
Vec<ValueInfo> |
empty | Indexed by ValueId |
slots |
Vec<Slot> |
empty | Indexed by SlotId; each has a name and a type |
BasicBlock has an id, its parameters: Vec<ValueId>, its
instructions: Vec<Instruction> and one terminator: Terminator.
Instruction has an optional result: ValueId, an operation: Operation
and the span of the source it came from, which becomes the line of a
TracebackEntry. An Operation cannot raise. It is a constant, a load
or store, an int, float or bool operation that cannot raise, a
comparison or truth test of one ScalarKind, a call of a runtime function
or a program function that cannot raise, MakeTuple { ty, items }, which
takes over its items' references, TupleItem { tuple, index }, which
borrows an item, or an Increment or Decrement of a counted value; for a
tuple it counts each counted item.
RuntimeFunction lists the runtime's functions, each with its parameter
types, result type and whether it may raise, such as int_floor_divide
(may raise ZeroDivisionError) and str_concat (cannot raise). The
runtime's symbol is typy_rt_ and the function's name.
Terminator
| Variant | Meaning |
|---|---|
Jump { target, arguments } |
Continue in target, passing its parameters |
Branch { condition, then, otherwise } |
Continue in one of two blocks, which take no parameters |
Invoke { callee, arguments, normal, error } |
Call a function that may raise; normal takes the result as its one parameter, and error the exception |
Return(value) |
Return a value, which the caller then owns; a unit value for -> None |
ReturnError(exception) |
Return the exception to the caller (ADR-0004) |
Unreachable |
A point no path reaches |
FunctionBuilder builds a function one block at a time. Code added after a
block's terminator goes to a new block with no predecessors, as dead code
after return does.
8.2 Verifier and text form¶
verify(module) -> Result<(), VerifyError> checks that the entry
function exists and every function has an entry block; that every value is
defined once, and that its definition comes before each use on every path,
by dominance; that operands, stored values, arguments and results have the
types their operations take; that a call of a function that may raise is
an invoke and every other call is not; that an invoke's blocks take
the result and the exception; that branch targets take no parameters; that
only a function marked may_raise returns an exception; that only
counted values are counted; that only a new instance of a class that is
not an exception is held in the frame; that each argument of a
CallForeign has a type its CType::takes; and that each export of a library
calls a function whose parameters and result are the TyPy types of its C
types. A VerifyError names the function and the
block, and the caller reports it as E9001 with the pass that ran last. The
verifier runs after every pass in every build of the compiler. Dominance comes
from the immediate dominators, found with Cooper, Harvey and Kennedy's
iteration in reverse postorder, and a preorder and postorder numbering of
the dominator tree answers each query in constant time, so a function of
many blocks verifies in time and memory in proportion to them
(ADR-0020).
print(module) -> String writes the deterministic text form used by
--emit=tyir-mem, and print_without_counting(module) the same text without
its inc and dec instructions, for --emit=tyir. An instance held in the
frame ends its line with in frame, as in v6 = new class#0 in frame. A
library adds a line library NAME, or library NAME version X.Y, after
the entry, and a line
export add(int64_t, int64_t) -> int64_t = fn3 for each export:
string s0 = "hi\n"
entry fn0
fn _TY8__main__3div(v0: int, v1: int) -> int raises { # div
slot s0 q: int
b0(v0: int, v1: int):
v2 = const int 0
v3 = cmp.int.ne v1, v2
branch v3, b1, b2
b1():
invoke rt.int_floor_divide(v0, v1) to b3 unwind b4
b2():
v6 = int.add v0, v2
return v6
b3(v4: int):
store s0, v4
return v4
b4(v5: exception):
return_error v5
}
8.3 Escape analysis¶
place_instances(module) runs once the whole program is lowered, before
the verifier, in every build mode, so a debug and a release build hold the
same instances in frames (NFR-15, ADR-0042). It marks Stack the result of
each NewInstance of a class that is not an exception when no value that
may hold the instance escapes.
Within a function the analysis puts values and slots in sets with a
union-find forest, without regard to order: a store joins the value and
its slot, a load the slot and its result, Downcast, Upcast, Wrap,
Unwrap, ToBits and FromBits their operand and result, and a jump
each argument and its target's parameter. A set escapes when one of its
values is returned or raised, stored in a module variable or in an
attribute, put in a tuple, passed to a runtime function, or passed to a
parameter that escapes. The runtime functions IsInstance,
ObjectHasTypeId and ObjectType read only the object's type, so their
arguments do not escape. An attribute read, a tuple item and a call's
result start sets of their own: an object stored in an attribute or a tuple
has already escaped, and a callee that returns its parameter lets it escape.
| Rule | Reason |
|---|---|
| Parameters start as not escaping, and each round adds those whose set escapes, until a round adds none | Every function is analysed against the others, also when they call each other |
| A call through a method table counts the function each table of the class and of every class derived from it holds at that position | Any of them may run, as for may_raise (ADR-0021) |
| A position past a callee's parameters escapes | An unknown argument is kept |
| An exception class never qualifies | An exception's layout is the runtime's, and it outlives the function as the error it returns |
9. typy-lower¶
lower(program) -> Result<TyirModule, Diagnostics> turns each
TypedFunction into a TyirFunction and verifies the module. Its only
errors are internal, E9001. The crate has one module per responsibility:
the function lowerer, ownership, invoke and its error path, statements,
expressions, arithmetic, comparisons, conversions, builtins, the call
graph, symbol names and the string pool.
| Concern | Behaviour |
|---|---|
| Locals | Each parameter and local gets a slot. The entry block, written last, stores every parameter and gives every counted slot an immortal value, so every error path can release every slot |
| Imports | Each module has a state global, an int: 0 not started, 1 running, 2 done, all 0 from the entry's first block. An import tests it and, when 0, stores 1, calls the module's code under Cleanup::Initialising, which stores 0 on the error path, and stores 2 (MOD-03). A binding stores Constant::Module, the module's immortal object, or the source variable's value after testing its flag, else typy_rt_module_import_error. ModuleAttribute tests the flag and calls typy_rt_module_attribute_error, and Submodule tests that the submodule is done; both pass the module's state, which says whether it is still initialising. A function tests a module variable's flag unless it is its own module's code |
% formatting |
PercentFormat evaluates the right operand once and borrows each value from it. A field's flags, width and precision become one string, such as -08.3, built at run time when a * gives a number. %s, %r and %a convert the value, then typy_rt_percent_str keeps the precision's code points and pads; integer conversions call typy_rt_percent_int with the int of the value, which writes at least the precision's digits; float conversions call typy_rt_percent_float; %c calls typy_rt_percent_char or typy_rt_percent_char_str. The runtime functions build a format Spec and reuse the layout of format(). The pieces are joined as an f-string's are (LIB-10) |
| Lazy imports | Each function of a lazy import has an armed bool global, false from the start. LazyImport stores true in each and unbinds each variable: its value is released and its bound flag cleared. A read of a variable bound lazily, also in its own module's code, calls the function when the flag is armed and the variable is unbound, then tests the bound flag. Lazy calls the function when it is armed. The function clears its flag only when the import returns, so after an error the next use tries again (MOD-06, MOD-08) |
| Module variables | Each is a TyirGlobal, an internal LLVM variable of zeros, with a second bool global that is true once it is bound (ADR-0015). The entry function's first block gives every counted one an immortal value, so a store can release the value it replaces. LoadGlobal and StoreGlobal read and write them; a function tests the flag first and calls typy_rt_name_error, which raises NameError: name 'x' is not defined, while the module's own code does not, since definite assignment shows the variable is bound |
| Ownership | A lowered value is owned or borrowed. Constants are immortal and borrowed, and so are slot values; results of calls and of int arithmetic are owned. A store moves an owned value or counts a borrowed one, then releases the slot's old value. Owned temporaries are tracked until they are released or moved |
| Raising | A call of a runtime function or program function that may raise is an invoke. Its error block releases the live owned temporaries and every slot, then returns the exception (ADR-0004) |
RaiseAnalysis |
A function may raise when its lowered code contains an invoke. Lowering runs again until that set of functions stops growing, so the analysis always agrees with the code; the verifier checks it |
| Statements | if, while and for with else; break and continue jump to the loop's targets. for over range evaluates its arguments once into hidden slots, raises ValueError for a zero step unless the step is a nonzero constant, and keeps its counter apart from the target, as CPython's iterator does |
| Expressions | and and or return an operand and stop early (EXP-09); comparison chains evaluate each operand once and stop at the first false link (EXP-06); mixed int and float arithmetic converts the int with an invoke, because a large int raises OverflowError; int and float compare exactly in the runtime |
| Tuples | A display evaluates every item first, so an exception from a later item still releases the earlier ones, then takes over their references. An item read from a tuple the function owns gets its own reference before the tuple is released. A tuple target stores each item into its target from left to right after the whole value is evaluated, so a, b = b, a swaps. str and repr join the repr of each item, with a comma after a single item, as (1,). Two tuples compare item by item: the first pair that is not equal decides, and tuples whose items are all equal are equal. A counted tuple slot starts as a tuple of zeros, empty strings and None |
| Lists | Every list operation is a runtime call that borrows its arguments; a list adds its own reference to an item it stores. A display creates the list with room for its items, then appends each in order. list[index] borrows the item, which gets its own reference when the list is a temporary. list[index] = value evaluates the value, then the list and the index, as CPython does. for follows CPython's iterators: each sequence is evaluated once and kept in a hidden slot with its position, and taking the next value reads the length again, checks the position, fetches the item and moves the position on before the body runs, so items appended in the body are visited and a list that shrinks ends the loop. reversed starts at the length less one and also stops below zero. enumerate keeps its count in a slot. zip asks its sources in order and stops at the first that runs out; with strict true, a later source that runs out is shorter, and when the first runs out each other source is asked once more and is longer if it gives a value, both raising ValueError through typy_rt_zip_unequal. The branch taken when a source runs out is lowered with the temporaries tracked before that source was asked, so an error raised there releases only values that exist on it. A list slot starts as the placeholder list. A slice passes each bound as an int word, 0 when it is missing, and a mask of the bounds written, 1 for the start, 2 for the stop and 4 for the step, so the runtime applies Python's defaults. xs += ys and xs *= n change the list in place, as list.__iadd__ and list.__imul__ do, so every name that shares it sees the change, while a tuple[T, ...], a list object too, gets a new tuple and leaves the one other names hold unchanged; *= builds the repetition first, so a count that raises leaves the list as it was. xs[i] op= value evaluates the list and the index once and reads the item before the value, as CPython does; the item keeps its own reference while the value is evaluated, because the value may change the list. sorted copies the list and sorts the copy. max(xs, default=d) tests the length first: an empty list gives d with a reference of its own, and any other list the item the runtime found, so both reach the join with one reference |
| Sets | Every set operation is a runtime call that borrows its arguments; a set adds its own reference to an item it stores. A display creates the set, then adds each item in order. A display of three to thirty items that are all constants, literals, numbers with signs and tuples of them, is then copied, as CPython 3.15 merges the frozenset its compiler builds into a new set (OI-19). set(s) of a set copies it, which merges its table as CPython does; set(source) of any other source adds each value in order. Each operator and method takes CPython's steps: \| and union copy the set and extend the copy with each other collection, skipping the receiver itself; & and intersection intersect with each in turn, and with none copy; - and difference subtract the first and remove the items of the rest; &= and intersection_update replace the set's table with the intersection's. The in-place forms and the updating methods change the set, so every name that shares it sees the change. Another collection is passed as its address, a Bits value, since it may be a set or a list. for keeps the set, the next slot and its number of items in hidden slots; typy_rt_set_next returns the next active slot, or -1 at the end, and raises RuntimeError when the number of items changed, as CPython's set iterator does. pop gives an owned item. A set slot starts as the placeholder set |
| Dicts | Every dict operation is a runtime call that borrows its arguments; a dict adds its own references to a key and a value it stores. A display creates the dict presized for its entries, as CPython's BUILD_MAP does, then sets each key and value in order. d[k] borrows the value, which gets its own reference when the dict is a temporary; d[k] = v evaluates the value, then the dict and the key. get, setdefault and pop give a value with a reference of its own, and popitem builds its pair from the key and value the dict gives up. keys(), values() and items() make a view object that holds the dict, so a view reads the dict as it changes and keeps its own type ID for in, len and its text, also inside a list or a tuple. for over a dict or a view keeps the source, the next entry, the number of entries when the loop began and the entries visited in hidden slots, and typy_rt_dict_next raises CPython's RuntimeError when the size or the keys changed. \| copies and updates, and \|= updates in place. A dict slot starts as the placeholder dict, and a view slot as null, which counting passes over, so a return lowered before a later loop adds its hidden view slot leaves nothing of that slot to release |
| Comprehensions | A comprehension creates its list, set or dict, then lowers its clauses as loops, each around the next. A clause starts its iterator, binds its target and tests each condition, going back for the next value when one fails; the innermost loop evaluates the element and appends or adds it, or evaluates the key, then the value, and sets the entry, as CPython does. The collection is a temporary throughout, so an exception from an element releases it |
| Ranges | A range source evaluates its arguments once, in order, and raises ValueError for a step that is not a nonzero constant and is zero. It keeps a counter, a limit and the step, and gives each value with a reference of its own before the counter moves on. reversed(range(...)) starts from typy_rt_range_last and steps back to start - step, as CPython's range_reverse does |
| Classes | Class(arguments) evaluates the arguments, makes the instance, then calls __init__ with the instance first. An attribute is read with Field, borrowed from the instance, and gets a reference of its own when the instance is a temporary. Assigning an attribute evaluates the value, then the instance, stores the value with SetField and releases the value the attribute held; object.attribute op= value reads the attribute once, with a reference of its own, before the value. str of an instance calls __str__, then __repr__, and repr calls __repr__; a class with neither is written by typy_rt_instance_text. ==, !=, is and is not on instances are Identical, and an instance is true. A slot of a class type starts as NoInstance |
| f-strings | The pieces are joined left to right with str_concat. A field evaluates its value; with a conversion it is written by str, repr or repr and ascii, and then formatted as a str when a specification follows. Without a conversion a field with no specification is str(value), and one with a specification calls the formatter of the value's type: format_int, format_bool with the bool as an int word, format_float or format_str, an optional value branching on IsNone, and any other type format_other with its str and its type's name. The specification is itself an f-string, evaluated after the value, as CPython evaluates it |
try |
The lowerer keeps a stack of Cleanups. A try body pushes Handlers { dispatch }, so the error path of a call inside it releases the live temporaries, runs the cleanups opened inside the body and jumps to dispatch with the exception; an error path outside every try body releases the slots and returns the exception. dispatch tests the clauses in order with exception_matches against each class. A matching clause calls exception_enter, keeps the result, and stores the exception in its variable or holds it, then pushes Handling { previous, exception }. Leaving the clause, by its end, return, break, continue or an error, calls exception_leave(previous) and drops the exception: the variable is set to null and its value released. break and continue run the cleanups opened since their loop started, and return runs them all, after it has taken its own reference to the value it returns. An exception no clause matches takes the error path from after the try. else runs after the body, outside its clauses. A finally block pushes Finally with a copy of its statements, so every way out runs it: falling through lowers it after the rest, return, break and continue run it as a cleanup, and an error path runs it with the pending exception entered as the one being handled, so an exception the block raises takes it as its context and releases it, and otherwise leaves it and goes on. While a copy runs, only the cleanups outside the try are open. return holds its value as a live temporary while cleanups run, and an error path sets aside the temporaries it released before it runs any finally block. A function that returns a value ends in unreachable where its body ends, since the checker rejects one that can reach its end |
| Slice assignment | AssignTarget::Slice evaluates the value, then the list and its bounds, and calls list_assign_slice with the bounds and mask a slice read passes and the new items. The runtime copies the items, each with a reference of its own, before the list changes; a slice of step 1 replaces its run and moves the rest of the list, and any other step takes exactly as many items or raises ValueError (DAT-13) |
del |
A list item calls list_delete, which raises IndexError: list assignment index out of range outside the list; a slice calls list_delete_slice with the bounds and mask a slice read passes, which compacts the list and releases what the slice selects; a dict key calls dict_pop and releases the value |
with |
The manager is kept in a hidden slot, __enter__ is called on it and its result bound to the target, and the body runs inside a With cleanup. Falling through, return, break and continue call __exit__ with three None values. An exception calls it with exception_class of the exception, the exception and traceback, each wrapped as optional, while the exception is entered as the one being handled; when __exit__ may suppress, a true result releases the exception and jumps after the with, and otherwise the exception goes on. An exception __exit__ raises releases the first. A text file (WithProtocol::TextFile) enters as itself after typy_rt_text_enter checks that it is open, and leaving calls typy_rt_text_close, which suppresses nothing; an error of closing takes the exception being handled as its context. str and repr of an iterator or of the traceback call object_repr |
| Iterators | for over a class keeps what __iter__ returned in a hidden slot and calls __next__ at each step through call_next, whose error path releases a StopIteration and leaves the loop, and takes the error path for any other exception. iter() calls iterator_new with the source, or a view's dict, the kind's code and, for a dict's items, the descriptor of the pair. next() calls iterator_advance, which raises StopIteration at the end when told to, and iterator_item, whose borrowed bits become a value with a reference of its own; with a default the two ends meet in a block whose parameter is the result. for over an iterator steps it the same way, and type() of one calls object_type |
assert |
The test branches to the rest of the code when true; when false, the exception is built, which evaluates the message, and raised as by raise, in every build mode |
| Exception classes | A class of the program derived from an exception class has a TyirClass whose exception_base names its base's type object, the runtime's or a class's, and whose repr and str are its own or its nearest base's methods. Its instances are exceptions: NewException with an InitCall evaluates the arguments in order, makes the exception with those written by position as its arguments, then calls __init__ with it first. Field and SetField on an exception carry the class, since every exception has the type exception, and reach the attributes past the fixed part. SetArguments evaluates the arguments into a tuple and calls exception_set_arguments |
| Exceptions | A call of an exception class evaluates the arguments in order into a tuple and calls exception_new with the class, the tuple's descriptor and its address; the runtime copies the tuple and retains its items, and the tuple is released. raise evaluates the exception, or calls exception_handled for a bare raise, sets the cause with exception_set_cause, null for from None, then invokes exception_throw, which always returns the exception as an error, so the call's error path adds the line of the raise to the traceback and releases what the frame owns; the normal block is unreachable. str and repr of an exception call exception_text, which goes through the class's str and repr, and type(e) calls exception_class |
| Type objects | type(x) evaluates x, releases it and gives the constant type object of its type; for an optional value it selects NoneType or its base's by IsNone. A type object is immortal and never counted. str and repr of one call typy_rt_type_text, __name__ calls typy_rt_type_name, == and is are Identical, and a type object is true |
| Optional values | None where T \| None is expected is the constant NoneOf, and a value of T is wrapped; wrapping and unwrapping hand the reference on with the value. is None and is not None on an optional value test IsNone, and on any other value are constants. str, repr and truth tests branch on IsNone and read the value only where it is not None, so a null object never reaches the runtime. An optional slot starts as None, and an optional parameter's default is None or its wrapped constant. dict.get(key) finds the key and wraps its value with a reference of its own, or gives None |
| Builtins | Each builtin of 0.1 becomes runtime calls. print evaluates every argument in source order, then converts and writes each value. min and max keep the first value on a tie, as CPython does |
| Stack | A function in a cycle of calls invokes the runtime's stack check on entry (FUN-02) |
Arguments are borrowed (MEM-04). A parameter the function never assigns
keeps the reference its caller holds: its slot is not counted on entry and
not released on the way out, so passing an argument counts nothing. The
caller keeps every argument alive for the call: a variable or a constant
already is, because no callee can assign a caller's variable, and any other
borrowed argument, such as an item read from a list or an attribute, gets
a reference of the caller's own, released after the call, since the callee
could free it by changing the container. A return of a variable hands its
slot's reference to the caller and skips that slot when the others are
released.
A variable's last use moves its value when it is the value of an
assignment or an item a display, append, insert or add stores
(MEM-09). last_use.rs computes liveness backwards over the function's
statements: a loop is iterated until what is live at its start stops
growing, and inside a try or a with whatever an except clause, a
finally block or the code after a suppressing __exit__ reads stays
live. A read moves when the statement reads the variable nowhere else, it
is outside any comprehension, and the variable is not live after the
statement. Only variables the function owns are moved: not a borrowed
parameter, not a module variable, not the name of an except clause, and
not a value whose zero value allocates, such as a view of a dict. A moved
read loads the slot and stores the slot's zero value, so the value is an
owned temporary and every later release of the slot passes over it.
An owned value, a moved variable or the result of a call, goes into a
collection through a store that takes over its reference:
typy_rt_list_append_moved, _list_insert_moved, _list_set_moved,
typy_rt_set_add_moved, which releases an item equal to one the set
holds, and typy_rt_dict_set_moved, which takes the value and counts the
key. store_call chooses the store, and the value is then neither
counted nor released. EscapeAnalysis is planned for 0.4 and
CycleAnalysis for 1.0.
10. typy-codegen¶
10.1 Types and functions¶
TargetTriple is one of the supported triples. In 0.1 they are
x86_64-unknown-linux-gnu and x86_64-unknown-linux-musl. Target holds a
triple, and Target::host() returns the triple typy was built for.
Optimization is None, Speed or Size, for the three build modes.
LLVM 23 has no Oz pipeline, so Size runs the O2 pipeline with every
function marked minsize and optsize.
| Function | Returns |
|---|---|
object(module, target, optimization, lookup) |
The ObjectFile: its target and its bytes |
llvm_ir(module, target, optimization, lookup) |
The optimised LLVM IR as text, for --emit=llvm-ir |
assembly(module, target, optimization, lookup) |
Assembly text, for --emit=asm |
Each declares every function first, emits each one, verifies the LLVM
module, runs the pipeline and writes the output. A tuple is an LLVM struct
of its items that have a value, so an item of type None takes no field and
the empty tuple is {}; MakeTuple inserts each field and TupleItem
extracts one. Tuples pass to and return from program functions by value. A
tuple type that a list holds gets a descriptor, emitted once as two private
constants: the TupleType header and its table of Field entries, whose
offsets the target's data layout gives, so the runtime reads each field
where the generated code stores it. The
target machine uses the baseline CPU of the triple, so the output runs on
every machine of the target and does not depend on the build host (NFR-06).
Only the entry function is visible outside the object. lookup gives the
file names and lines of traceback entries. Every error is internal, E9001.
The crate has one module per responsibility: the target, the optimisation level, the shared layout, static objects, the calling convention, runtime declarations, function setup, operation dispatch, constants, arithmetic, comparisons, calls and counting, terminators, and emission.
10.2 Contract with the runtime¶
| Item | Form |
|---|---|
int |
A 64-bit word. An inline value has its lowest bit set and holds the number shifted left by one; any other word is the address of a large integer. Every int argument of a runtime function is such a word |
float, bool, objects |
A double, an i1, a pointer. A bool returned by the runtime is one byte |
| Object header | 8 bytes: reference_count: u32, then type_id: u32. A count of u32::MAX marks an immortal object, whose header the runtime never writes |
| Type IDs | str is 1, an exception is 2, a large integer is 3, a list is 4, a tuple[T, ...] is 5, a bytes is 6, a set is 7, a dict is 8, a view of a dict is 9, a type object is 10 and an instance of a class is 11 |
| Type objects | The runtime exports its type objects as typy_rt_object and typy_rt_type_ followed by int, float, bool, str, bytes, none, list, tuple, set, dict, dict_keys, dict_values, dict_items or type, and its exception classes as typy_rt_exception_ followed by the class name, such as typy_rt_exception_ValueError. The type type hashes a type object by identity and writes it as <class 'name'>, or <class 'module.Name'> for a class |
| Exception class | A class derived from an exception class has a type object whose base is the runtime's class, such as typy_rt_exception_ValueError, or its base's typy.classN; whose repr and str are typy_rt_exception_repr and typy_rt_exception_str unless it or a base defines __repr__ or __str__; and whose descriptor lays out its attributes after the exception's fixed part of EXCEPTION_PREFIX, 80 bytes |
| Traceback | One immortal object with the type ID 13 and the type traceback, exported as typy_rt_traceback, written as <traceback object at 0x...>; it holds no frames, which the exception keeps for its report (CST-10) |
| Iterator | Header with the type ID 12, kind (0 list, 1 tuple, 2 str, 3 bytes, 4 set, 5 dict keys, 6 dict values, 7 dict items, 8 an ASCII str, 9 a range, 10 a range past a C long), source, null once exhausted, position, length and visited for the checks of a set or dict, current, the borrowed item of the last step, held, a str iterator's current string, number, a range iterator's current int word, and pair_type and pair for a dict's items. Its type object has CPython's name, such as list_iterator or dict_itemiterator. The compiler gives a range kind 9, and the runtime makes it 10, longrange_iterator, when the start, stop, step or length does not fit a C long, as CPython does |
| Range | Header with the type ID 14, then start, stop, step and length, four int words the range owns, so a range reaches past 64 bits as CPython's does. Its type object typy_rt_type_range is true when the range is not empty, hashes and compares as CPython's range_hash and range_richcompare do, by length, then first value, then step, and is written range(0, 3) or range(0, 9, 3). A range slot starts as null (DAT-17) |
| Instance | Header, class: ptr to the class's type object, then the attributes laid out as the class's tuple descriptor says. typy_rt_instance_new(class) makes one with every attribute zero; the runtime frees it by releasing each attribute the descriptor counts. typy_rt_instance_on_stack(block, class) and typy_rt_instance_off_stack(block) make and release an instance in a frame block (section 10.4). A class's type object is a private constant typy.classN, whose base is the runtime's typy_rt_object, whose hash is typy_rt_identity_hash and whose repr is typy_rt_instance_repr, or typy.classN.repr, which calls the class's __repr__ and adds a null exception when it cannot raise |
| Type object | Header, name: ptr to UTF-8 bytes, name_length: u64, base: ptr (null only for object), then five nullable function pointers with the C calling convention: truth(object) -> bool, hash(object) -> i64, equal(left, right) -> bool, order(left, right, comparison) -> bool and repr(object) -> { str, exception }, then module: ptr to UTF-8 bytes, module_length: u64, fields: ptr to the tuple descriptor of a class's attributes, null for the runtime's own types, and last str(object) -> { str, exception }, null when str is repr. An absent function means what it means for object: true, equal only to itself, and no hash, order or text. A class's str is typy.classN.str, which calls its __str__, or null when it has none. The runtime frees its own objects by type ID, so the type objects stay out of a program that never tests, compares or writes an object of unknown type (ADR-0012) |
| Exception | Header, class: ptr at the offset an instance keeps its class, message: ptr to the str the runtime gave it or null, arguments_type: ptr to the tuple descriptor of the arguments the program built it with or null, arguments: ptr to them or null when there are none, the traceback, cause: ptr, context: ptr and suppress_context: u64, then a class's attributes as its descriptor lays them out (ADR-0013). typy_rt_exception_new(class, arguments_type, arguments) makes one with a copy of the arguments, whose items it retains |
str object |
Header, length: u64 in code points, kind: u32 (1, 2 or 4 bytes per code point), 4 bytes of padding, then the code points |
| Large integer | Header, sign_and_length: i32, 4 bytes of padding, then that many u64 limbs, least significant first |
| Constants | Strings, bytes and large integers are immortal static objects in read-only memory; a bytes constant is a one-byte str object with the bytes type ID |
| List items | Eight bytes: the int word, the float bits, a bool as 0 or 1, the pointer, 0 for None, or for a tuple the address of its bytes. A tuple descriptor is { size: u64, length: u64, fields: ptr }, and each field { kind: u32, offset: u32, tuple: ptr }, with the kind codes of ItemKind: 0 int, 1 float, 2 bool, 3 object, 4 None, 5 tuple, and for T \| None 6 int, 7 float, 8 bool (one byte in a tuple), 9 object and 10 tuple, whose descriptor is that of its pair of a flag and the tuple (ADR-0011). The runtime writes, compares, hashes and tests an optional item as None or as its base, and counting does nothing for the word 0 and the null address |
| A function that may raise | Returns { result, exception }, or only the exception when it returns nothing; a null exception means none was raised. A runtime function's bool result is one byte in either form. The entry function always uses this form |
| A function that cannot raise | Returns its result directly |
invoke |
When the exception is not null, generated code calls typy_rt_traceback_add(exception, function, file, line) with C strings, then takes the error path |
| Entry | The runtime's main calls __typy_main, the body of __main__ |
| Runtime functions | typy_rt_ and the name from TyIR's RuntimeFunction, plus increment, decrement, int_increment, int_decrement, int_add, int_subtract, int_multiply, int_and, int_or, int_xor, int_negate, int_invert, int_compare, traceback_add, exception_raise, exception_enter, exception_leave and exception_matches, with the C calling convention. TyIR's ExceptionNew, ExceptionSetCause, ExceptionThrow, ExceptionHandled, ExceptionClass and ExceptionText are exception_new(class, descriptor, arguments), exception_set_cause(exception, cause), exception_throw(exception), exception_handled(), exception_class(exception) and exception_text(exception, repr); each borrows its arguments |
In 0.1 every int operation calls the runtime. Inline fast paths for
inline values come with the benchmarks of NFR-03.
10.3 Symbol names¶
A TyPy function's symbol is _TY, then the length and text of each part of
its module name, then the length and text of the function name:
_TY8__main__3fib. The module body of __main__ is __typy_main. Symbols
are internal and may change in any release.
10.4 Frame blocks¶
For each instance marked Stack, code generation adds an alloca of
16 bytes plus the attributes' size, as the class's descriptor gives it,
aligned to 16, in the entry block, and stores zero in its header there.
NewInstance then calls typy_rt_instance_on_stack(block, class), and
each return and return_error first calls
typy_rt_instance_off_stack(block) for every block of the function.
| Block state | on_stack does |
off_stack does |
|---|---|---|
| Header cleared | Makes the instance in the block with a count of 2^30 + 1 | Nothing |
| Instance at count 2^30 | Releases its attributes and makes the new instance in the block | Releases its attributes and clears the header |
| Instance above count 2^30 | Returns typy_rt_instance_new(class) from the heap |
Stops the program: typy runtime: an instance in a frame is still held when its function returns |
The bias keeps the count of an instance in a frame from reaching zero, so
the ordinary decrements never free it, and a count back at 2^30 means no
reference is left. A site in a loop reuses its block only then, and an
instance still held, as previous = current keeps one, sends the next to
the heap.
10.5 Heap hooks¶
Every object defines typy_rt_heap_allocate(size),
typy_rt_heap_reallocate(pointer, size) and typy_rt_heap_free(pointer),
which the runtime's allocator calls for all its memory (MEM-08,
ADR-0043). Each is hidden, so a shared library does not export it, and
makes a tail call of the function TyirModule::allocator names, through
its address with the hook's own type, so a name the module already has
with another type still verifies. TyirAllocator::default() names
malloc, realloc and free; the driver sets the module's allocator from
ProjectSettings::allocator after lowering, for every stage that emits
code.
10.6 C calls¶
TyirModule::foreign_libraries names the libraries a program declares,
and foreign_functions holds each TyirForeignFunction: its symbol, its
library and the CType of each parameter and of the result, from
Int8 to UInt64, Float, Double, Bool, CharPointer,
VoidPointer and Pointer of a class of cdata_classes. Lowering gives
each argument the type its C type takes: a bool becomes an int for an
integer type, an int becomes a float through CtypesFloatArgument,
which raises ctypes.ArgumentError for one too large, any value becomes
its truth for a c_bool, and None becomes the None of an optional
bytes, int or ctypes object. Operation::CallForeign then
calls the function, which the text form shows as
call_foreign name(arguments).
Code generation declares the function with the C types, an integer and a
bool as a 64-bit word widened by its sign or by zeros, as libffi
passes it for CPython, and calls it through its address, so a name the
module already has with another type still verifies. An integer argument
is typy_rt_int_low_bits of its word, wrapped to its width and widened
again; a float narrows for c_float; bytes passes
typy_rt_bytes_data, its own buffer, which a zero ends; and an int for
c_void_p becomes an address. A result of 32 bits or fewer becomes an
inline int word in place; int64_t, uint64_t, char * and void *
results go through typy_rt_int_from_int64, typy_rt_int_from_uint64,
typy_rt_bytes_from_c_string and typy_rt_int_from_address, which give
None for null. A ctypes object, or None in its place, passes
typy_rt_cdata_argument, and a Pointer result becomes a new pointer
object through typy_rt_cdata_from_address.
TyirModule::cdata_classes holds each TyirCDataClass a program uses,
which lowering's CDataClasses adds the first time it meets one, the
class of its item first: its name, its CDataShape (Scalar, Pointer
or Array of another entry, or Structure with each field's offset and
class), its size and its alignment. Constant::CDataClass(index) is the
class's address, of type bits, which cdata::cdata_class emits
once as the private constant typy.cdata.N in the layout of the runtime's
CDataClass. Each CDataExpr and AssignTarget::CData lowers to calls
of the Cdata* runtime functions: a read of an item takes a short-lived
object over it from CdataItem, reads it with the getter of its type and
releases it.
A CType::Structure parameter or result passes by value as the System V
calling convention of x86-64 Linux passes a C struct (ADR-0049):
structure_abi::passing gives Nothing for no bytes, Memory above 16
bytes, and otherwise one Piece for each eight bytes, in a vector
register when each scalar in it is a float or a double. Registers
counts the six general and eight vector registers the arguments before
take; a structure whose pieces do not fit, or one in memory, passes as a
byval pointer to its buffer. Each piece is loaded without reading past
the structure: a float, a double, or its bytes widened to a word. A
result in registers is a piece, or a pair of two, stored into a new
object; a larger one is written by the callee into a new object's
buffer, passed first as sret.
10.7 Library objects¶
TyirModule::library holds the TyirLibrary of a library build: its name,
the release of its version, and a TyirExport for each export, with its
function, the CType of each parameter and of the result (Int64,
Double or Bool, and None for void), and the parameters' Python
names. LibraryEmitter adds to the object (ADR-0044):
| Item | Form |
|---|---|
| Constructor | typy.library.load, internal, in llvm.global_ctors at priority 65535: calls typy_rt_library_load(__typy_main) |
| Destructor | typy.library.unload in llvm.global_dtors: calls typy_rt_library_unload() |
| Export | A function with the export's name and C types, a bool as a zeroext i8: calls typy_rt_library_enter(), converts each int64_t with typy_rt_int_from_int64 and each bool by a test against zero, calls the program's function, drops the converted words, ends the process through the no-return typy_rt_library_raised when the call raised, converts an int result with typy_rt_int_into_int64, which may raise OverflowError, calls typy_rt_library_leave() and returns |
An export whose name the object already has is an internal error, which
the checker's rules prevent. c_header(module) writes the header: a guard
of the name in capitals, stdbool.h and stdint.h, extern "C" for C++,
and a declaration of each export, whose parameters keep their Python
names, with an underscore after one that C or C++ reads as a keyword, such
as default_, and argumentN for one that is not ASCII.
python_binding(module, soname) writes the binding module of ADR-0046: it
imports ctypes, lists the exports in __all__, loads soname with
ctypes.CDLL into _library, with underscores added until no export or
parameter has the name, assigns each export's argtypes and restype
from c_int64, c_double, c_bool or None, and defines a function of
each export's name, parameters and Python types that calls it.
python_stub(module) writes the stub of those functions. A string literal
escapes ", the backslash and control characters.
11. typy-build¶
BuildMode is Debug, Release or ReleaseSmall, and
BuildMode::optimization() gives the codegen Optimization of CLI-05.
EmitStage is Ast, TypedAst, Tyir, TyirMem, LlvmIr, Asm or
Obj; EmitStage::parse accepts the lower-case names of CLI-04.
BuildRequest
| Field | Type | Default | Rule |
|---|---|---|---|
main |
Utf8PathBuf |
none | Ends in .py, or BuildRequest::new returns RequestError::NotPython; that the file exists is checked when it is loaded (E0105) |
mode |
BuildMode |
Debug |
--release and --release-small are exclusive |
target |
Target |
Target::host() |
Supported (section 10.1) |
emit |
Option<EmitStage> |
None |
When set, no executable is written |
output |
Option<Utf8PathBuf> |
None |
output_path() defaults an executable's to the main file without .py, next to it; a library's files are named by LibraryLayout |
library |
Option<LibraryKind> |
None |
Static or Shared builds a C library instead of an executable (CLI-09) |
Each field has a with_ method that returns the changed request.
RuntimeArchive::beside(executable, triple) opens
<prefix>/lib/typy/<triple>/libtypy_runtime.a for a compiler at
<prefix>/bin/typy, and RuntimeArchive::open(path) opens an archive by
path. Both read the archive and find the runtime's version marker,
typy-runtime-version: and the version, which must equal the compiler's
package version. A missing archive, a file without the marker and another
version each give E0703 with the path. The driver opens the archive before
it compiles a build that links, so a bad installation fails before any work
and typy check never needs it.
Linker::new(triple, archive) picks the C toolchain program.
Linker::link(object, output) -> Result<Artifact, Diagnostics> writes the
object to a temporary file in the output's directory, then runs the program
as a subprocess with an argument list and no shell:
| Target | Program | Arguments |
|---|---|---|
x86_64-unknown-linux-gnu |
cc |
-o TEMP OBJECT -Wl,--push-state,--no-as-needed -lm -Wl,--pop-state RUNTIME_ARCHIVE [-lLIBRARY] -lm -Wl,--gc-sections |
x86_64-unknown-linux-musl |
musl-gcc |
-static, then the same |
Linker::with_libraries(libraries) adds -l and each library after the
runtime archive, and with_library_paths(paths) adds -L and each path
before the object. The driver passes the allocator's library (MEM-08),
and for each library a ctypes declaration names its
LibraryLink::linker_name, -l:NAME or -l:BASE.a as typy.toml says,
with the directories of library-paths (FFI-01, FFI-04).
LibraryLayout::beside(directory, kind, name, release) names a library's
files as ADR-0045 decides: libNAME.so.X.Y with the soname libNAME.so.X
and links of that name and libNAME.so, or libNAME.X.Y.a with a link
libNAME.a, and without a release libNAME.so, whose soname is its name,
or libNAME.a. LibraryLayout::at(path, kind) is the layout of -o
PATH: the file alone, a shared library's soname its file name.
Linker::link_library(object, layout, kind, exports) builds the library in
a temporary directory beside it, renames it into place when every step
succeeded, then makes each link in the temporary directory and renames it
over the name it replaces (ADR-0044):
| Kind | Steps |
|---|---|
Static |
cc -r -nostdlib -o library.o program.o RUNTIME_ARCHIVE; objcopy --keep-global-symbols=exports.txt library.o, with --strip-debug for a release; ar rcsD library.a library.o, whose D writes no times or owners |
Shared |
cc -shared -o library.so -Wl,-soname,SONAME with the arguments an executable's link has after its output, and -Wl,--version-script=exports.map, which makes the exports global and every other symbol local |
tool::run_tool runs each program, with E0701 when it is missing and
E0702 when it fails, as for an executable.
A --release or --release-small build adds -Wl,--strip-debug, so the
executable holds no debug information, which the runtime archive brings
from core and compiler_builtins with LTO; the symbol table stays. A
debug build keeps it for a debugger.
The C maths library comes before the runtime archive. The archive holds
Rust's compiler_builtins, which defines weak copies of some C maths
functions, such as cbrt, that differ from glibc's in the last bit; the
linker takes an archive member only for a symbol still undefined, so with
-lm first, and kept by --no-as-needed, the program calls the C library
CPython calls (LIB-05). The second -lm serves the static musl link, where
the order of archives matters the other way.
TEMP is created with mode 0755 less the umask, the mode of a new
executable. On success it is renamed to output, so a failed link leaves no
file and an existing file is replaced whole (UC-02, flow 5a); both temporary
files are removed on every path. A missing program gives E0701 with its
name; a failed run gives E0702 with the command line and the linker's
standard error in its help; an output directory that cannot be written
gives E0702 with the directory. None of them has a source location. An
Artifact holds its path and its kind: Executable, StaticLibrary,
SharedLibrary, Object or Text.
12. typy-driver¶
Pipeline::new(compiler) takes the path of the typy executable, which
finds the runtime archives (section 11). with_overrides adds the
settings of the command line (CLI-07) and with_deny_warnings makes
warnings fail (DIA-07); the pipeline holds nothing else, so one pipeline
runs any number of builds, and the conformance suite runs many in one
process. Each method returns a Success, the Outcome with the
Warnings of loading and checking the program, or a Failure.
| Method | Behaviour |
|---|---|
check(main) |
Loads and checks the program, and returns Outcome::Checked with its Sources (CLI-03) |
run(request) with emit set |
Runs the stages up to that stage and returns Outcome::Emitted with an Emission: Text for every stage but obj, Bytes for obj (CLI-04). ast needs only a program that parses; every later stage needs one that checks |
run(request) without emit |
Opens the runtime archive, then loads, checks, lowers, generates the object and links it, and returns Outcome::Executable with the Artifact |
run(request) with library set |
Loads the main file as a library, so its module is named after its file, then links with link_library into the layout of -o, or LibraryLayout::beside the main file with the module's name and release, and writes beside the library the C header NAME.h, and the binding NAME_binding.py and its stub NAME_binding.pyi, which load the soname of the shared library of the build's release, also for a static build (CLI-09, CLI-13, FFI-05, FFI-08) |
The root of the project is the main file's directory. A Failure holds
the Diagnostics and the Sources they are rendered against:
Sources::Project once the program loaded, Sources::Unloaded with the
files of a program that did not, and Sources::None when the build stopped
before reading any file, as a missing runtime archive does. Each stage runs
only when the stages before it found no errors.
Warnings come from two places (ADR-0017, ADR-0033). parse_with_warnings
scans the parser's tokens for invalid escapes, W0101, and
Project::warnings keeps them; the subset walker gives W0301, the body
checker W0401, and TypedProgram::warnings keeps the checker's. After
checking, the pipeline joins them. With deny_warnings set, any warning
makes a Failure before lowering, so no
code is generated; otherwise Warnings holds them with the project's
Sources, and with Sources::None when there are none, so a clean build
copies nothing.
13. typy-cli¶
| Item | Design |
|---|---|
| Arguments | Parsed with clap; the commands check FILE, build FILE and run FILE [-- ARGS] in 0.1, explain in 0.3. Each of the three takes --python-floor VERSION, which becomes SettingOverrides for Pipeline::with_overrides. build takes --emit=STAGE and -o PATH; build and run take --release or --release-small, never both. build and check take --lib static or --lib shared, which sets SettingOverrides::library and the request's library (CLI-09). A main file that is not a .py file is wrong usage |
| Output | Reporter writes diagnostics in the --error-format the command line chose, a global flag: sorted, rendered with render_text and written to standard error with a blank line between two of them, or with render_json one per line. The panic guard and internal errors use the same reporter, so a tool reading JSON also gets E9001 as JSON (UC-10). --emit output goes to -o, or to standard output without it. A file that cannot be written gives E0106 |
typy explain CODE |
Prints the section of docs/diagnostics/troubleshooting.md for the code, embedded in the binary with include_str!, so the page and the command never disagree: its heading and prose, its samples indented by four spaces without their fences and typy-expect comments, links as their text, then the rule from the registry (CLI-10, UC-04). A retired code says so and names its replacement. A text that is not a code, or a code the registry lacks, is wrong usage, status 2. Every section of a code whose cause is one program file holds a program that gives it and the corrected program, which the documentation test runs; the codes about files, settings, the C toolchain and imports between modules show their files as text, and E0429, which only a library build gives, shows modules that are correct as programs |
| Exit status | ExitStatus is Success = 0, ProgramErrors = 1, Usage = 2 and Internal = 101 (CLI-06); a failure with any E9xxx diagnostic is Internal. typy run exits with the program's status, or 128 plus the signal that ended it |
| Panic guard | The command runs on a thread of its own with a 512 MiB stack (ADR-0020), inside std::panic::catch_unwind, with a panic hook that records the location in a thread-local; a panic becomes E9001 with the location and message, and exit status 101 |
| Warnings | Reporter::warnings writes the warnings of a Success before the command's output. --deny-warnings is a global flag. typy run checks a program whose build it takes from the cache, so its warnings show on every run, and the cache key includes --deny-warnings |
typy run |
Builds into cache/<key>/<name>, where the cache is $TYPY_HOME/cache, else $XDG_CACHE_HOME/typy or ~/.cache/typy, and key is the SHA-256 of the compiler version, the mode, the target, the main file's name and its text, each preceded by its length. A build already in the cache is run again without compiling. The executable gets the arguments after -- and the same standard input, output and error |
| Logging | --log-level arrives in 0.3 with UC-10; the 0.1 compiler has no log events |
The pipeline is created with the path of the running executable, so an
installed typy finds <prefix>/lib/typy/ and a typy in
<target>/debug/ finds the archive make runtime stages.
14. typy-runtime¶
14.1 Types¶
| Type | Layout | Rule |
|---|---|---|
ObjectHeader |
reference_count: u32, type_id: u32 |
The first 8 bytes of every heap object (NFR-12) |
TypeObject |
Header, name, name_length, base, the functions truth, hash, equal, order and repr, then module, module_length, fields and str, as in section 10.2 |
An immortal object of type type; every chain of bases ends at object, which has none (ADR-0012, CLS-10). The runtime's own types are statics, and the compiler emits one for each class |
InstanceObject |
Header, class: *const TypeObject, then the attributes |
An instance of a class, under the type ID 11; its class's fields lays out the attributes |
Int |
One u64 word, as in section 8.1 |
Arithmetic on two inline values that does not overflow never calls the runtime |
BigInt |
Header, sign_and_length: i32, then that many u64 limbs, least significant first |
Normalised: no leading zero limb, and a value that fits inline is never a BigInt |
Str |
Header, length: u64 in code points, kind: StrKind, then the data |
kind is Latin1, Ucs2 or Ucs4, the narrowest that holds every code point, as in PEP 393 |
ExceptionObject |
Header, class: *const TypeObject, message: *mut Str, arguments_type: *const TupleType, arguments: *mut u8, traceback: TracebackBuffer, cause and context: *mut ExceptionObject, suppress_context: u64, then the attributes |
An exception under the type ID 2, whose class is an exception class (ADR-0013). The runtime gives message to the exceptions it raises and the program gives arguments to those it builds; a Unicode error has both, its five arguments for repr |
ExceptionKind |
One variant per exception class the runtime raises | Names its class: ExceptionKind::ValueError.class() is typy_rt_exception_ValueError |
TracebackEntry |
function: *const u8, file: *const u8, line: u32 |
One per frame passed through |
ListObject |
Header, length: usize, capacity: usize, kind: u32, words: u32, items: *mut u64, tuple: *const TupleType |
The items live in a buffer of their own, words eight-byte words each, so the list keeps its address as it grows; kind is an ItemKind: Int, Float, Bool, Object, NoneValue or Tuple, and tuple is the descriptor of tuple items or null |
SetObject |
Header, fill: usize, used: usize, mask: usize, finger: usize, kind: u32, words: u32, tuple: *const TupleType, table: *mut u64, states: *mut u8 |
A table of mask + 1 slots, a power of two and at least eight, in one allocation with a state byte per slot after it: empty, active or dummy. A slot is a hash word followed by the item's words. fill counts the active and dummy slots, used the active ones, and finger is where pop starts, as in CPython's PySetObject |
DictObject |
Header, used: usize, filled: usize, usable: usize, log2: u32, key_kind: u32, value_kind: u32, key_words: u32, key_tuple, value_tuple, indices: *mut i64, entries: *mut u64, live: *mut u8 |
CPython's combined table: the entries in insertion order, each a hash word, the key's words and the value's words, with a byte that says whether it holds an item, and 2**log2 index slots that hold an entry's position, -1 when empty or -2 when its entry was removed. used counts the items, filled the entries written, holes included, and usable the entries left before a resize. A new dict has no table, log2 0 |
ViewObject |
Header, kind: u32, padding, dict: *mut DictObject |
A view of a dict under the type ID 9: 0 shows the keys, 1 the values, 2 the items. It holds a reference to its dict, so the dict lives as long as the view |
CDataClass |
name: *const u8, name_length: u64, kind: u32, ctype: u32, size: u64, item: *const CDataClass, length: u64 |
A ctypes class the compiler emits once: kind 0 for a scalar of C type ctype, numbered as CType, 1 for a pointer, 2 for an array of length items of item, and 3 for a structure, whose name holds its module (ADR-0048, ADR-0049) |
CDataObject |
Header, class: *const CDataClass, address: *mut u8, owner: *mut ObjectHeader, kept: *mut Kept |
A ctypes object under the type ID 23: a scalar or an array owns a buffer after this part; an object over another's memory counts owner, the object whose buffer it is or, for C's memory, the pointer it was read through; kept is the table of the objects the values in its buffer refer to, by address: a pointer's target and the bytes of each c_char_p written into it or through an object over it |
14.2 Behaviour¶
| Module | Behaviour |
|---|---|
entry |
main keeps its arguments for sys.argv, records the stack limit, calls __typy_main, reports an uncaught exception, flushes standard output and returns the exit status sys::exit_status gives |
cdata |
typy_rt_cdata_new(class) makes an object, every byte zero; the getters and setters read and write a scalar of each C type, wrapping an int to its width; typy_rt_cdata_pointer(class, target) points to an object and keeps it; typy_rt_cdata_from_address makes a pointer of a C result; typy_rt_cdata_contents and typy_rt_cdata_item give an object over the target or an item, ValueError: NULL pointer access for a null pointer and IndexError: invalid index outside an array, which counts from its end when negative; typy_rt_cdata_argument gives the address a C function takes; typy_rt_cdata_field gives an object over a structure's field, typy_rt_cdata_copy copies an object's bytes into another of its class and keeps the source, and typy_rt_cdata_set_pointer writes a pointer field or item. The type object's slots give CPython's truth and repr (FFI-03, ADR-0048) |
foreign |
typy_rt_int_low_bits(word) gives the low 64 bits of an int word in two's complement, and 0 for the word 0; typy_rt_int_from_uint64, typy_rt_int_from_address and typy_rt_bytes_from_c_string make a C result a TyPy value, null giving None; typy_rt_ctypes_float_argument(word, position) converts an int for a c_double, raising ctypes.ArgumentError, a class whose module is ctypes, with CPython's text for one too large (FFI-03, ADR-0047) |
library |
typy_rt_library_load(module) records the stack limit, keeps [''] for sys.argv, runs the module bodies and flushes standard output; _unload flushes it and closes the files; _enter records the stack limit for the calling thread, _leave flushes standard output; _raised(exception) exits with a SystemExit's status, or reports any other exception as uncaught and aborts (ADR-0044). stack::record_limit reads getrlimit once and measures from the current stack pointer |
allocator |
allocate, reallocate and free over the program object's hooks typy_rt_heap_allocate, typy_rt_heap_reallocate and typy_rt_heap_free (section 10.5), asking for at least one byte. Running out of memory prints MemoryError and aborts. The unit tests define the hooks over malloc, realloc and free, since they link no program object |
refcount |
typy_rt_increment and typy_rt_decrement, not atomic (A-03). A decrement to zero calls the type's drop, then frees the memory |
int |
Addition, subtraction, multiplication, floor division, modulo, power, negation, comparison, conversion to and from str and float, all with Python's results (EXP-02, EXP-05). typy_rt_int_from_int64 and typy_rt_int_into_int64 convert a C int64_t for an export, the second dropping the word and raising OverflowError: Python int too large to convert to C int64_t outside its range (FFI-05) |
float |
Division, floor division, modulo and power with the special cases of CPython's float_divmod and float_pow. int / int, float(int) and float(str) round once to the nearest double, ties to even, by CPython's long_true_divide method on exact integers. int(float) and int against float comparisons are exact. round rounds half to even on the exact value: round(x) to an integer, round(x, n) by rounding x * 10**n exactly and reading the result back with one correct rounding, as CPython's _Py_dg_dtoa and _Py_dg_strtod do, so round(2.675, 2) is 2.67. Beyond 323 digits a float rounds to itself and below -308 to zero; a result too large raises OverflowError: rounded value too large to represent. round of an int with negative ndigits gives the even multiple of 10**-ndigits, as long_round does. repr writes the shortest digits that read back as the same double, found exactly with large integers, and breaks an exact tie toward the even digit as CPython does; the layout is positional when the decimal exponent is from -4 to 15, otherwise scientific with a sign and at least two exponent digits, and inf, -inf, nan and -0.0 are written as CPython writes them (DAT-02) |
libm |
floor, fmod, pow and ldexp from the C maths library, which is why every link passes -lm |
math |
The functions behind the math module (LIB-05, ADR-0018), one typy_rt_math_* symbol each. A function of the C library runs inside CPython 3.15's math_1 or math_2 checks: a NaN from a number is ValueError, with the function's 3.15 message such as expected a nonnegative input, got -1.0, or math domain error, and an infinity from a finite number OverflowError: math range error where the function can overflow. pow, fmod, remainder, ldexp, ulp, nextafter, fma, isclose, hypot and dist, with CPython's vector_norm, gamma and lgamma, with its Lanczos sum, fsum, with Shewchuk's partials, and sumprod, with its machine-integer path, its triple-length path and the exact total it falls back to, are ported from mathmodule.c; prod of an int start keeps the exact product until its first float item, and each reads the member a union item holds (LIB-15), and fmax and fmin keep x when the two are equal, as the oracle's Clang build does. A logarithm of an int too large for a float is that of its top 55 bits, rounded once, plus its bit count times the logarithm of 2, by fma, as 3.15 does. factorial multiplies by halves, isqrt runs Newton's method from a power of two, comb and perm take exact running quotients, and gcd is Euclid's; each refuses what CPython refuses, with its message and its limit of a C long. The C functions are named with #[link(name = "m")], so a Rust test binary of the runtime also calls the C library before compiler_builtins |
string |
Every str and bytes object, and every constant one, keeps a zero unit after its data, so typy_rt_bytes_data gives C the object's own buffer as a string (ADR-0047). Creation, concatenation, repetition, indexing with negative indexes (DAT-11), slicing into a string as narrow as the code points it selects (DAT-12), length, the methods of LIB-03 that need no Unicode tables, with CPython's index adjustment for start and end and its rules for an empty substring, whitespace as str.isspace defines it, and the line breaks of str.splitlines, built through a StrBuilder that makes the new string as narrow as its widest code point, and the case methods and is tests. Their tables in string/unicode/case_tables.rs are read off CPython 3.15 itself, code point by code point, by scripts/generate-unicode-tables.py: for each property the code points where it changes, and for each case mapping runs of a constant difference over a stride of one or two, about 200 for each, with the few mappings to more than one code point listed apart. A program links only the tables its methods use. lower makes a capital sigma final sigma, when a cased code point comes before it and none after it, skipping case-ignorable ones, as CPython does; both properties are read from CPython by lowering a sigma beside each code point. title lowercases a code point after a cased one, comparison, repr, chr and ord. repr escapes the code points str.isprintable rejects, and int() and float() read the decimal digits of every script, from tables that scripts/generate-unicode-tables.py writes from CPython 3.15's Unicode 17.0 data (OI-15). The tables hold 676 boundaries up to U+FFFF as u16, 806 above it as u32, and 77 digit zeros, about 4.9 KB |
list |
typy_rt_list_new(kind, capacity, descriptor), _append, _length, _get, _set, _equal, _contains, _repr, _slice, _pop, _insert, _extend, _remove, _index, _count, _clear, _copy, _reverse, _concat, _repeat, _compare, _sort, _extreme, _sum_int, _sum_float, _any and _all. A list grows to n + n/8 + 4 items while small and to n + n/32 + 4 from 65,536 items, so a list of a million int values built one append at a time takes at most 8.5 MB (NFR-13). A position counts from the end when negative; one outside the list raises IndexError: list index out of range, or list assignment index out of range, and a large integer cannot fit 'int' into an index-sized integer. An Object item is compared and written by the type ID in its header, so a list of lists needs nothing from the compiler. repr joins the repr of each item once, in a string as wide as its widest piece. pop takes its index with an int flag, 1 when an index is given, so no bool crosses into the runtime as an argument, and returns the item with the reference the list held. insert clamps its position to the list, as CPython does. extend reads the other list's length first, so xs.extend(xs) doubles the list once. clear sets the length to zero before releasing the items, so an item's drop never sees a half-cleared list. remove and index raise ValueError with CPython 3.14's messages, list.remove(x): x not in list and list.index(x): x not in list (OI-14). repeat raises OverflowError for a count that does not fit an index. compare decides by the first unequal pair, then by length. sort is list::listsort, a port of CPython 3.15.2's listsort that calls < on the same items in the same order: natural runs that count_run finds, extended by binary insertion to the minrun that minrun_next spreads over the list, merged by the powersort strategy with galloping through a buffer of half the list; a comparison whose slot method raised ends it, the items a permutation of the input as CPython leaves them (OI-27). With reverse it reverses, sorts and reverses again, as CPython does, so equal items keep their order; a nan makes the order of a list depend on the algorithm, and CPython's own versions differ on it. extreme keeps the first item until a later one is less, for min, or greater, for max, so the first of equal items wins, and an empty list raises ValueError: min() iterable argument is empty. sum_float is CPython 3.12's compensated sum, Neumaier's improvement of Kahan's: each rounding error is collected and added once at the end unless it is not finite. A tuple[T, ...] is a list object whose header has the tuple type ID, 5, set by typy_rt_tuple_seal once its items are in: repr writes it between parentheses, (1,) for one item, its index error is tuple index out of range and index raises tuple.index(x): x not in tuple, and a slice, + or * of a tuple is a tuple. typy_rt_placeholder_list is an immortal empty list. Freeing a list releases each item |
format |
typy_rt_format_int, _bool, _float, _str and _other give format(value, spec) as f-strings call it (LEX-07), a port of CPython's formatter_unicode.c: the specification [[fill]align][sign][z][#][0][width][grouping][.precision][type] is read with CPython's checks and ValueError messages, the digits of a number are grouped by , or _ with zeros added inside the groups when 0 pads it after its sign, and the number or text is padded and aligned. A float is written as PyOS_double_to_string writes it for e, f, g and repr, from digits rounded half to even on the exact value with large integers, as _Py_dg_dtoa rounds them, then laid out as format_float_short lays them out. A bool comes as an int word, and any other type as its str and its type's name, which an empty specification gives back and any other raises TypeError. typy_rt_ascii escapes every code point outside ASCII for !a. scripts/generate-format-cases.py writes the test table, 4,628 results that CPython 3.12 to 3.15 all give |
message |
Builds an exception message of up to 1024 code points from text, numbers and repr of a string, so an error about a long input stays short |
io |
Standard output is buffered in 8 KiB and flushed when full, at exit, and before anything is written to standard error. string::write_utf8 writes a surrogate as each standard stream's handler in CPython's UTF-8 mode does: Surrogates::Escape writes U+DC80 to U+DCFF to standard output as the bytes 0x80 to 0xFF, and Surrogates::Backslash writes \ud800 to standard error and in reports. string::unescapable finds the first surrogate outside U+DC80 to U+DCFF and the end of its run, where typy_rt_write_stdout_checked and a write to sys.stdout raise CPython's UnicodeEncodeError before writing any of the string (LEX-10) |
text_file |
TextFile objects under the type ID 17: a header, a descriptor, flags (readable, writable, closed, standard), the name of a standard file, the path and mode of a file open made, its Reader or Writer, and its place on the list of open files. sys.stdout and sys.stderr are immortal statics that print as <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, and sys.stdin one with mode r, read through a static Reader that escapes a byte outside UTF-8 as U+DC80 to U+DCFF, as CPython 3.15's UTF-8 mode does, and that input() shares; writing to it raises io.UnsupportedOperation: not writable; standard output is written through its buffer and standard error at once, and closing either only writes it out. typy_rt_open installs the functions that free a file and close every open file, which destroy and main call through a hook, so a program that never opens a file links none of this code (NFR-01) |
text_file::disk |
typy_rt_open(path, mode, how, buffering, encoding, newline) encodes the path as os.fsencode does (U+DC80 to U+DCFF as bytes, ValueError: embedded null byte for U+0000), opens it with O_CLOEXEC and the flags of the access, retrying EINTR, and refuses a directory opened for reading with EISDIR, as CPython's fstat check does. Every operation on a closed file raises ValueError: I/O operation on closed file.; reading a file opened for writing, or the other way round, raises io.UnsupportedOperation. A mode with + opens the descriptor O_RDWR and gives the file a Reader and a Writer: a read writes out the writer first, and a write drops what the reader holds, so it lands where the reader stopped reading the descriptor, as CPython's text layer over BufferedRandom writes. close writes out the buffer, closes the descriptor even when that fails, frees the buffers and takes the file off the list; a file whose last reference goes is closed with its error dropped |
text_file::state |
typy_rt_text_name and typy_rt_text_mode give the path and mode a file was opened with, or the name of a standard file and r or w. typy_rt_text_fileno gives the descriptor, and typy_rt_text_state(file, question) answers isatty, readable, writable, seekable (lseek to the current position succeeds) and closed. On a closed file these raise ValueError: I/O operation on closed file without the full stop, as CPython's raw file does, except closed, and readable or writable for the direction the file lacks, which answer False as CPython's buffered layer does |
text_file::position |
tell, seek and truncate (LIB-14). A position is a byte offset: CPython's cookie for UTF-8 when no decoding is pending, which holds at every character a read gives out. tell writes out what the file holds and gives the descriptor's position less the bytes its Reader holds. seek checks as CPython's TextIOWrapper.seek does, in its order: whence a C int, the file open and seekable, whence 0, 1 or 2, a move from 1 or 2 only by zero, then no negative position; it writes out the buffer, resets the Reader and moves the descriptor. A position from 2^63 is EINVAL, as lseek gives it. truncate writes out the buffer, refuses a file not opened for writing with io.UnsupportedOperation: truncate, reads pos as a C long, and calls ftruncate without moving the file. A for loop reads through loop_line, which marks the Reader so that tell raises OSError: telling position disabled by next() call until the loop ends or a seek resets it. typy_rt_open moves a file opened with a to its end, as CPython's FileIO does |
text_file::reader |
A Reader holds one chunk of 8192 bytes, TextIOWrapper._CHUNK_SIZE, the up to three bytes of a character cut off at its end, and a \r that waits for the next byte, and decodes as its Decoding says: a Codec, a Handler and a Newline. Each chunk is checked by the codec when it is read, as CPython's incremental decoder checks it, so with strict an invalid byte raises UnicodeDecodeError with the chunk as its object and its position in the chunk, at the read that reaches the chunk, and with xmlcharrefreplace CPython's TypeError. Another handler decodes every byte: ignore and replace take a whole error, and backslashreplace and surrogateescape one byte at a time; the rest of a backslashreplace escape that a size cuts waits in the reader for the next read. read() without a size decodes the rest at once. With universal newlines \r\n and \r read as \n; otherwise lines end where the Newline says and nothing is translated |
text_file::writer |
A Writer translates \n as its Newline says, encodes the whole text with its codec and handler through codec::encode first, so an error raises UnicodeEncodeError with the translated text before anything is written, then holds the bytes in a buffer of 8192 that is written out when full, on flush, on close, and after a write of a \n or \r when the file is line-buffered |
text_file (standard files) |
io::STANDARD_CLOSED holds a bit for each of standard input, output and error the program closed; flags_of adds CLOSED for one, so every method checks a standard file as it checks one open made, and the descriptor stays open, as CPython's closefd=False leaves it. input() checks standard output before its prompt and standard input before reading. With standard error closed, report_uncaught writes lost sys.stderr and sys.exit with a message writes only the line ending. print() without file calls typy_rt_write_stdout_checked, which may raise, only when TypedProgram::closes_text_files is set, as the checker sets it for a close() of a text file or a with block over one, or TypedProgram::makes_surrogates, which it sets for a call of chr and for a literal or a default with a surrogate outside U+DC80 to U+DCFF; any other program keeps typy_rt_write_stdout, which cannot raise, so its code and runtime do not grow (NFR-02). sys.argv, standard input and surrogateescape decoding make only U+DC80 to U+DCFF, which standard output writes as bytes |
text_file::disk (descriptors) |
typy_rt_open_descriptor checks a descriptor as CPython's FileIO does: past a C int is TypeError: expected str, bytes or os.PathLike object, not int, negative is ValueError: negative file descriptor, fstat failing is its OSError, and a directory is IsADirectoryError naming the number. The file is flagged DESCRIPTOR, so its repr shows name=3, and KEEP_OPEN for closefd=False, so close leaves the descriptor open; typy_rt_open refuses closefd=False with ValueError: Cannot use closefd=False with file name before it opens the path |
text_file::binary |
Binary files, flagged BINARY, and RAW for a FileIO (ADR-0031). A buffered file reads through a Reader of latin-1 with read_bytes, and writes through its Writer with add; a BufferedRandom moves the descriptor back by what the reader holds before a write, so it writes where tell says, and tell counts what the writer holds without writing it out, as CPython's buffered layer does. A FileIO calls the descriptor for each operation and reads a line a byte at a time. Each operation refuses in CPython's order and words: a buffered file checks that it can read or write before it checks that it is open, and names the operation (read of closed file, seek of closed file); a FileIO checks that it is open first. seek of a buffered file allows whence 0 to 4 and refuses an offset past an i64 with ValueError; a FileIO gives whence to lseek and refuses such an offset with OverflowError |
text_file::newline |
Newline::of reads newline: None is Universal, "" Untranslated, and "\n", "\r" and "\r\n" end lines there; any other value raises ValueError: illegal newline value: x. typy_rt_open checks buffering as a C int before opening and checks buffering=0 and newline after, closing the descriptor, as CPython opens the file before it checks them |
text_file::os_error |
os_error(errno, filename) gives the OSError subclass CPython's errno table names, written as [Errno 2] No such file or directory: 'name' with the repr of the file name, and with the arguments (errno, strerror) for its repr. The texts are glibc's, in a table make errno-texts writes, so both targets print the oracle's words |
text_file::os_error (programs) |
typy_rt_os_error_new makes an OSError the program built with two to five arguments of any type: OSError itself becomes the subclass for errno when it is an int that fits a word or a bool. typy_rt_os_error_str, the str of every OSError class, writes the runtime's text, or for arguments of two to five [Errno N] strerror with str of each, then a colon and the repr of filename, and an arrow and the repr of filename2, when they are not None, raising what a str raises, as CPython's OSError_str does; other arguments are written as BaseException writes them (ADR-0035). args, str and repr show only errno and strerror when a filename other than None was given, as CPython does; exit_code reads arguments without that rule, so a program links no OSError code for its exit status (EXC-14, NFR-02) |
format::percent::dynamic |
typy_rt_percent_dynamic formats format % values when the format is not a literal or the values are a tuple[T, ...] (LIB-11): it reads the format with the grammar of CPython's unicode_format_arg_parse, takes the values from a tuple of their layout, from one value, or from the list that holds a tuple[T, ...], reads a key from a dict, converts each value by its kind with the code of the literal path, and raises CPython 3.15's errors |
exception::part |
typy_rt_exception_part_text, typy_rt_exception_args_length, typy_rt_exception_part_truth and typy_rt_exception_part_equal read args or code from an exception: args is the tuple of its arguments, or of the runtime's text, and code None, the one argument as the value it holds, or the tuple of several. == takes the other value as a tuple of one, and values_equal compares an optional or a union with a value of another type as the value it holds (ADR-0030). typy_rt_exception_arguments_match(exception, layout) says whether args is laid out as a member of the union of tuples args has, and typy_rt_exception_arguments_address gives the address of its items: the arguments, or the field that holds the runtime's text (EXC-15, ADR-0034) |
sys |
typy_rt_sys_argv builds the list once from the arguments of main, decoded as UTF-8 with each stray byte as U+DC80 to U+DCFF, as CPython's surrogateescape does. typy_rt_input writes the prompt, flushes standard output, then reads a line through sys.stdin's reader without its \n, and raises EOFError: EOF when reading a line when nothing is left. typy_rt_sys_getrecursionlimit and typy_rt_sys_setrecursionlimit keep the limit, checked as a C int of at least 1. exit_status gives the status of an uncaught exception: for a SystemExit or a subclass, 0 for no code or None, the low byte of an int, 255 for one outside i128 as CPython 3.15 gives, and 1 after writing any other value to standard error, all without a traceback; 1 with the report for any other exception |
exception |
Defines the built-in exception classes with CPython's bases: BaseException, with SystemExit and Exception; under Exception ArithmeticError with OverflowError and ZeroDivisionError, LookupError with IndexError and KeyError, RuntimeError with NotImplementedError and RecursionError, ValueError with UnicodeError and its UnicodeDecodeError and UnicodeEncodeError, OSError with its fifteen subclasses and io.UnsupportedOperation, which derives_from also counts as a ValueError, its second base in CPython, and AssertionError, AttributeError, EOFError, StopIteration and TypeError (EXC-02, EXC-10). Each hashes by identity and writes its objects with typy_rt_exception_repr and typy_rt_exception_str. str is the runtime's message, or from the arguments: empty for none, the str of one, its repr for a KeyError, and the repr of the tuple of several; repr is the class name and the arguments between parentheses, with no comma after one, and a runtime message stands for the one argument (EXC-08). typy_rt_exception_matches walks the chain of bases (EXC-01). The exception being handled is one value, which typy_rt_exception_enter sets and returns the one before and typy_rt_exception_leave restores; every exception raised meanwhile, by the runtime or through typy_rt_exception_raise, takes it as its context, unless it is that exception, and a link back to the raised exception in the context's chain is cut first, as CPython's _PyErr_SetObject does. typy_rt_exception_set_cause sets the cause, or none, and suppresses the context (EXC-04). The runtime creates its exceptions with CPython's message for each operation. Where the oracle versions word a message differently, the runtime uses the wording of the newest, CPython 3.15 (OI-14): division by zero for every division and modulo of int and float, and zero to a negative power for 0.0 ** -1.0. A conformance test cannot check such a message, because the oracles disagree on it, so the runtime's unit tests check it. It adds TracebackEntry values, and it prints an uncaught exception as CPython does: the oldest exception of its chain first, each followed by The above exception was the direct cause of the following exception: for a cause or During handling of the above exception, another exception occurred: for a context that from did not suppress, each exception shown once; for each, Traceback (most recent call last): when it was raised, one line per entry, an entry repeated in a row shown at most three times and the rest counted as [Previous line repeated N more times], as CPython's TB_RECURSIVE_CUTOFF does, then Type: message, or Type alone when the message is empty, with the module before the class name unless it is __main__, and <exception str() failed> when str raises (STD-05). The report calls str through the class, so a program that raises nothing links none of the code that writes arguments (NFR-01) |
bytes |
A bytes is a str object of one byte per code point under the type ID 6, so it shares the string algorithms; typy_rt_bytes_concat, _repeat, _slice, _length, _compare and _contains call them and keep the result a bytes. _get gives an item as an int, with IndexError: index out of range outside; _contains_byte raises ValueError: byte must be in range(0, 256) for an int outside; _zeros and _from_list build one, bytes must be in range(0, 256) and negative count as CPython words them; _repr writes b'...' in double quotes when the bytes hold a single quote and no double quote, and escapes every byte outside 0x20 to 0x7e as \xhh |
bytes methods |
The string algorithms read the type ID of the object they are given: on a bytes, whitespace is only space, \t, \n, \v, \f and \r, line breaks are \n, \r and \r\n, a missing index raises subsection not found, a fill of the wrong length raises CPython 3.15's center(): argument 2 must be a byte string of length 1, not a bytes object of length 2 (OI-14), and every text they build is a bytes. typy_rt_bytes_search and the other bytes_ symbols forward to them under bytes signatures; _case, _test and _hex are ASCII only, as CPython's are |
codec |
typy_rt_str_encode and typy_rt_bytes_decode with the codecs and handlers above. A run of code points the codec cannot encode, surrogates for utf-8 or those past its range for ascii and latin-1, is one error, and UTF-8 is decoded as CPython's decoder does: a lead byte outside C2 to F4 is an invalid start byte, a byte outside the range its lead allows ends an invalid continuation, and data that runs out is an unexpected end of data; replace gives one U+FFFD for each error range. The messages of UnicodeEncodeError and UnicodeDecodeError are CPython's, such as 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte |
tuple |
A list of tuples stores each tuple inline, in the words its descriptor gives, and the list's header keeps the descriptor. A TupleType holds the item size, a multiple of eight bytes and at least eight, the number of items, and a Field for each: its ItemKind code, its byte offset, and a nested tuple's descriptor. A bool field is one byte and None takes none. retain, release, equal, order and repr walk the fields, so the compiler emits data and no code: tuples order as CPython orders them, by the first pair of items that are not equal, and repr writes (1,) for one item |
hash |
The hashes CPython gives (ADR-0010): an int is its magnitude modulo 2^61 - 1 with its sign, a large one reduced limb by limb since 2^64 is 8 modulo that prime; a float follows _Py_HashDouble on its mantissa and exponent, so it hashes as an equal int, an infinity as 314159 with its sign, and a NaN as 0 (OI-18); None is 4238894112 and a bool 0 or 1; a tuple combines its items' hashes with xxHash's rounds, its length and CPython's constants. A str or bytes is SipHash-1-3 under a key of zeros of its code points one, two or four bytes each, as CPython stores them, so its hash is the one CPython gives with PYTHONHASHSEED=0, and an empty one hashes to 0. A hash of -1 becomes -2, as in CPython |
set |
typy_rt_set_new(kind, descriptor), _add, _contains, _discard, _remove, _pop, _clear, _length, _next, _item, _repr, _copy and _compare, a port of CPython's setobject.c (ADR-0010). A probe visits a run of ten slots, or one near the end of the table, then jumps to (slot * 5 + 1 + perturb) & mask with perturb shifted right by 5 from the hash. An item goes to the first empty slot of its probe, or to the last dummy slot before it; when fill * 5 >= mask * 3 the table is rebuilt for four times the items, or twice above 50000, at the smallest power of two above that, which drops the dummies. A merge into an empty set copies the slots when the tables have one size and no dummies, and otherwise places each item in its first empty slot. _extend(set, other, skip), _intersect, _subtract, _remove_all, _symmetric, _toggle, _replace and _relation take another collection by its address and read its type ID, as CPython's PyAnySet_Check and PyDict_CheckExact do: a set is combined table by table, a dict key by key with the hashes it stored, and a list or tuple[T, ...] item by item, in order (DAT-15). With a dict, _extend first resizes for every key, _subtract tests each item of a set at most four times larger than the dict and otherwise copies and removes, and _toggle removes or adds each key directly. They follow CPython's choices: an intersection tests the items of the smaller set, a difference copies and removes when the set is four times larger than the other, a difference update first intersects a set eight times larger and then rebuilds a table more than a quarter dummy, and ^ merges the right operand and updates with the left. With a list, an intersection adds each item the set holds until it is as large as the set, a difference copies the set and removes each item then rebuilds a table more than a quarter dummy, a symmetric difference update first makes a set of the list, and issubset intersects and compares sizes. _remove raises KeyError with the item as its argument and the repr of the item as its text, and _pop raises KeyError: 'pop from an empty set'; exception::key_error keeps the key in the exception's argument block with a layout of one field after it, so a tuple key needs no descriptor of the compiler's. _next(set, slot, length) returns the next active slot or -1, and raises RuntimeError: Set changed size during iteration when the set no longer has length items |
dict |
typy_rt_dict_new(key_kind, key_descriptor, value_kind, value_descriptor, capacity), _set, _find, _contains, _get, _get_or, _key, _value, _pop, _pop_or, _popitem, _setdefault, _length, _clear, _next, _copy, _update, _equal, _has_value, _has_item, _repr and _view_repr, a port of CPython's dictobject.c for a combined table (ADR-0010). A probe starts at hash & mask and jumps to (slot * 5 + perturb + 1) & mask, perturb shifted right by 5 each time. Setting a key that is there keeps its key and position and replaces the value; a new key is written after the last entry, and when no entry is usable the table is rebuilt for used * 3 slots, at least eight, at the next power of two, which drops the holes; a table holds two thirds of its slots in entries. capacity presizes a display's table, as CPython's BUILD_MAP does. Removing a key leaves a hole; popitem takes the last item and drops the entries after it. _copy clones a table at most a third holes and otherwise merges into a new dict; a merge into an empty dict clones a source without holes that is at the minimum size or more than half full, and otherwise resizes the target for both first when it has too few usable entries. _update takes a dict or a list of (key, value) tuples by its address. _next(dict, position, length, visited) returns the next entry that holds an item, or -1, and raises RuntimeError: dictionary changed size during iteration when the dict no longer has length items, and dictionary keys changed during iteration when it finds an item after visited reached length. _get and _pop raise KeyError with the key as its argument and its repr as its text, and _popitem raises KeyError: 'popitem(): dictionary is empty'. _view_repr writes dict_keys([...]), dict_values([...]) or dict_items([(key, value), ...]). typy_rt_dict_view(dict, kind) makes a view, and _view_dict, _view_length, _view_contains(view, item, pair) and _view_text read one; a pair looked for among the items is read through its tuple descriptor. typy_rt_dict_view_combine(left, right, operator, result) fills an empty set with \|, &, - or ^ of two operands, one a view, as CPython's dictviews_or, _PyDictView_Intersect, dictviews_sub and dictviews_xor do: the left operand becomes a set, a keys view from its dict, and is updated with the right one, read item by item; & intersects a set at least as large as the view with the view, puts the larger of two views first, and otherwise adds each item of the other operand that the view holds; ^ of two items views copies the left dict, takes out each key the right maps to an equal value, adds the right's other pairs, then the copy's pairs. typy_rt_dict_view_compare(left, right, comparison, kind, descriptor) compares by sizes, then by whether each item of the smaller side is in the other, as dictview_richcompare does; a pair tested against a view is looked up by its key, so its value needs no hash. The set functions read a view operand by its type ID, like a list |
module |
A module object is an immortal static object codegen emits for each module: header with the type ID 15, then its name and its path from the project root as str objects. The type typy_rt_type_module writes it <module 'm' from 'm.py'> (OI-20). typy_rt_module_attribute_error and typy_rt_module_import_error raise CPython's module 'm' has no attribute 'x' and cannot import name 'x' from 'm' (m.py); while the module initialises they add CPython 3.13 to 3.15's note (consider renaming 'm.py' if it has the same name as a library you intended to import), the newest oracle's wording (OI-14). typy_rt_name_error raises name 'x' is not defined |
range |
typy_rt_range_check_step raises ValueError: range() arg 3 must not be zero, and typy_rt_range_last(start, stop, step) returns start + (n - 1) * step for the n values of the range, by CPython's compute_range_length, on int words of any size: the last value, or start - step for an empty range. typy_rt_range_new makes a range value, with the same ValueError for a zero step; _length raises OverflowError: Python int too large to convert to C ssize_t for a length that is not inline, as len() does; _bound gives the start, stop or step; _get raises IndexError: range object index out of range; _slice builds a range from the slice's indices, as CPython's compute_slice does; _contains tests an int by arithmetic without stepping; _equal compares as == does. A slice of a range whose length passes sys.maxsize raises OverflowError, where CPython computes it with large integers |
zip |
typy_rt_zip_unequal(index, longer) raises CPython's ValueError for a strict zip: zip() argument 2 is shorter than argument 1, or zip() argument 3 is longer than arguments 1-2 |
stack |
The check at the start of recursive functions reads the stack pointer register and compares it with the recorded limit; the register is read rather than the address of a local, because a sanitizer may keep locals on a separate fake stack. The limit leaves 64 KiB of stack unused; past it the check raises RecursionError: maximum recursion depth exceeded |
An int word that is not inline is the address of a large integer. The
runtime makes it with expose_provenance and turns it back into a pointer
with with_exposed_provenance_mut, so Miri checks the accesses through it.
The archive holds TYPY_RUNTIME_VERSION, the bytes typy-runtime-version:,
the package version and a line feed, which RuntimeArchive checks; nothing
refers to it, so a program's link drops it. The runtime also defines
rust_eh_personality, which the precompiled core and compiler_builtins
name in their unwind tables; the runtime aborts on panic, so it is never
called.
When the module body returns an exception, main reports it and then
releases it, so a program that ends with an uncaught exception also frees
all its memory (NFR-08).
The runtime uses no core::fmt. Each of its unsafe blocks is covered by
tests under AddressSanitizer and Miri (ADR-0006).
15. typy-conformance¶
A conformance test is a Python file that CPython can run, with its expectations in comments at the top:
# typy-test: EXP-03
# typy-expect: accepted
def ratio(a: int, b: int) -> float:
return a / b
print(ratio(1, 4))
print(ratio(1, 0))
# typy-test: EXP-04
# typy-expect: rejected E0302 at 4:12
n: int = 3
print(2 ** n)
| Field | Rule |
|---|---|
typy-test |
One or more requirement IDs, separated by commas; each exists in the specification |
typy-expect |
accepted, or rejected CODE at LINE:COLUMN |
typy-python |
Optional; the test's Python floor, 3.12 when absent |
The file name starts with the first requirement ID, such as
EXP-03_float_division_by_zero_raises.py. A test of several modules is a
directory named so, such as MOD-01_modules_import_names/, whose
main.py holds the header and whose other files are its modules. For an
accepted test, the suite runs
python3.X -E -s -B -X utf8 -W ignore::SyntaxWarning FILE in the file's
directory for each oracle version from the test's Python floor up, which
keeps that directory first on the module path as -I would not, removes
the directory's absolute path from what CPython shows (OI-20), builds the
file
with Pipeline::run in debug and in release mode, and compares standard
output, exit status and the last line of standard error (UC-09). The
second test above is a valid Python program that CPython runs; TyPy rejects
it because the exponent is a variable.
| Module | Behaviour |
|---|---|
test_case |
TestCase::parse(path, text) reads the comment lines at the top of the file; a missing or malformed field, or a file name that does not start with the first ID, is a CaseError |
expectation |
Expectation::Accepted, or Rejected with a DiagnosticCode, a line and a column |
observation |
Observation::of(command) runs a program with no standard input, reads both pipes on their own threads, and keeps standard output, the exit status and the last line of standard error; a program that runs longer than 60 seconds is stopped, a limit set far above the run time of any conformance program so that an oracle slowed by a loaded machine is not taken for a hang |
oracle |
Runs python3.X -E -s -B -X utf8 -W ignore::SyntaxWarning FILE for each oracle version from the floor up, each in an empty working directory of its own, so a test that writes files writes nothing into the test tree and meets no other run's files; the file's directory still leads the module path. It removes that directory's absolute path from the output |
subject |
Builds an accepted test in debug and release mode into a temporary directory and runs each build in an empty working directory of its own, as the oracle runs, and with Suite::with_sanitizers also links its debug object file with cc -fsanitize=address and runs it with leak detection, where a report ends the run with status 86; checks a rejected test with Pipeline::check, which passes when exactly one diagnostic is reported, with that code at that line and column |
verdict |
Pass, Fail with the reason, or Invalid when the oracles disagree, an oracle cannot run the file, or the header is not valid (STD-03) |
suite |
Suite::run(root) judges every .py file under root, as many at once as the machine has cores, and returns a Report with each verdict and, for each requirement ID, how many accepted and rejected tests name it |
The suite runs as the tests crates/typy-conformance/tests/suite.rs and,
with the sanitizers, tests/sanitized.rs, so
make test and make check run it on every commit. That test fails on any
verdict but Pass, and on any requirement the test plan gives the suite for
0.1 that lacks an accepted or a rejected test. A test is built and checked
at its own floor, through Pipeline::with_overrides, as python-floor in
its typy.toml would set it (CLI-12).
16. Design class diagrams¶
The dependencies between these crates are shown in the building blocks diagram of architecture section 4.1.
17. Traceability for release 0.1¶
Each requirement whose release is 0.1, and the sections of this document that meet it. The test plan traces the same requirements to their tests.
| Requirement | Sections |
|---|---|
| STD-01 | 5.2 |
| STD-02 | 14.2, 15 |
| STD-03 | 15 |
| STD-04 | 7.2 |
| STD-06 | 5.1, 14.2 |
| STD-07 | 6 |
| CLI-01 | 11, 13 |
| CLI-02 | 13 |
| CLI-03 | 12, 13 |
| CLI-04 | 8.2, 11, 12 |
| CLI-05 | 10.1, 11 |
| CLI-06 | 13 |
| CLI-11 | 13 |
| LEX-01 | 5.1 |
| LEX-02 | 5.2 |
| LEX-03 | 5.2 |
| LEX-04 | 5.2 |
| LEX-05 | 5.2 |
| LEX-08 | 4.3, 7.2 |
| LEX-09 | 5.1 |
| TYP-01 | 7.1, 7.2 |
| TYP-02 | 7.2 |
| TYP-03 | 7.1, 7.2 |
| TYP-04 | 7.2 |
| TYP-09 | 7.1 |
| TYP-10 | 4.3, 7.1 |
| TYP-11 | 7.1, 8.1 |
| TYP-12 | 7.2 |
| DAT-01 | 8.1, 14.1 |
| DAT-02 | 14.2 |
| DAT-03 | 4.3, 7.2 |
| DAT-04 | 7.1, 8.1 |
| DAT-05 | 14.1, 14.2 |
| EXP-01 | 9 |
| EXP-02 | 9, 14.2 |
| EXP-03 | 9, 14.2 |
| EXP-04 | 4.3 |
| EXP-05 | 14.2 |
| EXP-06 | 4.3, 9 |
| EXP-08 | 4.3 |
| EXP-09 | 4.3, 9 |
| EXP-10 | 4.3 |
| EXP-11 | 4.3 |
| EXP-16 | 7.1 |
| SMT-01 | 7.2 |
| SMT-03 | 7.2 |
| CST-01 | 7.2, 9 |
| CST-02 | 7.2, 9 |
| FUN-01 | 7.2 |
| FUN-02 | 9, 14.2 |
| LIB-01 | 6 |
| LIB-09 | 6 |
| MEM-01 | 14.2 |
| DIA-01 | 4.1 |
| DIA-02 | 4.1, 4.2 |
| DIA-03 | 4.1, 7.2 |
| DIA-04 | 4.1, 7.1 |
| DIA-06 | 4.3 |
| NFR-01 | 11, 14.2 |
| NFR-02 | 14.2 |
| NFR-06 | 3, 10.1 |
| NFR-07 | 3, 14.2 |
| NFR-08 | 9 |
| NFR-09 | 3, 13 |
| NFR-10 | 15 |
| NFR-11 | 3 |
| NFR-15 | 9, 10.1 |