mypy.build

Add catch-all .gitignore to an existing directory.

No-op if the .gitignore already exists.

Analyze a program.

A single call to build performs parsing, semantic analysis and optionally type checking for the program and all imported modules, recursively.

Return BuildResult if successful or only non-blocking errors were found; otherwise raise CompileError.

If a flush_errors callback is provided, all error messages will be passed to it and the errors and messages fields of BuildResult and CompileError (respectively) will be empty. Otherwise those fields will report any error messages.

Args:

sources: list of sources to build options: build options alt_lib_path: an additional directory for looking up library modules

(takes precedence over other directories)

flush_errors: optional function to flush errors after a file is processed fscache: optionally a file-system cacher

Build a CacheMeta object from a json metadata dictionary

Args:

meta: JSON metadata read from the metadata cache file data_json: Path to the .data.json file containing the AST trees

Undocumented

Delete cache files for a module.

The cache files for a module are deleted when mypy finds errors there. This avoids inconsistent states with cache files from different mypy runs, see #4043 for an example.

Undocumented

Undocumented

Dump the graph as a JSON string to stdout.

This copies some of the work by process_graph() (sorted_components() and order_ascc()).

Exclude the directory from various archives and backups supporting CACHEDIR.TAG.

If the CACHEDIR.TAG file exists the function is a no-op.

Find if there are any newly added packages that were previously suppressed.

Exclude everything not in build for follow-imports=skip.

Find cache data for a module.

Args:

id: module ID path: module path manager: the build manager (for pyversion, log/trace, and build options)

Returns:

A CacheMeta instance if the cache data was found and appears valid; otherwise None.

Return the approximate location of setting_name within mypy config file.

Return -1 if can't determine the line unambiguously.

Find a module by name, respecting follow_imports and producing diagnostics.

If the module is not found, then the ModuleNotFound exception is raised.

Args:

id: module to find options: the options for the module being loaded caller_state: the state of the importing module, if applicable caller_line: the line number of the import ancestor_for: the child module this is an ancestor of, if applicable root_source: whether this source was specified on the command line skip_diagnose: skip any error diagnosis and reporting (but ModuleNotFound is

still raised if the module is missing)

The specified value of follow_imports for a module can be overridden if the module is specified on the command line or if it is a stub, so we compute and return the "effective" follow_imports of the module.

Returns a tuple containing (file path, target's effective follow_imports setting)

Generate fine-grained dependencies into a form suitable for serializing.

This does a couple things: 1. Splits fine-grained deps based on the module of the trigger 2. For each module we generated fine-grained deps for, load any previous

deps and merge them in.

Returns a dictionary from module ids to all dependencies on that module. Dependencies not associated with a module in the build will be associated with the nearest parent module that is in the build, or the fake module FAKE_ROOT_MODULE if none are.

Return the file names for the cache files.

Args:

id: module ID path: module path cache_dir: cache directory pyversion: Python version (major, minor)

Returns:

A tuple with the file names to be used for the meta JSON, the data JSON, and the fine-grained deps JSON, respectively.

Check if a missing module can potentially be a part of a package.

This checks if there is any existing parent __init__.pyi stub that defines a module-level __getattr__ (a.k.a. partial stub package).

Splits fine-grained dependencies based on the module of the trigger.

Returns a dictionary from module ids to all dependencies on that module. Dependencies not associated with a module in the build will be associated with the nearest parent module that is in the build, or the fake module FAKE_ROOT_MODULE if none are.

Undocumented

Given some source files, load the full dependency graph.

If an old_graph is passed in, it is used as the starting point and modified during graph loading.

If a new_modules is passed in, any modules that are loaded are added to the list. This is an argument and not a return value so that the caller can access it even if load_graph fails.

As this may need to parse files, this can raise CompileError in case there are syntax errors.

Load all configured plugins.

Return a plugin that encapsulates all plugins chained together. Always at least include the default plugin (it's last in the chain). The second return value is a snapshot of versions/hashes of loaded user plugins (for cache validation).

Load all configured plugins.

Return a list of all the loaded plugins from the config file. The second return value is a snapshot of versions/hashes of loaded user plugins (for cache validation).

Undocumented

Undocumented

Convert path to absolute; but to relative in bazel mode.

(Bazel's distributed cache doesn't like filesystem metadata to end up in output files.)

Come up with the ideal processing order within an SCC.

Using the priorities assigned by all_imported_modules_in_file(), try to reduce the cycle to a DAG, by omitting arcs representing dependencies of lower priority.

In the simplest case, if we have A <--> B where A has a top-level "import B" (medium priority) but B only has the reverse "import A" inside a function (low priority), we turn the cycle into a DAG by dropping the B --> A arc, which leaves only A --> B.

If all arcs have the same priority, we fall back to sorting by reverse global order (the order in which modules were first encountered).

The algorithm is recursive, as follows: when as arcs of different priorities are present, drop all arcs of the lowest priority, identify SCCs in the resulting graph, and apply the algorithm to each SCC thus found. The recursion is bounded because at each recursion the spread in priorities is (at least) one less.

In practice there are only a few priority levels (less than a dozen) and in the worst case we just carry out the same algorithm for finding SCCs N times. Thus the complexity is no worse than the complexity of the original SCC-finding algorithm -- see strongly_connected_components() below for a reference.

Process the modules in one group of modules from their cached data.

This can be used to process an SCC of modules This involves loading the tree from JSON and then doing various cleanups.

Process the modules in one SCC from source code.

Exception: If quick_and_dirty is set, use the cache for fresh modules.

Read and validate the fine-grained dependencies cache.

See the write_deps_cache documentation for more information on the details of the cache.

Returns None if the cache was invalid in some way.

Undocumented

Write a file containing missing stub packages.

This allows a subsequent "mypy --install-types" run (without other arguments) to install missing stub packages.

Return the graph's SCCs, topologically sorted by dependencies.

The sort order is from leaves (nodes without dependencies) to roots (nodes on which no other nodes depend).

This works for a subset of the full dependency graph too; dependencies that aren't present in graph.keys() are ignored.

Compute Strongly Connected Components of a directed graph.

Args:

vertices: the labels for the vertices edges: for each vertex, gives the target vertices of its outgoing edges

Returns:

An iterator yielding strongly connected components, each represented as a set of vertices. Each input vertex will occur exactly once; vertices not part of a SCC are returned as singleton sets.

From http://code.activestate.com/recipes/578507/.

Take plugin module snapshot by recording its version and hash.

We record _both_ hash and the version to detect more possible changes (e.g. if there is a change in modules imported by a plugin).

Topological sort.

Args:

data: A map from vertices to all vertices that it has an edge

connecting it to. NOTE: This data structure is modified in place -- for normalization purposes, self-dependencies are removed and entries representing orphans are added.

Returns:

An iterator yielding sets of vertices that have an equivalent ordering.

Example:

Suppose the input has the following structure:

{A: {B, C}, B: {D}, C: {D}}

This is normalized to:

{A: {B, C}, B: {D}, C: {D}, D: {}}

The algorithm will yield the following values:

{D} {B, C} {A}

From http://code.activestate.com/recipes/577413/.

Checks whether the cached AST of this module can be used.

Returns:

None, if the cached AST is unusable. Original meta, if mtime/size matched. Meta with mtime updated to match source file, if hash/size matched but mtime/path didn't.

Write cache files for a module.

Note that this mypy's behavior is still correct when any given write_cache() call is replaced with a no-op, so error handling code that bails without writing anything is okay.

Args:

id: module ID path: module path tree: the fully checked module data dependencies: module IDs on which this module depends suppressed: module IDs which were suppressed as dependencies dep_prios: priorities (parallel array to dependencies) dep_lines: import line locations (parallel array to dependencies) old_interface_hash: the hash from the previous version of the data cache file source_hash: the hash of the source code ignore_all: the ignore_all flag for this module manager: the build manager (for pyversion, log/trace)

Returns:

A tuple containing the interface hash and CacheMeta corresponding to the metadata that was written (the latter may be None if the cache could not be written).

Write cache files for fine-grained dependencies.

Serialize fine-grained dependencies map for fine grained mode.

Dependencies on some module 'm' is stored in the dependency cache file m.deps.json. This entails some spooky action at a distance: if module 'n' depends on 'm', that produces entries in m.deps.json. When there is a dependency on a module that does not exist in the build, it is stored with its first existing parent module. If no such module exists, it is stored with the fake module FAKE_ROOT_MODULE.

This means that the validity of the fine-grained dependency caches are a global property, so we store validity checking information for fine-grained dependencies in a global cache file:

• We take a snapshot of current sources to later check consistency between the fine-grained dependency cache and module cache metadata
• We store the mtime of all of the dependency files to verify they haven't changed