mypy.stubgen

module documentation
(source)

Generator of dynamically typed draft stubs for arbitrary modules.

The logic of this script can be split in three steps: * parsing options and finding sources:

  • use runtime imports be default (to find also C modules)
  • or use mypy's mechanisms, if importing is prohibited
  • (optionally) semantically analysing the sources using mypy (as a single set)
  • emitting the stubs text: - for Python modules: from ASTs using StubGenerator - for C modules using runtime introspection and (optionally) Sphinx docs

During first and third steps some problematic files can be skipped, but any blocking error during second step will cause the whole program to stop.

Basic usage:

$ stubgen foo.py bar.py some_directory => Generate out/foo.pyi, out/bar.pyi, and stubs for some_directory (recursively).

$ stubgen -m urllib.parse => Generate out/urllib/parse.pyi.

$ stubgen -p urllib => Generate stubs for whole urlib package (recursively).

For Python 2 mode, use --py2:

$ stubgen --py2 -m textwrap

For C modules, you can get more precise function signatures by parsing .rst (Sphinx) documentation for extra information. For this, use the --doc-dir option:

$ stubgen --doc-dir <DIR>/Python-3.4.2/Doc/library -m curses

Note: The generated stubs should be verified manually.

TODO:

  • support stubs for C modules in Python 2 mode

  • detect 'if PY2 / is_py2' etc. and either preserve those or only include Python 2 or 3 case

  • maybe use .rst docs also for Python modules

  • maybe export more imported names if there is no __all__ (this affects ssl.SSLError, for example) - a quick and dirty heuristic would be to turn this on if a module has something like

    'from x import y as _y'

  • we don't seem to always detect properties ('closed' in 'io', for example)

Class ​Alias​Printer Visitor used to collect type aliases _and_ type variable definitions.
Class ​Annotation​Printer Visitor used to print existing annotations in a file.
Class ​Definition​Finder Find names of things defined at the top level of a module.
Class ​Import​Tracker Record necessary imports during stub generation.
Class ​Options Represents stubgen options.
Class ​Reference​Finder Find all name references (both local and global).
Class ​Self​Traverser Undocumented
Class ​Stub​Generator Generate stub text from a mypy AST.
Class ​Stub​Source A single source for stub: can be a Python or C module.
Function collect​_build​_targets Collect files for which we need to generate stubs.
Function collect​_docs​_signatures Gather all function and class signatures in the docs.
Function find​_defined​_names Undocumented
Function find​_method​_names Undocumented
Function find​_module​_paths​_using​_imports Find path and runtime value of __all__ (if possible) for modules and packages.
Function find​_module​_paths​_using​_search Find sources for modules and packages requested.
Function find​_referenced​_names Undocumented
Function find​_self​_initializers Find attribute initializers in a method.
Function generate​_asts​_for​_modules Use mypy to parse (and optionally analyze) source files.
Function generate​_stub​_from​_ast Use analysed (or just parsed) AST to generate type stub for single file.
Function generate​_stubs Main entry point for the program.
Function get​_qualified​_name Undocumented
Function is​_blacklisted​_path Undocumented
Function is​_non​_library​_module Does module look like a test module or a script?
Function main Undocumented
Function mypy​_options Generate mypy options using the flag passed by user.
Function normalize​_path​_separators Undocumented
Function parse​_options Undocumented
Function parse​_source​_file Parse a source file.
Function remove​_blacklisted​_modules Undocumented
Function translate​_module​_name Undocumented
Constant BLACKLIST Undocumented
Constant CLASS Undocumented
Constant DESCRIPTION Undocumented
Constant EMPTY Undocumented
Constant EMPTY​_CLASS Undocumented
Constant ERROR​_MARKER Undocumented
Constant EXTRA​_EXPORTED Undocumented
Constant FUNC Undocumented
Constant HEADER Undocumented
Constant IGNORED​_DUNDERS Undocumented
Constant METHODS​_WITH​_RETURN​_VALUE Undocumented
Constant NOT​_IN​_ALL Undocumented
Constant VAR Undocumented
Constant VENDOR​_PACKAGES Undocumented
def collect_build_targets(options, mypy_opts): (source)

Collect files for which we need to generate stubs.

Return list of Python modules and C modules.

Parameters
options:OptionsUndocumented
mypy​_opts:MypyOptionsUndocumented
Returns
Tuple[List[StubSource], List[StubSource]]Undocumented
def collect_docs_signatures(doc_dir): (source)

Gather all function and class signatures in the docs.

Return a tuple (function signatures, class signatures). Currently only used for C modules.

Parameters
doc​_dir:strUndocumented
Returns
Tuple[Dict[str, str], Dict[str, str]]Undocumented
def find_defined_names(file): (source)

Undocumented

Parameters
file:MypyFileUndocumented
Returns
Set[str]Undocumented
def find_method_names(defs): (source)

Undocumented

Parameters
defs:List[Statement]Undocumented
Returns
Set[str]Undocumented
def find_module_paths_using_imports(modules, packages, interpreter, pyversion, verbose, quiet): (source)

Find path and runtime value of __all__ (if possible) for modules and packages.

This function uses runtime Python imports to get the information.

Parameters
modules:List[str]Undocumented
packages:List[str]Undocumented
interpreter:strUndocumented
pyversion:Tuple[int, int]Undocumented
verbose:boolUndocumented
quiet:boolUndocumented
Returns
Tuple[List[StubSource], List[StubSource]]Undocumented
def find_module_paths_using_search(modules, packages, search_path, pyversion): (source)

Find sources for modules and packages requested.

This function just looks for source files at the file system level. This is used if user passes --no-import, and will not find C modules. Exit if some of the modules or packages can't be found.

Parameters
modules:List[str]Undocumented
packages:List[str]Undocumented
search​_path:List[str]Undocumented
pyversion:Tuple[int, int]Undocumented
Returns
List[StubSource]Undocumented
def find_referenced_names(file): (source)

Undocumented

Parameters
file:MypyFileUndocumented
Returns
Set[str]Undocumented
def find_self_initializers(fdef): (source)

Find attribute initializers in a method.

Return a list of pairs (attribute name, r.h.s. expression).

Parameters
fdef:FuncBaseUndocumented
Returns
List[Tuple[str, Expression]]Undocumented
def generate_asts_for_modules(py_modules, parse_only, mypy_options, verbose): (source)
Use mypy to parse (and optionally analyze) source files.
Parameters
py​_modules:List[StubSource]Undocumented
parse​_only:boolUndocumented
mypy​_options:MypyOptionsUndocumented
verbose:boolUndocumented
def generate_stub_from_ast(mod, target, parse_only=False, pyversion=defaults.PYTHON3_VERSION, include_private=False, export_less=False): (source)

Use analysed (or just parsed) AST to generate type stub for single file.

If directory for target doesn't exist it will created. Existing stub will be overwritten.

Parameters
mod:StubSourceUndocumented
target:strUndocumented
parse​_only:boolUndocumented
pyversion:Tuple[int, int]Undocumented
include​_private:boolUndocumented
export​_less:boolUndocumented
def generate_stubs(options): (source)
Main entry point for the program.
Parameters
options:OptionsUndocumented
def get_qualified_name(o): (source)

Undocumented

Parameters
o:ExpressionUndocumented
Returns
strUndocumented
def is_blacklisted_path(path): (source)

Undocumented

Parameters
path:strUndocumented
Returns
boolUndocumented
def is_non_library_module(module): (source)
Does module look like a test module or a script?
Parameters
module:strUndocumented
Returns
boolUndocumented
def main(): (source)

Undocumented

def mypy_options(stubgen_options): (source)
Generate mypy options using the flag passed by user.
Parameters
stubgen​_options:OptionsUndocumented
Returns
MypyOptionsUndocumented
def normalize_path_separators(path): (source)

Undocumented

Parameters
path:strUndocumented
Returns
strUndocumented
def parse_options(args): (source)

Undocumented

Parameters
args:List[str]Undocumented
Returns
OptionsUndocumented
def parse_source_file(mod, mypy_options): (source)

Parse a source file.

On success, store AST in the corresponding attribute of the stub source. If there are syntax errors, print them and exit.

Parameters
mod:StubSourceUndocumented
mypy​_options:MypyOptionsUndocumented
def remove_blacklisted_modules(modules): (source)

Undocumented

Parameters
modules:List[StubSource]Undocumented
Returns
List[StubSource]Undocumented
def translate_module_name(module, relative): (source)

Undocumented

Parameters
module:strUndocumented
relative:intUndocumented
Returns
Tuple[str, int]Undocumented
BLACKLIST: list[str] = (source)

Undocumented

Value
['/six.py\n', '/vendored/', '/vendor/', '/_vendor/', '/_vendored_packages/']
CLASS: str = (source)

Undocumented

Value
'CLASS'
DESCRIPTION: str = (source)

Undocumented

Value
'''
Generate draft stubs for modules.

Stubs are generated in directory ./out, to avoid overriding files with
manual changes.  This directory is assumed to exist.
'''
EMPTY: str = (source)

Undocumented

Value
'EMPTY'
EMPTY_CLASS: str = (source)

Undocumented

Value
'EMPTY_CLASS'
ERROR_MARKER: str = (source)

Undocumented

Value
'<ERROR>'
EXTRA_EXPORTED: set[str] = (source)

Undocumented

Value
set(['pyasn1_modules.rfc2437.univ',
     'pyasn1_modules.rfc2459.char',
     'pyasn1_modules.rfc2459.univ'])
FUNC: str = (source)

Undocumented

Value
'FUNC'
HEADER: str = (source)

Undocumented

Value
'''%(prog)s [-h] [--py2] [more options, see -h]
                     [-m MODULE] [-p PACKAGE] [files ...]'''
IGNORED_DUNDERS: set[str] = (source)

Undocumented

Value
set(['__all__',
     '__author__',
     '__version__',
     '__about__',
     '__copyright__',
     '__email__',
     '__license__',
...
METHODS_WITH_RETURN_VALUE: set[str] = (source)

Undocumented

Value
set(['__ne__',
     '__eq__',
     '__lt__',
     '__le__',
     '__gt__',
     '__ge__',
     '__hash__',
...
NOT_IN_ALL: str = (source)

Undocumented

Value
'NOT_IN_ALL'
VAR: str = (source)

Undocumented

Value
'VAR'
VENDOR_PACKAGES: list[str] = (source)

Undocumented

Value
['packages', 'vendor', 'vendored', '_vendor', '_vendored_packages']