Python

This article introduces the Python programming language.

Overview

Python is a widely used high-level programming language for general-purpose programming, created by Guido van Rossum and first released in 1991. As an interpreted language, Python has a design philosophy which emphasizes code readability (notably using whitespace indentation to delimit code blocks rather than curly braces or keywords), and a syntax which allows programmers to express concepts in fewer lines of code than possible in languages such as C++ or Java. The language provides constructs intended to enable writing clear programs on both a small and large scale.

Python features a dynamic type system and automatic memory management and supports multiple programming paradigms, including object-oriented, imperative, functional programming, and procedural styles. It has a large and comprehensive standard library.

Python interpreters are available for many operating systems, allowing Python code to run on a wide variety of systems. CPython, the reference implementation of Python, is open source software and has a community-based development model, as do nearly all of its variant implementations. CPython is managed by the non-profit Python Software Foundation (PSF).

Python Releases

Refer to Python downloads page for Python releases, and What’s New in Python for release notes.

Refer to Should I use Python 2 or Python 3 for my development activity for the critical differences between Python 2.x and 3.x. Also refer to Porting Python 2 Code to Python 3.

According to PEP 373: Python 2.7 Release Schedule, the End Of Life date (EOL, sunset date) for Python 2.7 has been moved five years into the future, to 2020. This decision was made to clarify the status of Python 2.7 and relieve worries for those users who cannot yet migrate to Python 3. See also PEP 466: Network Security Enhancements for Python 2.7.x. This declaration does not guarantee that bugfix releases will be made on a regular basis, but it should enable volunteers who want to contribute bugfixes for Python 2.7 and it should satisfy vendors who still have to support Python 2 for years to come.

According to PEP 404: Python 2.8 Un-release Schedule:

  • There is no official Python 2.8 release. There never will be an official Python 2.8 release. It is an ex-release. Python 2.7 is the end of the Python 2 line of development.
  • The official upgrade path from Python 2.7 is to Python 3.

And the Python 2.7 Countdown shows a down counter of Python 2.7 retirement. The moving to require Python 3 shows the projects that have pledged to drop support for Python 2.7 no later than 2020.

Python 2

Python 2.0.x

Release Date
2.0.1 2001-06-22

Python 2.1.x

Release Date
2.1.3 2002-04-09

Python 2.2.x

Release Date
2.2.0 2001-12-21
2.2.1 2002-04-10
2.2.2 2002-10-14
2.2.3 2003-05-30

Python 2.3.x

Release Date
2.3.0 2003-07-29
2.3.1 2003-09-23
2.3.2 2003-10-03
2.3.3 2003-12-19
2.3.4 2004-05-27
2.3.5 2005-02-08
2.3.6 2006-11-01
2.3.7 2008-03-11

Python 2.4.x

Release Date
2.4.0 2004-11-30
2.4.1 2005-03-30
2.4.2 2005-09-27
2.4.3 2006-04-15
2.4.4 2006-10-18
2.4.5 2008-03-11
2.4.6 2008-12-19

Python 2.5.x

Release Date
2.5.0 2006-09-19
2.5.1 2007-04-19
2.5.2 2008-02-21
2.5.3 2008-12-19
2.5.4 2008-12-23
2.5.5 2010-01-31
2.5.6 2011-05-26

Python 2.6.x

Release Date
2.6.0 2008-10-02
2.6.1 2008-12-04
2.6.2 2009-04-14
2.6.3 2009-10-02
2.6.4 2009-10-26
2.6.5 2010-03-18
2.6.6 2010-08-24
2.6.7 2011-06-03
2.6.8 2012-04-10
2.6.9 2013-10-29

Python 2.7.x

Release Date
2.7.0 2010-07-03
2.7.1 2010-11-27
2.7.2 2011-06-11
2.7.3 2012-04-09
2.7.4 2013-04-06
2.7.5 2013-05-12
2.7.6 2013-11-10
2.7.7 2014-06-01
2.7.8 2014-07-02
2.7.9 2014-12-10
2.7.10 2015-05-23
2.7.11 2015-12-05
2.7.12 2016-06-25
2.7.13 2016-12-17
2.7.14 2017-09-16
2.7.15 2018-05-01
2.7.16 2019-03-04
2.7.17 2019-10-19
2.7.18 2020-04-20

Python 3

Python 3.0.x

Release Date
3.0.0 2008-12-03
3.0.1 2009-02-13

Python 3.1.x

Release Date
3.1.0 2009-06-26
3.1.1 2009-08-17
3.1.2 2010-03-20
3.1.3 2010-11-27
3.1.4 2011-06-11
3.1.5 2012-04-09

Python 3.2.x

Release Date
3.2.0 2011-02-20
3.2.1 2011-07-09
3.2.2 2011-09-03
3.2.3 2012-04-10
3.2.4 2013-04-06
3.2.5 2013-05-15
3.2.6 2014-10-12

Python 3.3.x

Release Date
3.3.0 2012-09-29
3.3.1 2013-04-06
3.3.2 2013-05-15
3.3.3 2013-11-17
3.3.4 2014-02-09
3.3.5 2014-03-09
3.3.6 2014-10-12
3.3.7 2017-09-19

Python 3.4.x

Release Date
3.4.0 2014-03-17
3.4.1 2014-05-19
3.4.2 2014-10-13
3.4.3 2015-02-25
3.4.4 2015-12-21
3.4.5 2016-06-27
3.4.6 2017-01-17
3.4.7 2017-08-09
3.4.8 2018-02-05
3.4.9 2018-08-02
3.4.10 2019-03-18

Python 3.5.x

Release Date
3.5.0 2015-09-13
3.5.1 2015-12-07
3.5.2 2016-06-27
3.5.3 2017-01-17
3.5.4 2017-08-08
3.5.5 2018-02-05
3.5.6 2018-08-02
3.5.7 2019-03-18
3.5.8 2019-10-29
3.5.9 2019-11-02

Python 3.6.x

Release Date
3.6.0 2016-12-23
3.6.1 2017-03-21
3.6.2 2017-07-17
3.6.3 2017-10-03
3.6.4 2017-12-19
3.6.5 2018-03-28
3.6.6 2018-06-27
3.6.7 2018-10-20
3.6.8 2018-12-24
3.6.9 2019-07-02
3.6.10 2019-12-18

Python 3.7.x

Release Date
3.7.0 2018-06-27
3.7.1 2018-10-20
3.7.2 2018-12-24
3.7.3 2019-03-25
3.7.4 2019-07-08
3.7.5 2019-10-15
3.7.6 2019-12-18
3.7.7 2020-03-10

Python 3.8.x

Release Date
3.8.0 2019-10-14
3.8.1 2019-12-18
3.8.2 2020-02-24

Python Software Foundation (PSF)

The Python Software Foundation (PSF) is an independent non-profit organization that holds the copyright on Python versions 2.1 and newer. The PSF’s mission is to advance open source technology related to the Python programming language and to publicize the use of Python.

Python Enhancement Proposals (PEPs)

Python’s development is conducted largely through the Python Enhancement Proposal (PEP) process. The PEP process is the primary mechanism for proposing major new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. Outstanding PEPs are reviewed and commented upon by the Python community and by Van Rossum, the Python project’s benevolent dictator for life. Refer to Index of Python Enhancement Proposals (PEPs) and Python Developer’s Guide.

Python Implementations

There are some alternate implementations which are of particular interest to different audiences. Refer to Alternative Python Implementations for more information. The known implementations include:

  • CPython

    This is the original and most-maintained implementation of Python, written in C. New language features generally appear here first. Refer to CPython for more information.

  • Jython

    Python implemented in Java. This implementation can be used as a scripting language for Java applications, or can be used to create applications using the Java class libraries. It is also often used to create tests for Java libraries. More information can be found at the Jython website.

  • Python for .NET

    This implementation actually uses the CPython implementation, but is a managed .NET application and makes .NET libraries available. It was created by Brian Lloyd. For more information, see the Python for .NET website.

  • IronPython

    An alternate Python for .NET. Unlike Python.NET, this is a complete Python implementation that generates IL, and compiles Python code directly to .NET assemblies. It was created by Jim Hugunin, the original creator of Jython. For more information, see the IronPython website.

  • PyPy

    An implementation of Python written completely in Python. It supports several advanced features not found in other implementations like stackless support and a Just in Time compiler (JIT). One of the goals of the project is to encourage experimentation with the language itself by making it easier to modify the interpreter (since it is written in Python). Additional information is available on the PyPy website.

CPython

The latest development sources of Python can be obtained at Git Repo:

chenwx@chenwx ~ $ git clone https://github.com/python/cpython.git

Directory structure of CPython source tree is listed below:

chenwx@chenwx ~ $ cd cpython/
chenwx@chenwx ~/cpython $ ls
Doc	 Mac		  Modules  Programs    config.guess   configure.ac
Grammar  Makefile	  Objects  Python      config.log     install-sh
Include  Makefile.pre	  PC	   README.rst  config.status  pyconfig.h
LICENSE  Makefile.pre.in  PCbuild  Tools       config.sub     pyconfig.h.in
Lib	 Misc		  Parser   aclocal.m4  configure      setup.py
  • Doc

    The official documentation. This is what https://docs.python.org/ uses. See also Building the documentation.

  • Grammar

    Contains the Extended Backus-Naur Form (EBNF) grammar file for Python.

  • Include

    Contains all interpreter-wide header files.

  • Lib

    The part of the standard library implemented in pure Python.

  • Mac

    Mac-specific code (e.g., using IDLE as an OS X application).

  • Misc

    Things that do not belong elsewhere. Typically this is varying kinds of developer-specific documentation.

  • Modules

    The part of the standard library (plus some other code) that is implemented in C.

  • Objects

    Code for all built-in types.

  • PC

    Windows-specific code.

  • PCbuild

    Build files for the version of MSVC currently used for the Windows installers provided on python.org.

  • Parser

    Code related to the parser. The definition of the AST nodes is also kept here.

  • Programs

    Source code for C executables, including the main function for the CPython interpreter (in versions prior to Python 3.5, these files are in the Modules directory).

  • Python

    The code that makes up the core CPython runtime. This includes the compiler, eval loop and various built-in modules.

  • Tools

    Various tools that are (or have been) used to maintain Python.

Building CPython

On Debian, Ubuntu, and other apt based systems, try to get the dependencies for the Python version that you’re working on. If that package is not available for your system, try reducing the minor version until you find a package that is available.

chenwx@chenwx ~ $ sudo apt-get build-dep python3.5

Run the following command to build Python from source code, refer to Getting Started section of Python Developer’s Guide for details:

chenwx@chenwx ~ $ cd cpython/

chenwx@chenwx ~/cpython $ git co v3.6.1
HEAD is now at 69c0db5050f6... Update docs and patchlevel for 3.6.1 final

chenwx@chenwx ~/cpython $ ./configure --with-pydebug
...
configure: creating ./config.status
config.status: creating Makefile.pre
config.status: creating Modules/Setup.config
config.status: creating Misc/python.pc
config.status: creating Misc/python-config.sh
config.status: creating Modules/ld_so_aix
config.status: creating pyconfig.h
config.status: pyconfig.h is unchanged
creating Modules/Setup
creating Modules/Setup.local
creating Makefile

chenwx@chenwx ~/cpython $ make -s -j2
...
Python build finished successfully!
The necessary bits to build these optional modules were not found:
_bz2                  _lzma
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
atexit                pwd                   time

Failed to build these modules:
_tkinter

chenwx@chenwx ~/cpython $ ll python
-rwxrwxr-x 1 chenwx chenwx 11M Mar 29 17:33 python
chenwx@chenwx ~/cpython $ ./python -V
Python 3.6.1
chenwx@chenwx ~/cpython $ ./python -VV
Python 3.6.1 (v3.6.1:69c0db5050f6, Mar 29 2017, 17:33:44)
[GCC 5.4.0 20160609]

There is normally no need to install your built copy of Python because which is already installed by Linux distributions, refer to Python Installation.

Python Installation

Refer to PEP 394: The “python” Command on Unix-Like Systems, the Python is already installed on my LinuxMint:

chenwx@chenwx ~ $ which python
/usr/bin/python

chenwx@chenwx ~ $ ll /usr/bin/python
lrwxrwxrwx 1 root root 9 Dec 10  2015 /usr/bin/python -> python2.7

chenwx@chenwx ~ $ which python2
/usr/bin/python2

chenwx@chenwx ~ $ ll /usr/bin/python2
lrwxrwxrwx 1 root root 9 Dec 10  2015 /usr/bin/python2 -> python2.7

chenwx@chenwx ~ $ which python3
/usr/bin/python3

chenwx@chenwx ~ $ ll /usr/bin/python3
lrwxrwxrwx 1 root root 9 Mar 23  2016 /usr/bin/python3 -> python3.5

chenwx@chenwx ~ $ which python3m
/usr/bin/python3m

chenwx@chenwx ~ $ ll /usr/bin/python3m
lrwxrwxrwx 1 root root 10 Mar 23  2016 /usr/bin/python3m -> python3.5m

chenwx@chenwx ~ $ diff /usr/bin/python3.5 /usr/bin/python3.5m

chenwx@chenwx ~ $ ll /usr/bin/python*
lrwxrwxrwx 1 root root    9 Dec 10  2015 /usr/bin/python -> python2.7
lrwxrwxrwx 1 root root   16 Dec 10  2015 /usr/bin/python-config -> python2.7-config
lrwxrwxrwx 1 root root    9 Dec 10  2015 /usr/bin/python2 -> python2.7
lrwxrwxrwx 1 root root   16 Dec 10  2015 /usr/bin/python2-config -> python2.7-config
-rwxr-xr-x 1 root root 3.4M Nov 19 17:35 /usr/bin/python2.7
lrwxrwxrwx 1 root root   33 Nov 19 17:35 /usr/bin/python2.7-config -> x86_64-linux-gnu-python2.7-config
lrwxrwxrwx 1 root root    9 Mar 23  2016 /usr/bin/python3 -> python3.5
-rwxr-xr-x 2 root root 4.3M Nov 18 03:23 /usr/bin/python3.5
-rwxr-xr-x 2 root root 4.3M Nov 18 03:23 /usr/bin/python3.5m
lrwxrwxrwx 1 root root   10 Mar 23  2016 /usr/bin/python3m -> python3.5m

Alternatives to Python Interpreter

One alternative enhanced interactive interpreter that has been around for quite some time is IPython, which features tab completion, object exploration and advanced history management. It can also be thoroughly customized and embedded into other applications.

Another similar enhanced interactive environment is bpython.

Python Commands

Command Options

When invoking Python, you may specify any of these options, refer to command line and environment:

python2 [ -bBdEiOQRsStuUvVWxX3? ] [ -c command | -m module-name | script | - ] [ args ]
python3 [ -bBdEhiIOqsSuvVWx? ]    [ -c command | -m module-name | script | - ] [ args ]
Options Description
-?
-h
--help
Print a short description of all command line options.
-V
--version
-VV
Print the Python version number and exit. When given twice (-VV), print more information about the build.
-v
-vv
Print a message each time a module is initialized, showing the place (filename or built-in module) from which it is loaded. When given twice (-vv), print a message for each file that is checked for when searching for a module. Also provides information on module cleanup at exit. See also PYTHONVERBOSE.
-b
-bb
Issue a warning when comparing bytes or bytearray with str, or bytes with int. Issue an error when the option is given twice (-bb). Changed in version 3.5: Affects comparisons of bytes with int.
-B If given, Python won’t try to write .pyc files on the import of source modules. See also PYTHONDONTWRITEBYTECODE.
-d Turn on parser debugging output (for wizards only, depending on compilation options). See also PYTHONDEBUG.
-E Ignore all PYTHON* environment variables, e.g. PYTHONPATH and PYTHONHOME, that might be set, see Environment Variables.
-i When a script is passed as first argument or the -c option is used, enter interactive mode after executing the script or the command, even when sys.stdin does not appear to be a terminal. The PYTHONSTARTUP file is not read. This can be useful to inspect global variables or a stack trace when a script raises an exception. See also PYTHONINSPECT.
-I Run Python in isolated mode. This also implies -E and -s. In isolated mode sys.path contains neither the script’s directory nor the user’s site-packages directory. All PYTHON* environment variables are ignored too. Further restrictions may be imposed to prevent the user from injecting malicious code. New in version 3.4.
-O Turn on basic optimizations. See also PYTHONOPTIMIZE.
-OO Discard docstrings in addition to the -O optimizations.
-q Don’t display the copyright and version messages even in interactive mode. New in version 3.2.
-R Kept for compatibility. On Python 3.3 and greater, hash randomization is turned on by default. On previous versions of Python, this option turns on hash randomization, so that the __hash__() values of str, bytes and datetime are salted with an unpredictable random value. Although they remain constant within an individual Python process, they are not predictable between repeated invocations of Python. New in version 3.2.3. Changed in version 3.7: The option is no longer ignored.
-s Don’t add the user site-packages directory to sys.path, see Python FAQ and PEP 370: Per user site-packages directory.
-S Disable the import of the module site and the site-dependent manipulations of sys.path that it entails. Also disable these manipulations if site is explicitly imported later (call site.main() if you want them to be triggered).
-t Issue a warning when a source file mixes tabs and spaces for indentation in a way that makes it depend on the worth of a tab expressed in spaces. Issue an error when the option is given twice (-tt).
-u Force the binary layer of the stdout and stderr streams (which is available as their buffer attribute) to be unbuffered. The text I/O layer will still be line-buffered if writing to the console, or block-buffered if redirected to a non-interactive file. See also PYTHONUNBUFFERED.
-W arg Warning control. Python’s warning machinery by default prints warning messages to sys.stderr.
-x Skip the first line of the source, allowing use of non-Unix forms of #!cmd. This is intended for a DOS specific hack only.
-X Reserved for various implementation-specific options.
-J Reserved for use by Jython. You shouldn’t use this option.
-3 Warn about Python 3.x possible incompatibilities by emitting a DeprecationWarning for features that are removed or significantly changed in Python 3 and can’t be detected using static code analysis. New in version 2.6. See Porting Python 2 Code to Python 3 for more details.
args Arguments passed to program in sys.argv[1:].

Run the following command to print a short description of all command line options:

chenwx@chenwx ~ $ python -V
Python 2.7.12

chenwx@chenwx ~ $ python -h
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
-B     : don't write .py[co] files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d     : debug output from parser; also PYTHONDEBUG=x
-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-h     : print this help message and exit (also --help)
-i     : inspect interactively after running script; forces a prompt even
         if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-m mod : run library module as a script (terminates option list)
-O     : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x
-OO    : remove doc-strings in addition to the -O optimizations
-R     : use a pseudo-random salt to make hash() values of various types be
         unpredictable between separate invocations of the interpreter, as
         a defense against denial-of-service attacks
-Q arg : division options: -Qold (default), -Qwarn, -Qwarnall, -Qnew
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S     : don't imply 'import site' on initialization
-t     : issue warnings about inconsistent tab usage (-tt: issue errors)
-u     : unbuffered binary stdout and stderr; also PYTHONUNBUFFERED=x
         see man page for details on internal buffering relating to '-u'
-v     : verbose (trace import statements); also PYTHONVERBOSE=x
         can be supplied multiple times to increase verbosity
-V     : print the Python version number and exit (also --version)
-W arg : warning control; arg is action:message:category:module:lineno
         also PYTHONWARNINGS=arg
-x     : skip first line of source, allowing use of non-Unix forms of #!cmd
-3     : warn about Python 3.x incompatibilities that 2to3 cannot trivially fix
file   : program read from script file
-      : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]

Other environment variables:
PYTHONSTARTUP: file executed on interactive startup (no default)
PYTHONPATH   : ':'-separated list of directories prefixed to the
               default module search path.  The result is sys.path.
PYTHONHOME   : alternate <prefix> directory (or <prefix>:<exec_prefix>).
               The default module search path uses <prefix>/pythonX.X.
PYTHONCASEOK : ignore case in 'import' statements (Windows).
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONHASHSEED: if this variable is set to 'random', the effect is the same
   as specifying the -R option: a random value is used to seed the hashes of
   str, bytes and datetime objects.  It can also be set to an integer
   in the range [0,4294967295] to get hash values with a predictable seed.
chenwx@chenwx:~ $ python3 -V
Python 3.6.7

chenwx@chenwx:~ $ python3 -h
usage: python3 [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
-b     : issue warnings about str(bytes_instance), str(bytearray_instance)
         and comparing bytes/bytearray with str. (-bb: issue errors)
-B     : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d     : debug output from parser; also PYTHONDEBUG=x
-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-h     : print this help message and exit (also --help)
-i     : inspect interactively after running script; forces a prompt even
         if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I     : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O     : remove assert and __debug__-dependent statements; add .opt-1 before
         .pyc extension; also PYTHONOPTIMIZE=x
-OO    : do -O changes and also discard docstrings; add .opt-2 before
         .pyc extension
-q     : don't print version and copyright messages on interactive startup
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S     : don't imply 'import site' on initialization
-u     : force the binary I/O layers of stdout and stderr to be unbuffered;
         stdin is always buffered; text I/O layer will be line-buffered;
         also PYTHONUNBUFFERED=x
-v     : verbose (trace import statements); also PYTHONVERBOSE=x
         can be supplied multiple times to increase verbosity
-V     : print the Python version number and exit (also --version)
         when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
         also PYTHONWARNINGS=arg
-x     : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
file   : program read from script file
-      : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]

Other environment variables:
PYTHONSTARTUP: file executed on interactive startup (no default)
PYTHONPATH   : ':'-separated list of directories prefixed to the
               default module search path.  The result is sys.path.
PYTHONHOME   : alternate <prefix> directory (or <prefix>:<exec_prefix>).
               The default module search path uses <prefix>/lib/pythonX.X.
PYTHONCASEOK : ignore case in 'import' statements (Windows).
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONFAULTHANDLER: dump the Python traceback on fatal errors.
PYTHONHASHSEED: if this variable is set to 'random', a random value is used
   to seed the hashes of str, bytes and datetime objects.  It can also be
   set to an integer in the range [0,4294967295] to get hash values with a
   predictable seed.
PYTHONMALLOC: set the Python memory allocators and/or install debug hooks
   on Python memory allocators. Use PYTHONMALLOC=debug to install debug
   hooks.

Or, you can find descriptions of those options in man page:

chenwx@chenwx ~ $ man python
chenwx@chenwx ~ $ man python3

Or, run the following command to print a message each time a module is initialized, showing the place (filename or built-in module) from which it is loaded:

chenwx@chenwx ~ $ python -v
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# /usr/lib/python2.7/site.pyc matches /usr/lib/python2.7/site.py
import site # precompiled from /usr/lib/python2.7/site.pyc
# /usr/lib/python2.7/os.pyc matches /usr/lib/python2.7/os.py
import os # precompiled from /usr/lib/python2.7/os.pyc
import errno # builtin
import posix # builtin
# /usr/lib/python2.7/posixpath.pyc matches /usr/lib/python2.7/posixpath.py
import posixpath # precompiled from /usr/lib/python2.7/posixpath.pyc
# /usr/lib/python2.7/stat.pyc matches /usr/lib/python2.7/stat.py
import stat # precompiled from /usr/lib/python2.7/stat.pyc
# /usr/lib/python2.7/genericpath.pyc matches /usr/lib/python2.7/genericpath.py
import genericpath # precompiled from /usr/lib/python2.7/genericpath.pyc
# /usr/lib/python2.7/warnings.pyc matches /usr/lib/python2.7/warnings.py
import warnings # precompiled from /usr/lib/python2.7/warnings.pyc
# /usr/lib/python2.7/linecache.pyc matches /usr/lib/python2.7/linecache.py
import linecache # precompiled from /usr/lib/python2.7/linecache.pyc
# /usr/lib/python2.7/types.pyc matches /usr/lib/python2.7/types.py
import types # precompiled from /usr/lib/python2.7/types.pyc
# /usr/lib/python2.7/UserDict.pyc matches /usr/lib/python2.7/UserDict.py
import UserDict # precompiled from /usr/lib/python2.7/UserDict.pyc
# /usr/lib/python2.7/_abcoll.pyc matches /usr/lib/python2.7/_abcoll.py
import _abcoll # precompiled from /usr/lib/python2.7/_abcoll.pyc
# /usr/lib/python2.7/abc.pyc matches /usr/lib/python2.7/abc.py
import abc # precompiled from /usr/lib/python2.7/abc.pyc
# /usr/lib/python2.7/_weakrefset.pyc matches /usr/lib/python2.7/_weakrefset.py
import _weakrefset # precompiled from /usr/lib/python2.7/_weakrefset.pyc
import _weakref # builtin
# /usr/lib/python2.7/copy_reg.pyc matches /usr/lib/python2.7/copy_reg.py
import copy_reg # precompiled from /usr/lib/python2.7/copy_reg.pyc
# /usr/lib/python2.7/traceback.pyc matches /usr/lib/python2.7/traceback.py
import traceback # precompiled from /usr/lib/python2.7/traceback.pyc
# /usr/lib/python2.7/sysconfig.pyc matches /usr/lib/python2.7/sysconfig.py
import sysconfig # precompiled from /usr/lib/python2.7/sysconfig.pyc
# /usr/lib/python2.7/re.pyc matches /usr/lib/python2.7/re.py
import re # precompiled from /usr/lib/python2.7/re.pyc
# /usr/lib/python2.7/sre_compile.pyc matches /usr/lib/python2.7/sre_compile.py
import sre_compile # precompiled from /usr/lib/python2.7/sre_compile.pyc
import _sre # builtin
# /usr/lib/python2.7/sre_parse.pyc matches /usr/lib/python2.7/sre_parse.py
import sre_parse # precompiled from /usr/lib/python2.7/sre_parse.pyc
# /usr/lib/python2.7/sre_constants.pyc matches /usr/lib/python2.7/sre_constants.py
import sre_constants # precompiled from /usr/lib/python2.7/sre_constants.pyc
import _locale # builtin
# /usr/lib/python2.7/_sysconfigdata.pyc matches /usr/lib/python2.7/_sysconfigdata.py
import _sysconfigdata # precompiled from /usr/lib/python2.7/_sysconfigdata.pyc
# /usr/lib/python2.7/plat-x86_64-linux-gnu/_sysconfigdata_nd.pyc matches /usr/lib/python2.7/plat-x86_64-linux-gnu/_sysconfigdata_nd.py
import _sysconfigdata_nd # precompiled from /usr/lib/python2.7/plat-x86_64-linux-gnu/_sysconfigdata_nd.pyc
# /usr/lib/python2.7/sitecustomize.pyc matches /usr/lib/python2.7/sitecustomize.py
import sitecustomize # precompiled from /usr/lib/python2.7/sitecustomize.pyc
import encodings # directory /usr/lib/python2.7/encodings
# /usr/lib/python2.7/encodings/__init__.pyc matches /usr/lib/python2.7/encodings/__init__.py
import encodings # precompiled from /usr/lib/python2.7/encodings/__init__.pyc
# /usr/lib/python2.7/codecs.pyc matches /usr/lib/python2.7/codecs.py
import codecs # precompiled from /usr/lib/python2.7/codecs.pyc
import _codecs # builtin
# /usr/lib/python2.7/encodings/aliases.pyc matches /usr/lib/python2.7/encodings/aliases.py
import encodings.aliases # precompiled from /usr/lib/python2.7/encodings/aliases.pyc
# /usr/lib/python2.7/encodings/ascii.pyc matches /usr/lib/python2.7/encodings/ascii.py
import encodings.ascii # precompiled from /usr/lib/python2.7/encodings/ascii.pyc
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
dlopen("/usr/lib/python2.7/lib-dynload/readline.x86_64-linux-gnu.so", 2);
import readline # dynamically loaded from /usr/lib/python2.7/lib-dynload/readline.x86_64-linux-gnu.so
>>> exit()
# clear __builtin__._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.exitfunc
# clear sys.exc_type
# clear sys.exc_value
# clear sys.exc_traceback
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup __main__
# cleanup[1] encodings
# cleanup[1] site
# cleanup[1] sysconfig
# cleanup[1] abc
# cleanup[1] _weakrefset
# cleanup[1] sre_constants
# cleanup[1] re
# cleanup[1] _codecs
# cleanup[1] _warnings
# cleanup[1] zipimport
# cleanup[1] _sysconfigdata
# cleanup[1] encodings.ascii
# cleanup[1] codecs
# cleanup[1] readline
# cleanup[1] _sysconfigdata_nd
# cleanup[1] _locale
# cleanup[1] sitecustomize
# cleanup[1] signal
# cleanup[1] traceback
# cleanup[1] posix
# cleanup[1] encodings.aliases
# cleanup[1] exceptions
# cleanup[1] _weakref
# cleanup[1] sre_compile
# cleanup[1] _sre
# cleanup[1] sre_parse
# cleanup[2] copy_reg
# cleanup[2] posixpath
# cleanup[2] errno
# cleanup[2] _abcoll
# cleanup[2] types
# cleanup[2] genericpath
# cleanup[2] stat
# cleanup[2] warnings
# cleanup[2] UserDict
# cleanup[2] os.path
# cleanup[2] linecache
# cleanup[2] os
# cleanup sys
# cleanup __builtin__
# cleanup ints: 19 unfreed ints
# cleanup floats

chenwx@chenwx ~ $ python3 -v
import _frozen_importlib # frozen
import _imp # builtin
import sys # builtin
import '_warnings' # <class '_frozen_importlib.BuiltinImporter'>
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'>
import '_io' # <class '_frozen_importlib.BuiltinImporter'>
import 'marshal' # <class '_frozen_importlib.BuiltinImporter'>
import 'posix' # <class '_frozen_importlib.BuiltinImporter'>
import _thread # previously loaded ('_thread')
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import _weakref # previously loaded ('_weakref')
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
# installing zipimport hook
import 'zipimport' # <class '_frozen_importlib.BuiltinImporter'>
# installed zipimport hook
# /usr/lib/python3.5/encodings/__pycache__/__init__.cpython-35.pyc matches /usr/lib/python3.5/encodings/__init__.py
# code object from '/usr/lib/python3.5/encodings/__pycache__/__init__.cpython-35.pyc'
# /usr/lib/python3.5/__pycache__/codecs.cpython-35.pyc matches /usr/lib/python3.5/codecs.py
# code object from '/usr/lib/python3.5/__pycache__/codecs.cpython-35.pyc'
import '_codecs' # <class '_frozen_importlib.BuiltinImporter'>
import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8456d128>
# /usr/lib/python3.5/encodings/__pycache__/aliases.cpython-35.pyc matches /usr/lib/python3.5/encodings/aliases.py
# code object from '/usr/lib/python3.5/encodings/__pycache__/aliases.cpython-35.pyc'
import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d844fcbe0>
import 'encodings' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d84563cf8>
# /usr/lib/python3.5/encodings/__pycache__/ascii.cpython-35.pyc matches /usr/lib/python3.5/encodings/ascii.py
# code object from '/usr/lib/python3.5/encodings/__pycache__/ascii.cpython-35.pyc'
import 'encodings.ascii' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8450aa90>
import '_signal' # <class '_frozen_importlib.BuiltinImporter'>
# /usr/lib/python3.5/encodings/__pycache__/utf_8.cpython-35.pyc matches /usr/lib/python3.5/encodings/utf_8.py
# code object from '/usr/lib/python3.5/encodings/__pycache__/utf_8.cpython-35.pyc'
import 'encodings.utf_8' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8450d588>
# /usr/lib/python3.5/encodings/__pycache__/latin_1.cpython-35.pyc matches /usr/lib/python3.5/encodings/latin_1.py
# code object from '/usr/lib/python3.5/encodings/__pycache__/latin_1.cpython-35.pyc'
import 'encodings.latin_1' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8450d828>
# /usr/lib/python3.5/__pycache__/io.cpython-35.pyc matches /usr/lib/python3.5/io.py
# code object from '/usr/lib/python3.5/__pycache__/io.cpython-35.pyc'
# /usr/lib/python3.5/__pycache__/abc.cpython-35.pyc matches /usr/lib/python3.5/abc.py
# code object from '/usr/lib/python3.5/__pycache__/abc.cpython-35.pyc'
# /usr/lib/python3.5/__pycache__/_weakrefset.cpython-35.pyc matches /usr/lib/python3.5/_weakrefset.py
# code object from '/usr/lib/python3.5/__pycache__/_weakrefset.cpython-35.pyc'
import '_weakrefset' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d845124e0>
import 'abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8450dcc0>
import 'io' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8450da58>
# /usr/lib/python3.5/__pycache__/site.cpython-35.pyc matches /usr/lib/python3.5/site.py
# code object from '/usr/lib/python3.5/__pycache__/site.cpython-35.pyc'
# /usr/lib/python3.5/__pycache__/os.cpython-35.pyc matches /usr/lib/python3.5/os.py
# code object from '/usr/lib/python3.5/__pycache__/os.cpython-35.pyc'
import 'errno' # <class '_frozen_importlib.BuiltinImporter'>
# /usr/lib/python3.5/__pycache__/stat.cpython-35.pyc matches /usr/lib/python3.5/stat.py
# code object from '/usr/lib/python3.5/__pycache__/stat.cpython-35.pyc'
import '_stat' # <class '_frozen_importlib.BuiltinImporter'>
import 'stat' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d844b89e8>
# /usr/lib/python3.5/__pycache__/posixpath.cpython-35.pyc matches /usr/lib/python3.5/posixpath.py
# code object from '/usr/lib/python3.5/__pycache__/posixpath.cpython-35.pyc'
# /usr/lib/python3.5/__pycache__/genericpath.cpython-35.pyc matches /usr/lib/python3.5/genericpath.py
# code object from '/usr/lib/python3.5/__pycache__/genericpath.cpython-35.pyc'
import 'genericpath' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d844c1278>
import 'posixpath' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d844b8d30>
# /usr/lib/python3.5/__pycache__/_collections_abc.cpython-35.pyc matches /usr/lib/python3.5/_collections_abc.py
# code object from '/usr/lib/python3.5/__pycache__/_collections_abc.cpython-35.pyc'
import '_collections_abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d844c1908>
import 'os' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d845271d0>
# /usr/lib/python3.5/__pycache__/_sitebuiltins.cpython-35.pyc matches /usr/lib/python3.5/_sitebuiltins.py
# code object from '/usr/lib/python3.5/__pycache__/_sitebuiltins.cpython-35.pyc'
import '_sitebuiltins' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d845274a8>
# /usr/lib/python3.5/__pycache__/sysconfig.cpython-35.pyc matches /usr/lib/python3.5/sysconfig.py
# code object from '/usr/lib/python3.5/__pycache__/sysconfig.cpython-35.pyc'
import 'sysconfig' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8447c0b8>
# /usr/lib/python3.5/__pycache__/_sysconfigdata.cpython-35.pyc matches /usr/lib/python3.5/_sysconfigdata.py
# code object from '/usr/lib/python3.5/__pycache__/_sysconfigdata.cpython-35.pyc'
# /usr/lib/python3.5/plat-x86_64-linux-gnu/__pycache__/_sysconfigdata_m.cpython-35.pyc matches /usr/lib/python3.5/plat-x86_64-linux-gnu/_sysconfigdata_m.py
# code object from '/usr/lib/python3.5/plat-x86_64-linux-gnu/__pycache__/_sysconfigdata_m.cpython-35.pyc'
import '_sysconfigdata_m' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d84481b70>
import '_sysconfigdata' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d84481978>
# /usr/lib/python3.5/__pycache__/sitecustomize.cpython-35.pyc matches /usr/lib/python3.5/sitecustomize.py
# code object from '/usr/lib/python3.5/__pycache__/sitecustomize.cpython-35.pyc'
import 'sitecustomize' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8448fcf8>
import 'site' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d84519cc0>
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
# extension module 'readline' loaded from '/usr/lib/python3.5/lib-dynload/readline.cpython-35m-x86_64-linux-gnu.so'
# extension module 'readline' executed from '/usr/lib/python3.5/lib-dynload/readline.cpython-35m-x86_64-linux-gnu.so'
import 'readline' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7f3d8449c208>
import 'atexit' # <class '_frozen_importlib.BuiltinImporter'>
# /usr/lib/python3.5/__pycache__/rlcompleter.cpython-35.pyc matches /usr/lib/python3.5/rlcompleter.py
# code object from '/usr/lib/python3.5/__pycache__/rlcompleter.cpython-35.pyc'
import 'rlcompleter' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8449c320>
>>> exit()
# clear builtins._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.__interactivehook__
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup[2] removing encodings.utf_8
# cleanup[2] removing codecs
# cleanup[2] removing encodings.ascii
# cleanup[2] removing os
# cleanup[2] removing site
# destroy site
# cleanup[2] removing posixpath
# cleanup[2] removing encodings.latin_1
# cleanup[2] removing builtins
# cleanup[2] removing _codecs
# cleanup[2] removing _sysconfigdata_m
# destroy _sysconfigdata_m
# cleanup[2] removing io
# destroy io
# cleanup[2] removing _weakrefset
# destroy _weakrefset
# cleanup[2] removing sysconfig
# destroy sysconfig
# cleanup[2] removing _stat
# cleanup[2] removing zipimport
# cleanup[2] removing readline
# cleanup[2] removing sitecustomize
# destroy sitecustomize
# cleanup[2] removing _imp
# cleanup[2] removing marshal
# cleanup[2] removing stat
# cleanup[2] removing _io
# cleanup[2] removing atexit
# cleanup[2] removing genericpath
# cleanup[2] removing _warnings
# cleanup[2] removing _weakref
# cleanup[2] removing os.path
# cleanup[2] removing errno
# cleanup[2] removing _sysconfigdata
# destroy _sysconfigdata
# cleanup[2] removing _frozen_importlib_external
# cleanup[2] removing sys
# cleanup[2] removing __main__
# cleanup[2] removing encodings.aliases
# cleanup[2] removing _thread
# cleanup[2] removing _frozen_importlib
# cleanup[2] removing rlcompleter
# destroy rlcompleter
# cleanup[2] removing encodings
# destroy encodings
# cleanup[2] removing _signal
# cleanup[2] removing abc
# destroy abc
# cleanup[2] removing _sitebuiltins
# cleanup[2] removing _collections_abc
# destroy _collections_abc
# cleanup[2] removing posix
# destroy zipimport
# destroy _signal
# destroy _sitebuiltins
# destroy posixpath
# destroy errno
# destroy _stat
# destroy genericpath
# destroy stat
# destroy os
# destroy __main__
# destroy readline
# destroy atexit
# cleanup[3] wiping encodings.utf_8
# cleanup[3] wiping codecs
# cleanup[3] wiping encodings.ascii
# cleanup[3] wiping encodings.latin_1
# cleanup[3] wiping _codecs
# cleanup[3] wiping _imp
# cleanup[3] wiping marshal
# cleanup[3] wiping _io
# cleanup[3] wiping _warnings
# cleanup[3] wiping _weakref
# cleanup[3] wiping _frozen_importlib_external
# destroy io
# destroy marshal
# cleanup[3] wiping encodings.aliases
# cleanup[3] wiping _thread
# cleanup[3] wiping _frozen_importlib
# destroy _weakref
# destroy _thread
# destroy _imp
# destroy _warnings
# destroy _frozen_importlib_external
# cleanup[3] wiping posix
# destroy posix
# cleanup[3] wiping sys
# cleanup[3] wiping builtins
# destroy _frozen_importlib

Also you can run command python -vv or python3 -vv to print a message for each file that is checked for when searching for a module.

Environment Variables

The environment variables influence Python’s behavior, they are processed before the command-line switches other than -E or -I. It is customary that command-line switches override environmental variables where there is a conflict. Refer to command line and environment for details.

  • PYTHONHOME

    Change the location of the standard Python libraries. By default, the libraries are searched in prefix/lib/pythonversion and exec_prefix/lib/pythonversion, where prefix and exec_prefix are installation-dependent directories, both defaulting to /usr/local.

    When PYTHONHOME is set to a single directory, its value replaces both prefix and exec_prefix. To specify different values for these, set PYTHONHOME to prefix:exec_prefix.

  • PYTHONPATH

    Augment the default search path for module files. The format is the same as the shell’s PATH: one or more directory pathnames separated by os.pathsep, e.g. colons on Unix or semicolons on Windows. Non-existent directories are silently ignored.

    In addition to normal directories, individual PYTHONPATH entries may refer to zipfiles containing pure Python modules (in either source or compiled form). Extension modules cannot be imported from zipfiles.

    The default search path is installation dependent, but generally begins with prefix/lib/pythonversion, see PYTHONHOME above. It is always appended to PYTHONPATH.

    An additional directory will be inserted in the search path in front of PYTHONPATH as described above under Interface options. The search path can be manipulated from within a Python program as the variable sys.path.

  • PYTHONSTARTUP

    If this is the name of a readable file, the Python commands in that file are executed before the first prompt is displayed in interactive mode. The file is executed in the same namespace where interactive commands are executed so that objects defined or imported in it can be used without qualification in the interactive session. You can also change the prompts sys.ps1 and sys.ps2 and the hook sys.__interactivehook__ in this file.

  • PYTHONOPTIMIZE

    If this is set to a non-empty string it is equivalent to specifying the -O option. If set to an integer, it is equivalent to specifying -O multiple times.

  • PYTHONDEBUG

    If this is set to a non-empty string it is equivalent to specifying the -d option. If set to an integer, it is equivalent to specifying -d multiple times.

  • PYTHONINSPECT

    If this is set to a non-empty string it is equivalent to specifying the -i option. This variable can also be modified by Python code using os.environ to force inspect mode on program termination.

  • PYTHONUNBUFFERED

    If this is set to a non-empty string it is equivalent to specifying the -u option.

  • PYTHONVERBOSE

    If this is set to a non-empty string it is equivalent to specifying the -v option. If set to an integer, it is equivalent to specifying -v multiple times.

  • PYTHONCASEOK

    If this is set, Python ignores case in import statements. This only works on Windows and OS X.

  • PYTHONDONTWRITEBYTECODE

    If this is set to a non-empty string, Python won’t try to write .pyc files on the import of source modules. This is equivalent to specifying the -B option.

  • PYTHONHASHSEED

    If this variable is not set or set to random, a random value is used to seed the hashes of str, bytes and datetime objects.

    New in version 3.2.3.

  • PYTHONIOENCODING

    If this is set before running the interpreter, it overrides the encoding used for stdin, stdout, stderr, in the syntax encodingname:errorhandler. Both the encodingname and the :errorhandler parts are optional and have the same meaning as in str.encode().

  • PYTHONNOUSERSITE

    If this is set, Python won’t add the user site-packages directory to sys.path, refer to PEP 370: Per user site-packages directory.

  • PYTHONUSERBASE

    Defines the user base directory, which is used to compute the path of the user site-packages directory and Distutils installation paths for python setup.py install --user.

  • PYTHONEXECUTABLE

    If this environment variable is set, sys.argv[0] will be set to its value instead of the value got through the C runtime. Only works on Mac OS X.

  • PYTHONWARNINGS

    This is equivalent to the -W option. If set to a comma separated string, it is equivalent to specifying -W multiple times.

  • PYTHONFAULTHANDLER

    If this environment variable is set to a non-empty string, faulthandler.enable() is called at startup: install a handler for SIGSEGV, SIGFPE, SIGABRT, SIGBUS and SIGILL signals to dump the Python traceback. This is equivalent to -X faulthandler option.

    New in version 3.3.

  • PYTHONTRACEMALLOC

    If this environment variable is set to a non-empty string, start tracing Python memory allocations using the tracemalloc module. The value of the variable is the maximum number of frames stored in a traceback of a trace. For example, PYTHONTRACEMALLOC=1 stores only the most recent frame. See the tracemalloc.start() for more information.

    New in version 3.4.

  • PYTHONASYNCIODEBUG

    If this environment variable is set to a non-empty string, enable the debug mode of the asyncio module.

    New in version 3.4.

  • PYTHONMALLOC

    Set the Python memory allocators and/or install debug hooks.

    New in version 3.6.

  • PYTHONMALLOCSTATS

    If set to a non-empty string, Python will print statistics of the pymalloc memory allocator every time a new pymalloc object arena is created, and on shutdown.

  • PYTHONLEGACYWINDOWSFSENCODING

    If set to a non-empty string, the default filesystem encoding and errors mode will revert to their pre-3.6 values of mbcs and replace, respectively. Otherwise, the new defaults utf-8 and surrogatepass are used. This may also be enabled at runtime with sys._enablelegacywindowsfsencoding().

    Availability: Windows

  • PYTHONLEGACYWINDOWSIOENCODING

    If set to a non-empty string, does not use the new console reader and writer. This means that Unicode characters will be encoded according to the active console code page, rather than using utf-8. This variable is ignored if the standard streams are redirected (to files or pipes) rather than referring to console buffers.

    Availability: Windows

    New in version 3.6.

  • PYTHONTHREADDEBUG

    Setting this variable only has an effect in a debug build of Python, that is, if Python was configured with the --with-pydebug build option.

    If set, Python will print threading debug info.

  • PYTHONDUMPREFS

    Setting this variable only has an effect in a debug build of Python, that is, if Python was configured with the --with-pydebug build option.

    If set, Python will dump objects and reference counts still alive after shutting down the interpreter.

Python Language

Python documentation:

For C or C++ programmers, two additional manuals exist:

And you can find the index of all functions, classes, terms on:

Keywords

The following identifiers are used as reserved words, or keywords of the language, and cannot be used as ordinary identifiers. They must be spelled exactly as written here. More information can be found at Simple statements and Compound statements in Python 3.x Language Reference.

False      class      finally    is         return
None       continue   for        lambda     try
True       def        from       nonlocal   while
and        del        global     not        with
as         elif       if         or         yield
assert     else       import     pass
break      except     in         raise

The keyword module allows a Python program to determine if a string is a keyword. Its source code is located in Lib/keyword.py:

chenwx@chenwx ~/cpython $ ll Lib/keyword.py
-rwxrwxr-x 1 chenwx chenwx 2.2K Mar 26 18:40 Lib/keyword.py

chenwx@chenwx ~/cpython $ python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> import keyword

>>> keyword.kwlist
['False', 'None', 'True', 'and', 'as', 'assert', 'break', 'class', 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', 'not', 'or', 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield']

>>> print(*keyword.kwlist)
False None True and as assert break class continue def del elif else except finally for from global if import in is lambda nonlocal not or pass raise return try while with yield

>>> keyword.iskeyword("for")
True

Operators

The following tokens are operators:

+       -       *       **      /       //      %      @
<<      >>      &       |       ^       ~
<       >       <=      >=      ==      !=

Python_Operator_Precedence

Python operator precedence:

Python_Operator_Precedence

The operator module exports a set of efficient functions corresponding to the intrinsic operators of Python. Its source code is located in Lib/operator.py:

chenwx@chenwx ~/cpython $ python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> import operator

>>> dir(operator)
['__abs__', '__add__', '__all__', '__and__', '__builtins__', '__cached__', '__concat__', '__contains__', '__delitem__', '__doc__', '__eq__', '__file__', '__floordiv__', '__ge__', '__getitem__', '__gt__', '__iadd__', '__iand__', '__iconcat__', '__ifloordiv__', '__ilshift__', '__imatmul__', '__imod__', '__imul__', '__index__', '__inv__', '__invert__', '__ior__', '__ipow__', '__irshift__', '__isub__', '__itruediv__', '__ixor__', '__le__', '__loader__', '__lshift__', '__lt__', '__matmul__', '__mod__', '__mul__', '__name__', '__ne__', '__neg__', '__not__', '__or__', '__package__', '__pos__', '__pow__', '__rshift__', '__setitem__', '__spec__', '__sub__', '__truediv__', '__xor__', '_abs', 'abs', 'add', 'and_', 'attrgetter', 'concat', 'contains', 'countOf', 'delitem', 'eq', 'floordiv', 'ge', 'getitem', 'gt', 'iadd', 'iand', 'iconcat', 'ifloordiv', 'ilshift', 'imatmul', 'imod', 'imul', 'index', 'indexOf', 'inv', 'invert', 'ior', 'ipow', 'irshift', 'is_', 'is_not', 'isub', 'itemgetter', 'itruediv', 'ixor', 'le', 'length_hint', 'lshift', 'lt', 'matmul', 'methodcaller', 'mod', 'mul', 'ne', 'neg', 'not_', 'or_', 'pos', 'pow', 'rshift', 'setitem', 'sub', 'truediv', 'truth', 'xor']

>>> print(*dir(operator))
__abs__ __add__ __all__ __and__ __builtins__ __cached__ __concat__ __contains__ __delitem__ __doc__ __eq__ __file__ __floordiv__ __ge__ __getitem__ __gt__ __iadd__ __iand__ __iconcat__ __ifloordiv__ __ilshift__ __imatmul__ __imod__ __imul__ __index__ __inv__ __invert__ __ior__ __ipow__ __irshift__ __isub__ __itruediv__ __ixor__ __le__ __loader__ __lshift__ __lt__ __matmul__ __mod__ __mul__ __name__ __ne__ __neg__ __not__ __or__ __package__ __pos__ __pow__ __rshift__ __setitem__ __spec__ __sub__ __truediv__ __xor__ _abs abs add and_ attrgetter concat contains countOf delitem eq floordiv ge getitem gt iadd iand iconcat ifloordiv ilshift imatmul imod imul index indexOf inv invert ior ipow irshift is_ is_not isub itemgetter itruediv ixor le length_hint lshift lt matmul methodcaller mod mul ne neg not_ or_ pos pow rshift setitem sub truediv truth xor

Delimiters

The following tokens serve as delimiters in the grammar:

(       )       [       ]       {       }
,       :       .       ;       @       =       ->
+=      -=      *=      /=      //=     %=      @=
&=      |=      ^=      >>=     <<=     **=

The period (.) can also occur in floating-point and imaginary literals. A sequence of three periods (...) has a special meaning as an ellipsis literal. The second half of the list, the augmented assignment operators, serve lexically as delimiters, but also perform an operation.

The following printing ASCII characters have special meaning as part of other tokens or are otherwise significant to the lexical analyzer:

'       "       #       \

The following printing ASCII characters are not used in Python. Their occurrence outside string literals and comments is an unconditional error:

$       ?       `

Comments

One Line Comments

Typically, you just use the hash character (#) to comment out everything that follows it on that line. A comment may appear at the start of a line or following whitespace or code, but not within a string literal. A hash character (#) within a string literal is just a hash character.

# this is the first comment
spam = 1  # and this is the second comment
          # ... and now a third!
text = "# This is not a comment because it's inside quotes."

Multiple Line Comments

Multiple line comments simply use 3 single quotes (''') before and after the part you want commented:

'''
print("We are in a comment")
print("We are still in a comment")
'''
print("We are out of the comment")

Encoding Declaration

If a comment in the first or second line of the Python script matches the regular expression (see PEP 263: Defining Python Source Code Encodings):

^[ \t\v]*#.*?coding[:=][ \t]*([-_.a-zA-Z0-9]+)

or, the regular expression (see encoding declarations):

coding[=:]\s*([-\w.]+)

this comment is processed as an encoding declaration. The first group of this expression names the encoding of the source code file. If the encoding is unknown to Python, an error is raised during compilation. More supported encoding can be found at:

There must not be any Python statement on the line that contains the encoding declaration, that’s the encoding declaration must appear on a line of its own. If the first line matches the second line is ignored; if it is the second line, the first line must also be a comment-only line, for instance:

#!/usr/bin/python
# -*- coding: <encoding-name> -*-

or,

#!/usr/bin/python
# vim: set fileencoding=<encoding name> :

The recommended forms of an encoding expression are

# -*- coding: <encoding-name> -*-

which is recognized also by GNU Emacs, and

# vim: set fileencoding=<encoding name> :

which is recognized by Bram Moolenaar’s VIM.

If no encoding declaration is found, the default encoding is UTF-8, refer to PEP 3120: Using UTF-8 as the default source encoding. If an encoding is declared, the encoding name must be recognized by Python. The encoding is used for all lexical analysis, including string literals, comments and identifiers. Refer to PEP 263: Defining Python Source Code Encodings.

Indentation

Indentation is Python’s way of grouping statements. Each line within a basic block must be indented by the same amount.

Leading whitespace (spaces and tabs) at the beginning of a logical line is used to compute the indentation level of the line, which in turn is used to determine the grouping of statements.

Tabs are replaced (from left to right) by one to eight spaces such that the total number of characters up to and including the replacement is a multiple of eight (this is intended to be the same rule as used by Unix). The total number of spaces preceding the first non-blank character then determines the line’s indentation. Indentation cannot be split over multiple physical lines using backslashes; the whitespace up to the first backslash determines the indentation.

Indentation is rejected as inconsistent if a source file mixes tabs and spaces in a way that makes the meaning dependent on the worth of a tab in spaces; a TabError is raised in that case.

Tab or spaces?

According to PEP 8: Style Guide for Python Code, spaces are the preferred indentation method. Tabs should be used solely to remain consistent with code that is already indented with tabs.

Python 3 disallows mixing the use of tabs and spaces for indentation. Python 2 code indented with a mixture of tabs and spaces should be converted to using spaces exclusively. When invoking the Python 2 command line interpreter with the -t option, it issues warnings about code that illegally mixes tabs and spaces. When using -tt these warnings become errors. These options are highly recommended!

Data Structures

Lists

Run the following command to check help information of list:

>>> help(list)

Create a list:

>>> squares = []
>>> for x in range(10):
...     squares.append(x**2)
...
>>> squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

>>> type(squares)
<class 'list'>

or,

>>> squares = [x**2 for x in range(10)]
>>> squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

or,

>>> squares = list(map(lambda x: x**2, range(10)))
>>> squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Lists are mutable:

>>> squares[0] = 10
>>> squares[1] = 11
>>> squares
[10, 11, 4, 9, 16, 25, 36, 49, 64, 81]

There is a way to remove an item from a list given its index instead of its value: the del statement. This differs from the pop() method which returns a value. The del statement can also be used to remove slices from a list or clear the entire list.

>>> squares.pop()
81
>>> squares
[10, 11, 4, 9, 16, 25, 36, 49, 64]

>>> squares.pop()
64
>>> squares
[10, 11, 4, 9, 16, 25, 36, 49]

>>> del squares[0]
>>> squares
[11, 4, 9, 16, 25, 36, 49]

>>> del squares[2:4]
>>> squares
[11, 4, 25, 36, 49]

>>> del squares[:]
>>> squares
[]

The del statement can also be used to delete entire variables:

>>> squares = [x**2 for x in range(10)]
>>> squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

>>> del squares
>>> squares
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'squares' is not defined

Referencing the name squares hereafter is an error, at least until another value is assigned to it.

Tuples

Run the following command to check help information of tuple:

>>> help(tuple)

A tuple consists of a number of values separated by commas:

>>> t = (12345, 54321, 'hello!')    # or, t = 12345, 54321, 'hello!'
>>> t
(12345, 54321, 'hello!')
>>> t[0]
12345

>>> type(t)
<class 'tuple'>

Tuples may be nested:

>>> u = (t, (1, 2, 3, 4, 5))        # or, u = t, (1, 2, 3, 4, 5)
>>> u
((12345, 54321, 'hello!'), (1, 2, 3, 4, 5))

Tuples are immutable. It is not possible to assign to the individual items of a tuple, however it is possible to create tuples which contain mutable objects, such as lists:

>>> v = ([1, 2, 3], [3, 2, 1])      # or, v = [1, 2, 3], [3, 2, 1]
>>> v
([1, 2, 3], [3, 2, 1])

>>> v[0] = [11, 22, 33]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment

>>> v[0][1] = 22
>>> v
([1, 22, 3], [3, 2, 1])

A special problem is the construction of tuples containing 0 or 1 items: the syntax has some extra quirks to accommodate these. Empty tuples are constructed by an empty pair of parentheses; a tuple with one item is constructed by following a value with a comma (it is not sufficient to enclose a single value in parentheses). Ugly, but effective.

>>> empty = ()
>>> len(empty)
0
>>> empty
()

>>> singleton = ('hello',)          # or, singleton = 'hello',  <-- note trailing comma
>>> len(singleton)
1
>>> singleton
('hello',)

This is sequence unpacking and works for any sequence on the right-hand side. Sequence unpacking requires that there are as many variables on the left side of the equals sign as there are elements in the sequence. Note that multiple assignment is really just a combination of tuple packing and sequence unpacking.

>>> t = (12345, 54321, 'hello!')    # or, t = 12345, 54321, 'hello!'
>>> t
(12345, 54321, 'hello!')

>>> x, y, z = t
>>> x
12345
>>> y
54321
>>> z
'hello!'

Sets

Run the following command to check help information of set:

>>> help(set)

A set is an unordered collection with no duplicate elements. Basic uses include membership testing and eliminating duplicate entries. Set objects also support mathematical operations like union, intersection, difference, and symmetric difference.

Curly braces ({}) or the set() function can be used to create sets. Note that in order to create an empty set you have to use set(), not {}; the latter creates an empty dictionary.

>>> basket = {'apple', 'orange', 'apple', 'pear', 'orange', 'banana'}
>>> print(basket)                      # show that duplicates have been removed
{'orange', 'banana', 'pear', 'apple'}
>>> basket
{'orange', 'pear', 'banana', 'apple'}

>>> type(basket)
<class 'set'>

>>> 'orange' in basket                 # fast membership testing
True
>>> 'crabgrass' in basket
False

Demonstrate set() operations on unique letters from two words:

>>> a = set('abracadabra')
>>> a                                  # unique letters in a
{'a', 'r', 'b', 'c', 'd'}

>>> b = set('alacazam')
>>> b                                  # unique letters in b
{'z', 'a', 'm', 'c', 'l'}

>>> a - b                              # letters in a but not in b
{'r', 'd', 'b'}
>>> a | b                              # letters in either a or b
{'a', 'c', 'r', 'd', 'b', 'm', 'z', 'l'}
>>> a & b                              # letters in both a and b
{'a', 'c'}
>>> a ^ b                              # letters in a or b but not both
{'r', 'd', 'b', 'm', 'z', 'l'}

Dictionaries

Run the following command to check help information of dict:

>>> help(dict)

Unlike sequences, which are indexed by a range of numbers, dictionaries are indexed by keys, which can be any immutable type; strings and numbers can always be keys.

It is best to think of a dictionary as an unordered set of key:value pairs, with the requirement that the keys are unique (within one dictionary). A pair of braces creates an empty dictionary: {}. Placing a comma-separated list of key:value pairs within the braces adds initial key:value pairs to the dictionary; this is also the way dictionaries are written on output. If you store using a key that is already in use, the old value associated with that key is forgotten.

>>> tel = {}
>>> tel
{}

>>> type(tel)
<class 'dict'>

>>> tel = {'jack': 4098, 'sape': 4140}
>>> tel
{'sape': 4140, 'jack': 4098}

>>> tel = {'jack': 4098, 'sape': 4139, 'sape': 4141}
>>> tel
{'sape': 4141, 'jack': 4098}

The main operations on a dictionary are storing a value with some key and extracting the value given the key. It is also possible to delete a key:value pair with del statement. It is an error to extract a value using a non-existent key. To check whether a single key is in the dictionary, use the in keyword.

>>> tel = {'jack': 4098, 'sape': 4139}
>>> tel
{'sape': 4139, 'jack': 4098}

>>> tel['guido'] = 4127
>>> tel
{'sape': 4139, 'guido': 4127, 'jack': 4098}

>>> tel['jack']
4098

>>> del tel['sape']
{'guido': 4127, 'jack': 4098}

>>> tel['irv'] = 4127
>>> tel
{'guido': 4127, 'irv': 4127, 'jack': 4098}

>>> list(tel.keys())
['irv', 'guido', 'jack']

>>> sorted(tel.keys())
['guido', 'irv', 'jack']

>>> 'guido' in tel
True

>>> 'jack' not in tel
False

The dict() constructor builds dictionaries directly from sequences of key:value pairs:

>>> dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])
{'sape': 4139, 'jack': 4098, 'guido': 4127}

In addition, dict comprehensions can be used to create dictionaries from arbitrary key and value expressions:

>>> {x: x**2 for x in (2, 4, 6)}
{2: 4, 4: 16, 6: 36}

When the keys are simple strings, it is sometimes easier to specify pairs using keyword arguments:

>>> dict(sape=4139, guido=4127, jack=4098)
{'sape': 4139, 'jack': 4098, 'guido': 4127}

When looping through dictionaries, the key and corresponding value can be retrieved at the same time using the items() method.

>>> knights = {'gallahad': 'the pure', 'robin': 'the brave'}
>>> for k, v in knights.items():
...     print(k, v)
...
gallahad the pure
robin the brave

Statements

Multiple Assignment

The expressions on the right-hand side are all evaluated first before any of the assignments take place. The right-hand side expressions are evaluated from the left to the right.

# Fibonacci series: the sum of two elements defines the next
a, b = 0, 1
while (b < 10)
    print(b)
    a, b = b, a+b

assert

The assert statements are a convenient way to insert debugging assertions into a program:

assert <expression> [, <expression> ]

The simple form, assert <expression>, is equivalent to

if __debug__:
    if not <expression>: raise AssertionError

The extended form, assert <expression1>, <expression2>, is equivalent to

if __debug__:
    if not <expression1>: raise AssertionError(<expression2>)

These equivalences assume that __debug__ and AssertionError refer to the built-in variables with those names. In the current implementation, the built-in variable __debug__ is True under normal circumstances, False when optimization is requested (command line option -O). The current code generator emits no code for an assert statement when optimization is requested at compile time. Note that it is unnecessary to include the source code for the expression that failed in the error message; it will be displayed as part of the stack trace.

Assignments to __debug__ are illegal. The value for the built-in variable is determined when the interpreter starts.

pass

The pass statement does nothing. It can be used when a statement is required syntactically but the program requires no action.

>>> def initlog(*args):
...     pass   # Remember to implement this!
...

del

del <target_list>

Deletion is recursively defined very similar to the way assignment is defined. Rather than spelling it out in full details, here are some hints.

Deletion of a target list recursively deletes each target, from left to right.

Deletion of a name removes the binding of that name from the local or global namespace, depending on whether the name occurs in a global statement in the same code block. If the name is unbound, a NameError exception will be raised.

Deletion of attribute references, subscriptions and slicings is passed to the primary object involved; deletion of a slicing is in general equivalent to assignment of an empty slice of the right type (but even this is determined by the sliced object).

Changed in version 3.2: Previously it was illegal to delete a name from the local namespace if it occurs as a free variable in a nested block.

return

return <expression_list>

The return statement may only occur syntactically nested in a function definition, not within a nested class definition.

If an expression_list is present, it is evaluated, else None is substituted.

The return statement leaves the current function call with the expression_list or None as return value.

When return passes control out of a try statement with a finally clause, that finally clause is executed before really leaving the function.

In a generator function, the return statement indicates that the generator is done and will cause StopIteration to be raised. The returned value (if any) is used as an argument to construct StopIteration and becomes the StopIteration.value attribute.

In an asynchronous generator function, an empty return statement indicates that the asynchronous generator is done and will cause **StopAsyncIteration×× to be raised. A non-empty return statement is a syntax error in an asynchronous generator function.

yield

A yield statement is semantically equivalent to a yield expression. The yield statement can be used to omit the parentheses that would otherwise be required in the equivalent yield expression statement. For example, the yield statements

yield <expr>
yield from <expr>

are equivalent to the yield expression statements

(yield <expr>)
(yield from <expr>)

The yield expressions and statements are only used when defining a generator function, and are only used in the body of the generator function. Using yield in a function definition is sufficient to cause that definition to create a generator function instead of a normal function. Refer to yield expressions.

raise

raise [<expression] [from <expression>]]

If no expressions are present, the raise statement re-raises the last exception that was active in the current scope. If no exception is active in the current scope, a RuntimeError exception is raised indicating that this is an error.

Otherwise, the raise statement evaluates the first expression as the exception object. It must be either a subclass or an instance of BaseException. If it is a class, the exception instance will be obtained when needed by instantiating the class with no arguments.

The type of the exception is the exception instance’s class, the value is the instance itself.

Changed in version 3.3: None is now permitted as Y in raise X from Y.

break

The break statement may only occur syntactically nested in a for or while loop, but not nested in a function or class definition within that loop.

It terminates the nearest enclosing loop, skipping the optional else clause if the loop has one.

  • Loop statements (for and while) may have an else clause; it is executed when the loop terminates through exhaustion of the list (with for) or when the condition becomes false (with while), but not when the loop is terminated by a break statement.

If a for loop is terminated by break, the loop control target keeps its current value.

When break passes control out of a try statement with a finally clause, that finally clause is executed before really leaving the loop.

>>> for n in range(2, 10):
...     for x in range(2, n):
...         if n % x == 0:
...             print(n, 'equals', x, '*', n//x)
...             break
...     else:
...         # loop fell through without finding a factor
...         print(n, 'is a prime number')
...
2 is a prime number
3 is a prime number
4 equals 2 * 2
5 is a prime number
6 equals 2 * 3
7 is a prime number
8 equals 2 * 4
9 equals 3 * 3

continue

The continue statment may only occur syntactically nested in a for or while loop, but not nested in a function or class definition or finally clause within that loop. It continues with the next cycle of the nearest enclosing loop.

When continue statement passes control out of a try statement with a finally clause, that finally clause is executed before really starting the next loop cycle.

>>> for num in range(2, 10):
...     if num % 2 == 0:
...         print("Found an even number", num)
...         continue
...     print("Found a number", num)
Found an even number 2
Found a number 3
Found an even number 4
Found a number 5
Found an even number 6
Found a number 7
Found an even number 8
Found a number 9

import

According to import, Python code in one module gains access to the code in another module by the process of importing it. The import statement is the most common way of invoking the import machinery, but it is not the only way. Functions such as importlib.import_module() and built-in __import__() can also be used to invoke the import machinery.

The import statement combines two operations; it searches for the named module, then it binds the results of that search to a name in the local scope. The search operation of the import statement is defined as a call to the __import__() function, with the appropriate arguments. The return value of import() is used to perform the name binding operation of the import statement.

import <module1> [as <name1>] [, <moduleN> [as <nameN>]]

from <relative_module> import  <identifier1> [as <name1>] [, <identifierN> [as <nameN>]]
from <relative_module> import (<identifier1> [as <name1>] [, <identifierN> [as <nameN>]])

from <module> import *

global

global <identifier1> [, <identifier2> [, <identifierN>]]

The global statement is a declaration which holds for the entire current code block. It means that the listed identifiers are to be interpreted as globals. It would be impossible to assign to a global variable without global, although free variables may refer to globals without being declared global.

Names listed in a global statement must not be used in the same code block textually preceding that global statement.

Names listed in a global statement must not be defined as formal parameters or in a for loop control target, class definition, function definition, import statement, or variable annotation.

nonlocal

nonlocal <identifier1> [, <identifier2> [, <identifierN>]]

The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals. This is important because the default behavior for binding is to search the local namespace first. The statement allows encapsulated code to rebind variables outside of the local scope besides the global (module) scope.

Names listed in a nonlocal statement, unlike those listed in a global statement, must refer to pre-existing bindings in an enclosing scope (the scope in which a new binding should be created cannot be determined unambiguously).

Names listed in a nonlocal statement must not collide with pre-existing bindings in the local scope.

if..elif..else

The if statement is used for conditional execution. There can be zero or more elif parts, and the else part is optional:

if <expression1>:
    <suite1>
elif <expression2>:
    <suite2>
elif <expressionM>:
    <suiteM>
else:
    <suiteN>

It selects exactly one of the suites by evaluating the expressions one by one until one is found to be true; then that suite is executed (and no other part of the if statement is executed or evaluated). If all expressions are false, the suite of the else clause, if present, is executed. For instance:

>>> x = int(input("Please enter an integer: "))
Please enter an integer: 42
>>> if x < 0:
...     x = 0
...     print('Negative changed to zero')
... elif x == 0:
...     print('Zero')
... elif x == 1:
...     print('Single')
... else:
...     print('More')
...
More

for

The for statement is used to iterate over the elements of a sequence (such as a string, tuple or list) or other iterable object:

for <target_list> in <expression_list>:
    <suite>
else:
    <suite>

The expression_list is evaluated once; it should yield an iterable object. An iterator is created for the result of the expression_list. The suite is then executed once for each item provided by the iterator, in the order returned by the iterator. Each item in turn is assigned to the target list using the standard rules for assignments, and then the suite is executed. When the items are exhausted (which is immediately when the sequence is empty or an iterator raises a StopIteration exception), the suite in the else clause, if present, is executed, and the loop terminates.

A break statement executed in the first suite terminates the loop without executing the else clause’s suite. A continue statement executed in the first suite skips the rest of the suite and continues with the next item, or with the else clause if there is no next item.

Names in the target_list are not deleted when the loop is finished, but if the sequence is empty, they will not have been assigned to at all by the loop. Hint: the built-in function range() returns an iterator of integers suitable to emulate the effect of Pascal’s for i := a to b do; e.g., list(range(3)) returns the list [0, 1, 2].

>>> for i in range(5):
...     print(i)
...
0
1
2
3
4

>>> words = ['cat', 'window', 'defenestrate']
>>> for w in words:
...     print(w, len(w))
...
('cat', 3)
('window', 6)
('defenestrate', 12)

If you need to modify the sequence you are iterating over while inside the loop (for example to duplicate selected items), it is recommended that you first make a copy. Iterating over a sequence does not implicitly make a copy. The slice notation makes this especially convenient:

>>> for w in words[:]:  # Loop over a slice copy of the entire list.
...     if len(w) > 6:
...         words.insert(0, w)
...
>>> words
['defenestrate', 'cat', 'window', 'defenestrate']

while

In Python, like in C, any non-zero integer value is true; zero is false. The condition may also be a string or list value, in fact any sequence; anything with a non-zero length is true, empty sequences are false.

The while statement is used for repeated execution as long as an expression is true:

while <expression>:
    <suite>
else:
    <suite>

This repeatedly tests the expression and, if it is true, executes the first suite; if the expression is false (which may be the first time it is tested) the suite of the else clause, if present, is executed and the loop terminates.

A break statement executed in the first suite terminates the loop without executing the else clause’s suite. A continue statement executed in the first suite skips the rest of the suite and goes back to testing the expression.

>>> # Fibonacci series: the sum of two elements defines the next
... a, b = 0, 1
>>> while b < 10:
...     print(b)
...     a, b = b, a+b
... else:
...     print(100)
...
1
1
2
3
5
8
100

Functions

The keyword def introduces a function definition. It must be followed by the function name and the parenthesized list of formal parameters. The statements that form the body of the function start at the next line, and must be indented.

>>> def fib(n):    # write Fibonacci series up to n
...     """Print a Fibonacci series up to n."""
...     a, b = 0, 1
...     while a < n:
...         print(a, end=' ')
...         a, b = b, a+b
...     print()
...
>>> # Now call the function we just defined:
... fib(2000)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597

A function definition introduces the function name in the current symbol table. The value of the function name has a type that is recognized by the interpreter as a user-defined function. This value can be assigned to another name which can then also be used as a function. This serves as a general renaming mechanism:

>>> fib
<function fib at 10042ed0>
>>> f = fib
>>> f(100)
0 1 1 2 3 5 8 13 21 34 55 89

The return statement returns with a value from a function. return without an expression argument returns None. Falling off the end of a function also returns None.

>>> def fib2(n):  # return Fibonacci series up to n
...     """Return a list containing the Fibonacci series up to n."""
...     result = []
...     a, b = 0, 1
...     while a < n:
...         result.append(a)    # see below
...         a, b = b, a+b
...     return result
...
>>> f100 = fib2(100)    # call it
>>> f100                # write the result
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]

docstring

The first statement of the function body can optionally be a string literal; this string literal is the function’s documentation string, or docstring.

The first line should always be a short, concise summary of the object’s purpose. For brevity, it should not explicitly state the object’s name or type, since these are available by other means (except if the name happens to be a verb describing a function’s operation). This line should begin with a capital letter and end with a period.

If there are more lines in the documentation string, the second line should be blank, visually separating the summary from the rest of the description. The following lines should be one or more paragraphs describing the object’s calling conventions, its side effects, etc.

The Python parser does not strip indentation from multi-line string literals in Python, so tools that process documentation have to strip indentation if desired. This is done using the following convention. The first non-blank line after the first line of the string determines the amount of indentation for the entire documentation string. (We can’t use the first line since it is generally adjacent to the string’s opening quotes so its indentation is not apparent in the string literal.) Whitespace equivalent to this indentation is then stripped from the start of all lines of the string. Lines that are indented less should not occur, but if they occur all their leading whitespace should be stripped. Equivalence of whitespace should be tested after expansion of tabs (to 8 spaces, normally).

>>> def my_function():
...     """Do nothing, but document it.
...
...     No, really, it doesn't do anything.
...     """
...     pass
...
>>> print(my_function.__doc__)
Do nothing, but document it.

    No, really, it doesn't do anything.

Default Argument Values

The default values are evaluated at the point of function definition in the defining scope:

>>> i = 5
>>> def f(arg = i):
...     print(arg)
...
>>> i = 6
>>> f()
5

The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes. The following function accumulates the arguments passed to it on subsequent calls:

>>> def f(a, L = []):
...     L.append(a)
...     return L
...
>>> print(f(1))
[1]
>>> print(f(2))
[1, 2]
>>> print(f(3))
[1, 2, 3]

If you don’t want the default to be shared between subsequent calls, you can write the function like this instead:

>>> def f(a, L=None):
...     if L is None:
...         L = []
...     L.append(a)
...     return L
...
>>> print(f(1))
[1]
>>> print(f(2))
[2]
>>> print(f(3))
[3]

Keyword Arguments

Functions can also be called using keyword arguments of the form kwarg=value. The function:

>>> def parrot(voltage, state='a stiff', action='voom', type='Norwegian Blue'):
...     print("-- This parrot wouldn't", action, end=' ')
...     print("if you put", voltage, "volts through it.")
...     print("-- Lovely plumage, the", type)
...     print("-- It's", state, "!")
...

can be called in any of the following ways:

parrot(1000)                                          # 1 positional argument
parrot(voltage=1000)                                  # 1 keyword argument
parrot(voltage=1000000, action='VOOOOOM')             # 2 keyword arguments
parrot(action='VOOOOOM', voltage=1000000)             # 2 keyword arguments
parrot('a million', 'bereft of life', 'jump')         # 3 positional arguments
parrot('a thousand', state='pushing up the daisies')  # 1 positional, 1 keyword

Note:

  • In a function call, keyword arguments must follow positional arguments.
  • All the keyword arguments passed must match one of the arguments accepted by the function and their order is not important.
  • No argument may receive a value more than once.

When a final formal parameter of the form **name is present, it receives a dictionary containing all keyword arguments except for those corresponding to a formal parameter. This may be combined with a formal parameter of the form *name which receives a tuple containing the positional arguments beyond the formal parameter list. *name must occur before **name.

Arbitrary Argument Lists

A function can be called with an arbitrary number of arguments. These arguments will be wrapped up in a tuple. Before the variable number of arguments, zero or more normal arguments may occur.

def write_multiple_items(file, separator, *args):
    file.write(separator.join(args))

Normally, these variadic arguments will be last in the list of formal parameters, because they scoop up all remaining input arguments that are passed to the function. Any formal parameters which occur after the *args parameter are keyword-only arguments, meaning that they can only be used as keywords rather than positional arguments.

>>> def concat(*args, sep="/"):
...     return sep.join(args)
...
>>> concat("earth", "mars", "venus")            # sep = "/" in this case
'earth/mars/venus'
>>> concat("earth", "mars", "venus", sep=".")   # sep = "." in this case
'earth.mars.venus'

Unpacking Argument Lists

The reverse situation occurs when the arguments are already in a list or tuple but need to be unpacked for a function call requiring separate positional arguments. If they are not available separately, write the function call with the *-operator to unpack the arguments out of a list or tuple:

>>> list(range(3, 6))        # normal call with separate arguments
[3, 4, 5]
>>> args = [3, 6]
>>> list(range(*args))       # call with arguments unpacked from a list
[3, 4, 5]

In the same fashion, dictionaries can deliver keyword arguments with the **-operator:

>>> def parrot(voltage, state='a stiff', action='voom'):
...     print("-- This parrot wouldn't", action, end=' ')
...     print("if you put", voltage, "volts through it.", end=' ')
...     print("E's", state, "!")
...
>>> d = {"voltage": "four million", "state": "bleedin' demised", "action": "VOOM"}
>>> parrot(**d)
-- This parrot wouldn't VOOM if you put four million volts through it. E's bleedin' demised !

Lambda Expressions

Small anonymous functions can be created with the lambda keyword. This function returns the sum of its two arguments: lambda a, b: a+b. Lambda functions can be used wherever function objects are required. They are syntactically restricted to a single expression. Semantically, they are just syntactic sugar for a normal function definition. Like nested function definitions, lambda functions can reference variables from the containing scope:

>>> def make_incrementor(n):
...     return lambda x: x + n
...
>>> f = make_incrementor(42)
>>> f(0)
42
>>> f(1)
43

The above example uses a lambda expression to return a function. Another use is to pass a small function as an argument:

>>> pairs = [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')]
>>> pairs.sort(key=lambda pair: pair[1])
>>> pairs
[(4, 'four'), (1, 'one'), (3, 'three'), (2, 'two')]

Function Annotations

Function annotations are completely optional metadata information about the types used by user-defined functions, see PEP 484: Type Hints for more information.

Annotations are stored in the __annotations__ attribute of the function as a dictionary and have no effect on any other part of the function.

Parameter annotations are defined by a colon after the parameter name, followed by an expression evaluating to the value of the annotation.

Return annotations are defined by a literal ->, followed by an expression, between the parameter list and the colon denoting the end of the def statement. The following example has a positional argument, a keyword argument, and the return value annotated:

>>> def f(ham: str, eggs: str = 'eggs') -> str:
...     print("Annotations:", f.__annotations__)
...     print("Arguments:", ham, eggs)
...     return ham + ' and ' + eggs
...
>>> f('spam')
Annotations: {'ham': <class 'str'>, 'return': <class 'str'>, 'eggs': <class 'str'>}
Arguments: spam eggs
'spam and eggs'

Modules

If you quit from the Python interpreter and enter it again, the definitions you have made (functions and variables) are lost. Therefore, if you want to write a somewhat longer program, you are better off using a text editor to prepare the input for the interpreter and running it with that file as input instead.

To support this, Python has a way to put definitions in a file and use them in a script or in an interactive instance of the interpreter. Such a file is called a module; definitions from a module can be imported into other modules or into the main module (the collection of variables that you have access to in a script executed at the top level and in calculator mode).

A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. Within a module, the module’s name (as a string) is available as the value of the global variable __name__. For instance, use your favorite text editor to create a file called fibo.py in the current directory with the following contents:

# Fibonacci numbers module

def fib(n):    # write Fibonacci series up to n
    a, b = 0, 1
    while b < n:
        print(b, end=' ')
        a, b = b, a+b
    print()

def fib2(n):   # return Fibonacci series up to n
    result = []
    a, b = 0, 1
    while b < n:
        result.append(b)
        a, b = b, a+b
    return result

Now enter the Python interpreter and import this module and access the functions with the following command:

>>> import fibo

>>> fibo.fib(1000)
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
>>> fibo.fib2(100)
[1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]

>>> fibo.__name__
'fibo'

>>> fib = fibo.fib
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377

A module can contain executable statements as well as function definitions. These statements are intended to initialize the module. They are executed only the first time the module name is encountered in an import statement.

Each module has its own private symbol table, which is used as the global symbol table by all functions defined in the module. Thus, the author of a module can use global variables in the module without worrying about accidental clashes with a user’s global variables. On the other hand, if you know what you are doing you can touch a module’s global variables with the same notation used to refer to its functions, modname.itemname.

Modules can import other modules. It is customary but not required to place all import statements at the beginning of a module (or script, for that matter). The imported module names are placed in the importing module’s global symbol table.

There is a variant of the import statement that imports names from a module directly into the importing module’s symbol table. This does not introduce the module name from which the imports are taken in the local symbol table (so in the example, fibo is not defined):

>>> from fibo import fib, fib2
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377

There is even a variant to import all names that a module defines. This imports all names except those beginning with an underscore _:

>>> from fibo import *
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377

Executing modules as scripts

When you run a Python module with following command, the code in the module will be executed, just as if you imported it, but with the __name__ set to __main__.

python fibo.py <arguments>

or, by adding this code at the end of your module:

if __name__ == "__main__":
    import sys
    fib(int(sys.argv[1]))

you can make the file usable as a script as well as an importable module, because the code that parses the command line only runs if the module is executed as the main file:

$ python fibo.py 50
1 1 2 3 5 8 13 21 34

Module Search Path

When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path, which is initialized from these locations:

  • The directory containing the input script (or the current directory when no file is specified).
  • PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
  • The installation-dependent default.

Note: On file systems which support symlinks, the directory containing the input script is calculated after the symlink is followed. In other words the directory containing the symlink is not added to the module search path.

After initialization, Python programs can modify sys.path. The directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path. This means that scripts in that directory will be loaded instead of modules of the same name in the library directory. This is an error unless the replacement is intended.

Compiled Python Files

To speed up loading modules, Python caches the compiled version of each module in the __pycache__ directory under the name module.version.pyc, where the version encodes the format of the compiled file; it generally contains the Python version number, such as __pycache__/spam.cpython-33.pyc. This naming convention allows compiled modules from different releases and different versions of Python to coexist.

Python checks the modification date of the source against the compiled version to see if it’s out of date and needs to be recompiled.

Python does not check the cache in two circumstances:

  1. It always recompiles and does not store the result for the module that’s loaded directly from the command line.
  2. It does not check the cache if there is no source module. To support a non-source (compiled only) distribution, the compiled module must be in the source directory, and there must not be a source module.

Packages

Packages are a way of structuring Python’s module namespace by using dotted module names. For example, the module name A.B designates a submodule named B in a package named A.

sound/                          # Top-level package
      __init__.py               # Initialize the sound package
      formats/                  # Subpackage for file format conversions
              __init__.py
              wavread.py
              wavwrite.py
              aiffread.py
              aiffwrite.py
              auread.py
              auwrite.py
              ...
      effects/                  # Subpackage for sound effects
              __init__.py
              echo.py
              surround.py
              reverse.py
              ...
      filters/                  # Subpackage for filters
              __init__.py
              equalizer.py
              vocoder.py
              karaoke.py
              ...

When importing the package, Python searches through the directories on sys.path looking for the package subdirectory.

The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable.

Users of the package can import individual modules from the package:

import sound.effects.echo

This loads the submodule sound.effects.echo. It must be referenced with its full name.

sound.effects.echo.echofilter(input, output, delay=0.7, atten=4)

An alternative way of importing the submodule is:

from sound.effects import echo

This also loads the submodule echo, and makes it available without its package prefix, so it can be used as follows:

echo.echofilter(input, output, delay=0.7, atten=4)

Yet another variation is to import the desired function or variable directly:

from sound.effects.echo import echofilter

Again, this loads the submodule echo, but this makes its function echofilter() directly available:

echofilter(input, output, delay=0.7, atten=4)

__init__.py

The import statement uses the following convention:

If a package’s __init__.py code defines a list named __all__, it is taken to be the list of module names that should be imported when from package import * is encountered. The following code would mean that from sound.effects import * would import the three named submodules of the sound package:

__all__ = ["echo", "surround", "reverse"]

If __all__ is not defined, the statement from sound.effects import * does not import all submodules from the package sound.effects into the current namespace; it only ensures that the package sound.effects has been imported (possibly running any initialization code in __init__.py) and then imports whatever names are defined in the package. This includes any names defined (and submodules explicitly loaded) by __init__.py. It also includes any submodules of the package that were explicitly loaded by previous import statements.

dir() Function

The built-in function dir() is used to find out which names a module defines (including all types of names: variables, modules, functions, etc). It returns a sorted list of strings. Without arguments, dir() lists the names you have defined currently.

>>> dir()
['__builtins__', '__doc__', '__name__', '__package__']
>>>
>>> import sys
>>> dir(sys)
['__displayhook__', '__doc__', '__egginsert', '__excepthook__', '__name__', '__package__', '__plen', '__stderr__', '__stdin__', '__stdout__', '_clear_type_cache', '_current_frames', '_getframe', '_mercurial', 'api_version', 'argv', 'builtin_module_names', 'byteorder', 'call_tracing', 'callstats', 'copyright', 'displayhook', 'dont_write_bytecode', 'exc_clear', 'exc_info', 'exc_type', 'excepthook', 'exec_prefix', 'executable', 'exit', 'exitfunc', 'flags', 'float_info', 'float_repr_style', 'getcheckinterval', 'getdefaultencoding', 'getdlopenflags', 'getfilesystemencoding', 'getprofile', 'getrecursionlimit', 'getrefcount', 'getsizeof', 'gettrace', 'hexversion', 'long_info', 'maxint', 'maxsize', 'maxunicode', 'meta_path', 'modules', 'path', 'path_hooks', 'path_importer_cache', 'platform', 'prefix', 'ps1', 'ps2', 'py3kwarning', 'setcheckinterval', 'setdlopenflags', 'setprofile', 'setrecursionlimit', 'settrace', 'stderr', 'stdin', 'stdout', 'subversion', 'version', 'version_info', 'warnoptions']
>>>
>>> print(sys.version)
3.5.0 (default, Dec 21 2015, 22:50:16)
[GCC 4.8.2]

The dir() does not list the names of built-in functions and variables. If you want a list of those, they are defined in the standard module builtins:

>>> import builtins
>>> dir(builtins)
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']

Input and Output

Errors and Exceptions

There are (at least) two distinguishable kinds of errors: syntax errors and exceptions.

Syntax Errors

Syntax errors, also known as parsing errors, are perhaps the most common kind of complaint you get while you are still learning Python:

>>> while True print('Hello world')
  File "<stdin>", line 1
    while True print('Hello world')
                   ^
SyntaxError: invalid syntax

Exceptions

Errors detected during execution are called exceptions and are not unconditionally fatal: you will soon learn how to handle them in Python programs. Most exceptions are not handled by programs, however, and result in error messages as shown here:

>>> 1/0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero

Exceptions come in different types, and the type is printed as part of the message: ZeroDivisionError. Standard exception names are built-in identifiers (not reserved keywords). Built-in Exceptions lists the built-in exceptions and their meanings.

Handling Exceptions

It is possible to write programs that handle selected exceptions:

>>> while True:
...     try:
...         x = int(input("Please enter a number: "))
...         break
...     except ValueError:
...         print("Oops!  That was no valid number.  Try again...")
...
Please enter a number: abc
Oops!  That was no valid number.  Try again...
Please enter a number: 100

The try statement works as follows:

  • First, the try clause (the statement(s) between the try and except keywords) is executed.
  • If no exception occurs, the except clause is skipped and execution of the try statement is finished.
  • If an exception occurs during execution of the try clause, the rest of the clause is skipped. Then if its type matches the exception named after the except keyword, the except clause is executed, and then execution continues after the try statement.
  • If an exception occurs which does not match the exception named in the except clause, it is passed on to outer try statements; if no handler is found, it is an unhandled exception and execution stops with a message as shown above.

A try statement may have more than one except clause, to specify handlers for different exceptions. At most one handler will be executed. Handlers only handle exceptions that occur in the corresponding try clause, not in other handlers of the same try statement. An except clause may name multiple exceptions as a parenthesized tuple:

... except (RuntimeError, TypeError, NameError):
...     pass

The last except clause may omit the exception name(s), to serve as a wildcard. Use this with extreme caution, since it is easy to mask a real programming error in this way! It can also be used to print an error message and then re-raise the exception (allowing a caller to handle the exception as well):

import sys

try:
    f = open('myfile.txt')
    s = f.readline()
    i = int(s.strip())
except OSError as err:    # the except clause may specify a variable after the exception name
    print("OS error: {0}".format(err))
except ValueError:
    print("Could not convert data to an integer.")
except:
    print("Unexpected error:", sys.exc_info()[0])
    raise

The try...except statement has an optional else clause, which, when present, must follow all except clauses. It is useful for code that must be executed if the try clause does not raise an exception:

for arg in sys.argv[1:]:
    try:
        f = open(arg, 'r')
    except OSError:
        print('cannot open', arg)
    else:
        print(arg, 'has', len(f.readlines()), 'lines')
        f.close()

Exception handlers don’t just handle exceptions if they occur immediately in the try clause, but also if they occur inside functions that are called (even indirectly) in the try clause:

def this_fails():
    x = 1/0

try:
    this_fails()
except ZeroDivisionError as err:
    print('Handling run-time error:', err)

The try statement has another optional clause which is intended to define clean-up actions that must be executed under all circumstances:

>>> try:
...     raise KeyboardInterrupt
... finally:
...     print('Goodbye, world!')
...
Goodbye, world!
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
KeyboardInterrupt

A finally clause is always executed before leaving the try statement, whether an exception has occurred or not. When an exception has occurred in the try clause and has not been handled by an except clause (or it has occurred in an except or else clause), it is re-raised after the finally clause has been executed. The finally clause is also executed on the way out when any other clause of the try statement is left via a break, continue or return statement.

>>> def divide(x, y):
...     try:
...         result = x / y
...     except ZeroDivisionError:
...         print("division by zero!")
...     else:
...         print("result is", result)
...     finally:
...         print("executing finally clause")
...
>>> divide(2, 1)
result is 2.0
executing finally clause
>>>
>>> divide(2, 0)
division by zero!
executing finally clause
>>>
>>> divide("2", "1")
executing finally clause
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in divide
TypeError: unsupported operand type(s) for /: 'str' and 'str'

In real world applications, the finally clause is useful for releasing external resources (such as files or network connections), regardless of whether the use of the resource was successful.

Raising Exceptions

The raise statement allows the programmer to force a specified exception to occur:

try:
    raise NameError('HiThere')
except NameError:
    print('An exception flew by!')
    raise

User-defined Exceptions

Programs may name their own exceptions by creating a new exception class. Exceptions should typically be derived from the Exception class, either directly or indirectly.

>>> help(Exception)

Help on class Exception in module builtins:

class Exception(BaseException)
 |  Common base class for all non-exit exceptions.
 |  
 |  Method resolution order:
 |      Exception
 |      BaseException
 |      object
 |  
 |  Methods defined here:
 |  
 |  __init__(self, /, *args, **kwargs)
 |      Initialize self.  See help(type(self)) for accurate signature.
 |  
 |  __new__(*args, **kwargs) from builtins.type
 |      Create and return a new object.  See help(type) for accurate signature.
 |  
 |  ----------------------------------------------------------------------
 |  Methods inherited from BaseException:
 |  
 |  __delattr__(self, name, /)
 |      Implement delattr(self, name).
 |  
 |  __getattribute__(self, name, /)
 |      Return getattr(self, name).
 |  
 |  __reduce__(...)
 |      helper for pickle
 |  
 |  __repr__(self, /)
 |      Return repr(self).
 |  
 |  __setattr__(self, name, value, /)
 |      Implement setattr(self, name, value).
 |  
 |  __setstate__(...)
 |  
 |  __str__(self, /)
 |      Return str(self).
 |  
 |  with_traceback(...)
 |      Exception.with_traceback(tb) --
 |      set self.__traceback__ to tb and return self.
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors inherited from BaseException:
 |  
 |  __cause__
 |      exception cause
 |  
 |  __context__
 |      exception context
 |  
 |  __dict__
 |  
 |  __suppress_context__
 |  
 |  __traceback__
 |  
 |  args

Exception classes can be defined which do anything any other class can do, but are usually kept simple, often only offering a number of attributes that allow information about the error to be extracted by handlers for the exception. When creating a module that can raise several distinct errors, a common practice is to create a base class for exceptions defined by that module, and subclass that to create specific exception classes for different error conditions.

Most exceptions are defined with names that end in Error, similar to the naming of the standard exceptions.

Many standard modules define their own exceptions to report errors that may occur in functions they define.

Classes

Compared with other programming languages, Python’s class mechanism adds classes with a minimum of new syntax and semantics. It is a mixture of the class mechanisms found in C++ and Modula-3. Python classes provide all the standard features of Object Oriented Programming: the class inheritance mechanism allows multiple base classes, a derived class can override any methods of its base class or classes, and a method can call the method of a base class with the same name. Objects can contain arbitrary amounts and kinds of data. As is true for modules, classes partake of the dynamic nature of Python: they are created at runtime, and can be modified further after creation.

Scopes and Namespaces

A namespace is a mapping from names to objects. Namespaces are created at different moments and have different lifetimes:

  • The namespace containing the built-in names is created when the Python interpreter starts up, and is never deleted.
  • The global namespace for a module is created when the module definition is read in; normally, module namespaces also last until the interpreter quits.
  • The statements executed by the top-level invocation of the interpreter, either read from a script file or interactively, are considered part of a module called __main__, so they have their own global namespace.
  • The local namespace for a function is created when the function is called, and deleted when the function returns or raises an exception that is not handled within the function.

The built-in names actually also live in a module named builtins:

>>> import builtins
>>> dir(builtins)
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']

A scope is a textual region of a Python program where a namespace is directly accessible. Although scopes are determined statically, they are used dynamically. At any time during execution, there are at least three nested scopes whose namespaces are directly accessible:

  • the innermost scope, which is searched first, contains the local names
  • the scopes of any enclosing functions, which are searched starting with the nearest enclosing scope, contains non-local, but also non-global names
  • the next-to-last scope contains the current module’s global names
  • the outermost scope (searched last) is the namespace containing built-in names

The global statement can be used to indicate that particular variables live in the global scope and should be rebound there; the nonlocal statement indicates that particular variables live in an enclosing scope and should be rebound there.

Class Definition Syntax

class ClassName:
    <statement-1>
    ...
    <statement-N>

Class definitions, like function definitions (def statements) must be executed before they have any effect. You could conceivably place a class definition in a branch of an if statement, or inside a function.

In practice, the statements inside a class definition will usually be function definitions, but other statements are allowed, and sometimes useful.

When a class definition is entered, a new namespace is created, and used as the local scope.

When a class definition is left normally (via the end), a class object is created.

>>> class MyClass:
...     '''A simple example class'''
...     def __init__(self, var):
...         self.i = var
...     def f(self):
...         return "Hello world"
...     i = 123
...
>>>
>>> help(MyClass)
Help on class MyClass in module __main__:

class MyClass(builtins.object)
 |  A simple example class
 |  
 |  Methods defined here:
 |  
 |  __init__(self, var)
 |      Initialize self.  See help(type(self)) for accurate signature.
 |  
 |  f(self)
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |  
 |  i = 123

The special thing about methods is that the instance object is passed as the first argument of the function, that’s self. In general, calling a method with a list of n arguments is equivalent to calling the corresponding function with an argument list that is created by inserting the method’s instance object before the first argument.

Class Objects

Class objects support two kinds of operations: attribute references and instantiation.

  • Attribute references use the standard syntax used for all attribute references in Python: obj.name. Valid attribute names are all the names that were in the class’s namespace when the class object was created.
  • Class instantiation uses function notation. Just pretend that the class object is a parameterless function that returns a new instance of the class: x = MyClass(). When a class defines an __init__() method, class instantiation automatically invokes __init__() for the newly-created class instance.

Class and Instance Variables

Instance variables are for data unique to each instance and class variables are for attributes and methods shared by all instances of the class:

>>> class Dog:
...     kind = 'canine'         # class variable shared by all instances
...     def __init__(self, name):
...         self.name = name    # instance variable unique to each instance
...
>>> d = Dog('Fido')
>>> e = Dog('Buddy')
>>> d.kind                      # shared by all dogs
'canine'
>>> e.kind                      # shared by all dogs
'canine'
>>> d.name                      # unique to d
'Fido'
>>> e.name                      # unique to e
'Buddy'

Random Remarks

Data attributes override method attributes with the same name; to avoid accidental name conflicts, which may cause hard-to-find bugs in large programs, it is wise to use some kind of convention that minimizes the chance of conflicts. Possible conventions include capitalizing method names, prefixing data attribute names with a small unique string (perhaps just an underscore), or using verbs for methods and nouns for data attributes.

Inheritance

The syntax for a derived class definition looks like this:

class DerivedClassName(BaseClassName):
    <statement-1>
    ...
    <statement-N>

Derived classes may override methods of their base classes. For C++ programmers: all methods in Python are effectively virtual.

An overriding method in a derived class may in fact want to extend rather than simply replace the base class method of the same name. There is a simple way to call the base class method directly: just call BaseClassName.methodName(self, arguments).

Python has two built-in functions that work with inheritance:

  • Use isinstance() to check an instance’s type: isinstance(obj, int) will be True only if obj.__class__ is int or some class derived from int.
  • Use issubclass() to check class inheritance: issubclass(bool, int) is True since bool is a subclass of int. However, issubclass(float, int) is False since float is not a subclass of int.

Python supports a form of multiple inheritance as well. A class definition with multiple base classes looks like this:

class DerivedClassName(Base1, Base2, Base3):
    <statement-1>
    ...
    <statement-N>

Private Variables

Private instance variables that cannot be accessed except from inside an object don’t exist in Python. However, there is a convention that is followed by most Python code: a name prefixed with an underscore (e.g. _spam) should be treated as a non-public part of the API (whether it is a function, a method or a data member). It should be considered an implementation detail and subject to change without notice.

Since there is a valid use-case for class-private members (namely to avoid name clashes of names with names defined by subclasses), there is limited support for such a mechanism, called name mangling. Any identifier of the form __spam (at least two leading underscores, at most one trailing underscore) is textually replaced with _classname__spam, where classname is the current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, as long as it occurs within the definition of a class.

Name mangling is helpful for letting subclasses override methods without breaking intraclass method calls.

class Mapping:
    def __init__(self, iterable):
        self.items_list = []
        self.__update(iterable)

    def update(self, iterable):
        for item in iterable:
            self.items_list.append(item)

    __update = update   # private copy of original update() method

class MappingSubclass(Mapping):
    def update(self, keys, values):
        # provides new signature for update()
        # but does not break __init__()
        for item in zip(keys, values):
            self.items_list.append(item)

Iterators

The most container objects can be looped over using a for statement:

for element in [1, 2, 3]:
    print(element)
for element in (1, 2, 3):
    print(element)
for key in {'one':1, 'two':2}:
    print(key)
for char in "123":
    print(char)
for line in open("myfile.txt"):
    print(line, end='')

The use of iterators pervades and unifies Python. Behind the scenes, the for statement calls iter() on the container object. The function returns an iterator object that defines the method __next__() which accesses elements in the container one at a time. When there are no more elements, __next__() raises a StopIteration exception which tells the for loop to terminate. You can call the __next__() method using the next() built-in function; this example shows how it all works:

>>> s = 'abc'
>>> it = iter(s)
>>> it
<iterator object at 0x00A1DB50>
>>> next(it)
'a'
>>> next(it)
'b'
>>> next(it)
'c'
>>> next(it)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
    next(it)
StopIteration

Having seen the mechanics behind the iterator protocol, it is easy to add iterator behavior to your classes. Define an __iter__() method which returns an object with a __next__() method. If the class defines __next__(), then __iter__() can just return self:

class Reverse:
    """Iterator for looping over a sequence backwards."""
    def __init__(self, data):
        self.data = data
        self.index = len(data)

    def __iter__(self):
        return self

    def __next__(self):
        if self.index == 0:
            raise StopIteration
        self.index = self.index - 1
        return self.data[self.index]
>>> rev = Reverse('spam')
>>> iter(rev)
<__main__.Reverse object at 0x00A1DB50>
>>> for char in rev:
...     print(char)
...
m
a
p
s

Help information of method iter():

Help on built-in function iter in module builtins:

iter(...)
    iter(iterable) -> iterator
    iter(callable, sentinel) -> iterator

    Get an iterator from an object.  In the first form, the argument must
    supply its own iterator, or be a sequence.
    In the second form, the callable is called until it returns the sentinel.

Help information of method next():

Help on built-in function next in module builtins:

next(...)
    next(iterator[, default])

    Return the next item from the iterator. If default is given and the iterator
    is exhausted, it is returned instead of raising StopIteration.

Generators

Generators are a simple and powerful tool for creating iterators. They are written like regular functions but use the yield statement whenever they want to return data. Each time next() is called on it, the generator resumes where it left off (it remembers all the data values and which statement was last executed). An example shows that generators can be trivially easy to create:

def reverse(data):
    for index in range(len(data)-1, -1, -1):
        yield data[index]
>>> for char in reverse('golf'):
...     print(char)
...
f
l
o
g

Special Methods

Python_Special_Methods

Virtual Environments

Refer to Virtual Environments and Packages

Python Libraries

Python Standard Library

The Python Standard Library:

The Python Module Index:

In addition to the Python standard library, there is a growing collection of several thousand components (from individual programs and modules to packages and entire application development frameworks), available from the Python Package Index (PyPI).

The Python standard library contains:

  • Data types that would normally be considered part of the core of a language, such as numbers and lists.
  • Built-in functions and exceptions can be used by all Python code without the need of an import statement. Some of these are defined by the core language, but many are not essential for the core semantics.
  • Collection of modules. Some modules are written in C and built in to the Python interpreter; others are written in Python and imported in source form. Some modules provide interfaces that are highly specific to Python, like printing a stack trace; some provide interfaces that are specific to particular operating systems, such as access to specific hardware; others provide interfaces that are specific to a particular application domain, like the World Wide Web. Some modules are available in all versions and ports of Python; others are only available when the underlying system supports or requires them; yet others are available only when a particular configuration option was chosen at the time when Python was compiled and installed.

Here is a list of Python 3.x standard library:

Modules Description
os provides functions for interacting with the operating system
glob provides functions for making file lists from directory wildcard searches
sys command line arguments
getopt processes sys.argv using the conventions of the Unix getopt() function
argparse more powerful and flexible command line processing
re provides regular expression tools for advanced string processing
math gives access to the underlying C library functions for floating point math
random provides tools for making random selections
statistics calculates basic statistical properties (the mean, median, variance, etc.) of numeric data
datetime supplies classes for manipulating dates and times in both simple and complex ways
zlib support common data archiving and compression formats
gzip support common data archiving and compression formats
bz2 support common data archiving and compression formats
lzma support common data archiving and compression formats
zipfile support common data archiving and compression formats
tarfile support common data archiving and compression formats
timeit demonstrates a modest performance advantage
profile provide tools for identifying time critical sections in larger blocks of code
doctest provides a tool for scanning a module and validating tests embedded in a program’s docstrings
unittest allows a more comprehensive set of tests to be maintained in a separate file

The built-in functions, constants, types and exceptions can be listed by dir(__builtins__) or dir(builtins):

chenwx@chenwx:~ $ python3
Python 3.6.8 (default, Jan 14 2019, 11:02:34) 
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> dir(__builtins__)
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'ModuleNotFoundError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']

>>> import builtins
>>> dir(builtins)
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'ModuleNotFoundError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '_', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']

Built-in Functions

The Python interpreter has a number of functions and types built into it that are always available, refer to Python 3.x Built-in Functions:

abs()
all()
any()
ascii()
bin()
bool()
bytearray()
bytes()
callable()
chr()
classmethod()
compile()
complex()
delattr()
dict()
dir()
divmod()
enumerate()
eval()
exec()
filter()
float()
format()
frozenset()
getattr()
globals()
hasattr()
hash()
help()
hex()
id()
input()
int()
isinstance()
issubclass()
iter()
len()
list()
locals()
map()
max()
memoryview()
min()
next()
object()
oct()
open()
ord()
pow()
print()
property()
range()
repr()
reversed()
round()
set()
setattr()
slice()
sorted()
staticmethod()
str()
sum()
super()
tuple()
type()
vars()
zip()
__import__()

You can get help information of the built-in functions by help(), such as:

>>> help(abs)

Help on built-in function abs in module builtins:

abs(x, /)
    Return the absolute value of the argument.

Built-in Constants

A small number of constants live in the built-in namespace. They are:

Constants Description
False The false value of the bool type. Assignments to False are illegal and raise a SyntaxError.
True The true value of the bool type. Assignments to True are illegal and raise a SyntaxError.
None The sole value of the type NoneType. None is frequently used to represent the absence of a value, as when default arguments are not passed to a function. Assignments to None are illegal and raise a SyntaxError.
NotImplemented Special value which should be returned by the binary special methods (e.g. __eq__(), __lt__(), __add__(), __rsub__(), etc.) to indicate that the operation is not implemented with respect to the other type; may be returned by the in-place binary special methods (e.g. __imul__(), __iand__(), etc.) for the same purpose. Its truth value is true.
Ellipsis The same as .... Special value used mostly in conjunction with extended slicing syntax for user-defined container data types.
__debug__ This constant is true if Python was not started with an -O option. Assignments to __debug__ are illegal and raise a SyntaxError.


The site module (which is imported automatically during startup, except if the -S command-line option is given) adds several constants to the built-in namespace. They are useful for the interactive interpreter shell and should not be used in programs. Constants added by the site module are:

Constants Description
quit(code=None)
exit(code=None)
Objects that when printed, print a message like Use quit() or Ctrl-D (i.e. EOF) to exit, and when called, raise SystemExit with the specified exit code.
copyright
license
credits
Objects that when printed, print a message like Type license() to see the full license text, and when called, display the corresponding text in a pager-like fashion (one screen at a time).

Built-in Types

Refer to Built-in Types for more information.

Built-in Exceptions

The class hierarchy for built-in exceptions is:

BaseException
 +-- SystemExit
 +-- KeyboardInterrupt
 +-- GeneratorExit
 +-- Exception
      +-- StopIteration
      +-- StopAsyncIteration
      +-- ArithmeticError
      |    +-- FloatingPointError
      |    +-- OverflowError
      |    +-- ZeroDivisionError
      +-- AssertionError
      +-- AttributeError
      +-- BufferError
      +-- EOFError
      +-- ImportError
           +-- ModuleNotFoundError
      +-- LookupError
      |    +-- IndexError
      |    +-- KeyError
      +-- MemoryError
      +-- NameError
      |    +-- UnboundLocalError
      +-- OSError
      |    +-- BlockingIOError
      |    +-- ChildProcessError
      |    +-- ConnectionError
      |    |    +-- BrokenPipeError
      |    |    +-- ConnectionAbortedError
      |    |    +-- ConnectionRefusedError
      |    |    +-- ConnectionResetError
      |    +-- FileExistsError
      |    +-- FileNotFoundError
      |    +-- InterruptedError
      |    +-- IsADirectoryError
      |    +-- NotADirectoryError
      |    +-- PermissionError
      |    +-- ProcessLookupError
      |    +-- TimeoutError
      +-- ReferenceError
      +-- RuntimeError
      |    +-- NotImplementedError
      |    +-- RecursionError
      +-- SyntaxError
      |    +-- IndentationError
      |         +-- TabError
      +-- SystemError
      +-- TypeError
      +-- ValueError
      |    +-- UnicodeError
      |         +-- UnicodeDecodeError
      |         +-- UnicodeEncodeError
      |         +-- UnicodeTranslateError
      +-- Warning
           +-- DeprecationWarning
           +-- PendingDeprecationWarning
           +-- RuntimeWarning
           +-- SyntaxWarning
           +-- UserWarning
           +-- FutureWarning
           +-- ImportWarning
           +-- UnicodeWarning
           +-- BytesWarning
           +-- ResourceWarning

Collection of Modules

Refer to Python Module Index for the index of all modules.

Text Processing Services

Modules Description CPython_Code
string Common string operations Lib/string.py
re Regular expression operations Lib/re.py
difflib Helpers for computing deltas Lib/difflib.py
textwrap Text wrapping and filling Lib/textwrap.py
unicodedata Unicode Database  
stringprep Internet String Preparation Lib/stringprep.py
readline GNU readline interface  
rlcompleter Completion function for GNU readline Lib/rlcompleter.py

Binary Data Services

Modules Description CPython_Code
struct Interpret bytes as packed binary data Lib/struct.py
codecs Codec registry and base classes Lib/codecs.py

Data Types

Modules Description CPython_Code
datetime Basic date and time types Lib/datetime.py
calendar General calendar-related functions Lib/calendar.py
collections Container datatypes Lib/collections/__init__.py
collections.abc Abstract Base Classes for Containers Lib/_collections_abc.py
heapq Heap queue algorithm Lib/heapq.py
bisect Array bisection algorithm Lib/bisect.py
array Efficient arrays of numeric values  
weakref Weak references Lib/weakref.py
types Dynamic type creation and names for built-in types Lib/types.py
copy Shallow and deep copy operations Lib/copy.py
pprint Data pretty printer Lib/pprint.py
reprlib Alternate repr() implementation Lib/reprlib.py
enum Support for enumerations Lib/enum.py

Numeric and Mathematical Modules

Modules Description CPython_Code
numbers Numeric abstract base classes Lib/numbers.py
math Mathematical functions  
cmath Mathematical functions for complex numbers  
decimal Decimal fixed point and floating point arithmetic Lib/decimal.py
fractions Rational numbers Lib/fractions.py
random Generate pseudo-random numbers Lib/random.py
statistics Mathematical statistics functions Lib/statistics.py

Functional Programming Modules

Modules Description CPython_Code
itertools Functions creating iterators for efficient looping  
functools Higher-order functions and operations on callable objects Lib/functools.py
operator Standard operators as functions Lib/operator.py

File and Directory Access

Modules Description CPython_Code
pathlib Object-oriented filesystem paths Lib/pathlib.py
os.path Common pathname manipulations Lib/posixpath.py (for POSIX)
Lib/ntpath.py (for Windows NT)
Lib/macpath.py (for Macintosh)
fileinput Iterate over lines from multiple input streams Lib/fileinput.py
stat Interpreting stat() results Lib/stat.py
filecmp File and Directory Comparisons Lib/filecmp.py
tempfile Generate temporary files and directories Lib/tempfile.py
glob Unix style pathname pattern expansion Lib/glob.py
fnmatch Unix filename pattern matching Lib/fnmatch.py
linecache Random access to text lines Lib/linecache.py
shutil High-level file operations Lib/shutil.py
macpath Mac OS 9 path manipulation functions Lib/macpath.py

Data Persistence

Modules Description CPython_Code
pickle Python object serialization Lib/pickle.py
copyreg Register pickle support functions Lib/copyreg.py
shelve Python object persistence Lib/shelve.py
marshal Internal Python object serialization  
dbm Interfaces to Unix databases Lib/dbm/__init__.py
sqlite3 DB-API 2.0 interface for SQLite databases Lib/sqlite3/

Data Compression and Archiving

Modules Description CPython_Code
zlib Compression compatible with gzip  
gzip Support for gzip files Lib/gzip.py
bz2 Support for bzip2 compression Lib/bz2.py
lzma Compression using the LZMA algorithm Lib/lzma.py
zipfile Work with ZIP archives Lib/zipfile.py
tarfile Read and write tar archive files Lib/tarfile.py

File Formats

Modules Description CPython_Code
csv CSV File Reading and Writing Lib/csv.py
configparser Configuration file parser Lib/configparser.py
netrc netrc file processing Lib/netrc.py
xdrlib Encode and decode XDR data Lib/xdrlib.py
plistlib Generate and parse Mac OS X .plist files Lib/plistlib.py

Cryptographic Services

Modules Description CPython_Code
hashlib Secure hashes and message digests Lib/hashlib.py
hmac Keyed-Hashing for Message Authentication Lib/hmac.py
secrets Generate secure random numbers for managing secrets Lib/secrets.py

Generic Operating System Services

Modules Description CPython_Code
os Miscellaneous operating system interfaces Lib/os.py
io Core tools for working with streams Lib/io.py
time Time access and conversions  
argparse Parser for command-line options, arguments and sub-commands Lib/argparse.py
getopt C-style parser for command line options Lib/getopt.py
logging Logging facility for Python Lib/logging/__init__.py
logging.config Logging configuration Lib/logging/config.py
logging.handlers Logging handlers Lib/logging/handlers.py
getpass Portable password input Lib/getpass.py
curses Terminal handling for character-cell displays  
curses.textpad Text input widget for curses programs  
curses.ascii Utilities for ASCII characters  
curses.panel A panel stack extension for curses  
platform Access to underlying platform’s identifying data Lib/platform.py
errno Standard errno system symbols  
ctypes A foreign function library for Python  

Concurrent Execution

Modules Description CPython_Code
threading Thread-based parallelism Lib/threading.py
multiprocessing Process-based parallelism Lib/multiprocessing/
concurrent    
concurrent.futures Launching parallel tasks Lib/concurrent/futures/thread.py
Lib/concurrent/futures/process.py
subprocess Subprocess management Lib/subprocess.py
sched Event scheduler Lib/sched.py
queue A synchronized queue class Lib/queue.py
dummy_threading Drop-in replacement for the threading module Lib/dummy_threading.py
_thread Low-level threading API  
_dummy_thread Drop-in replacement for the _thread module Lib/_dummy_thread.py

Interprocess Communication and Networking

Modules Description CPython_Code
socket Low-level networking interface Lib/socket.py
ssl TLS/SSL wrapper for socket objects Lib/ssl.py
select Waiting for I/O completion  
selectors High-level I/O multiplexing Lib/selectors.py
asyncio Asynchronous I/O, event loop, coroutines and tasks Lib/asyncio/
asyncore Asynchronous socket handler Lib/asyncore.py
asynchat Asynchronous socket command/response handler Lib/asynchat.py
signal Set handlers for asynchronous events  
mmap Memory-mapped file support  

Internet Data Handling

Modules Description CPython_Code
email An email and MIME handling package Lib/email/__init__.py
json JSON encoder and decoder Lib/json/__init__.py
mailcap Mailcap file handling Lib/mailcap.py
mailbox Manipulate mailboxes in various formats Lib/mailbox.py
mimetypes Map filenames to MIME types Lib/mimetypes.py
base64 Base16, Base32, Base64, Base85 Data Encodings Lib/base64.py
binhex Encode and decode binhex4 files Lib/binhex.py
binascii Convert between binary and ASCII  
quopri Encode and decode MIME quoted-printable data Lib/quopri.py
uu Encode and decode uuencode files Lib/uu.py

Structured Markup Processing Tools

Modules Description CPython_Code
html HyperText Markup Language support Lib/html/__init__.py
html.parser Simple HTML and XHTML parser Lib/html/parser.py
html.entities Definitions of HTML general entities Lib/html/entities.py
xml XML Processing Modules Lib/xml/
xml.etree.ElementTree The ElementTree XML API Lib/xml/etree/ElementTree.py
xml.dom The Document Object Model API Lib/xml/dom/__init__.py
xml.dom.minidom Minimal DOM implementation Lib/xml/dom/minidom.py
xml.dom.pulldom Support for building partial DOM trees Lib/xml/dom/pulldom.py
xml.sax Support for SAX2 parsers Lib/xml/sax/__init__.py
xml.sax.handler Base classes for SAX handlers Lib/xml/sax/handler.py
xml.sax.saxutils SAX Utilities Lib/xml/sax/saxutils.py
xml.sax.xmlreader Interface for XML parsers Lib/xml/sax/xmlreader.py
xml.parsers.expat Fast XML parsing using Expat  

Internet Protocols and Support

Modules Description CPython_Code
webbrowser Convenient Web-browser controller Lib/webbrowser.py
cgi Common Gateway Interface support Lib/cgi.py
cgitb Traceback manager for CGI scripts Lib/cgitb.py
wsgiref WSGI Utilities and Reference Implementation  
urllib URL handling modules Lib/urllib/
urllib.request Extensible library for opening URLs Lib/urllib/request.py
urllib.response Response classes used by urllib  
urllib.parse Parse URLs into components Lib/urllib/parse.py
urllib.error Exception classes raised by urllib.request Lib/urllib/error.py
urllib.robotparser Parser for robots.txt Lib/urllib/robotparser.py
http HTTP modules Lib/http/__init__.py
http.client HTTP protocol client Lib/http/client.py
ftplib FTP protocol client Lib/ftplib.py
poplib POP3 protocol client Lib/poplib.py
imaplib IMAP4 protocol client Lib/imaplib.py
nntplib NNTP protocol client Lib/nntplib.py
smtplib SMTP protocol client Lib/smtplib.py
smtpd SMTP Server Lib/smtpd.py
telnetlib Telnet client Lib/telnetlib.py
uuid UUID objects according to RFC 4122 Lib/uuid.py
socketserver A framework for network servers Lib/socketserver.py
http.server HTTP servers Lib/http/server.py
http.cookies HTTP state management Lib/http/cookies.py
http.cookiejar Cookie handling for HTTP clients Lib/http/cookiejar.py
xmlrpc XMLRPC server and client modules  
xmlrpc.client XML-RPC client access Lib/xmlrpc/client.py
xmlrpc.server Basic XML-RPC servers Lib/xmlrpc/server.py
ipaddress IPv4/IPv6 manipulation library Lib/ipaddress.py

Multimedia Services

Modules Description CPython_Code
audioop Manipulate raw audio data  
aifc Read and write AIFF and AIFC files Lib/aifc.py
sunau Read and write Sun AU files Lib/sunau.py
wave Read and write WAV files Lib/wave.py
chunk Read IFF chunked data Lib/chunk.py
colorsys Conversions between color systems Lib/colorsys.py
imghdr Determine the type of an image Lib/imghdr.py
sndhdr Determine type of sound file Lib/sndhdr.py
ossaudiodev Access to OSS-compatible audio devices  

Internationalization

Modules Description CPython_Code
gettext Multilingual internationalization services Lib/gettext.py
locale Internationalization services Lib/locale.py

Program Frameworks

Modules Description CPython_Code
turtle Turtle graphics Lib/turtle.py
cmd Support for line-oriented command interpreters Lib/cmd.py
shlex Simple lexical analysis Lib/shlex.py

Graphical User Interfaces with Tk

Modules Description CPython_Code
tkinter Python interface to Tcl/Tk Lib/tkinter/__init__.py
tkinter.ttk Tk themed widgets Lib/tkinter/ttk.py
tkinter.tix Extension widgets for Tk Lib/tkinter/tix.py
tkinter.scrolledtext Scrolled Text Widget Lib/tkinter/scrolledtext.py

Development Tools

Modules Description CPython_Code
typing Support for type hints Lib/typing.py
pydoc Documentation generator and online help system Lib/pydoc.py
doctest Test interactive Python examples Lib/doctest.py
unittest Unit testing framework Lib/unittest/__init__.py
unittest.mock mock object library Lib/unittest/mock.py
2to3 Automated Python 2 to 3 code translation  
test Regression tests package for Python  
test.support Utilities for the Python test suite  

Debugging and Profiling

Modules Description CPython_Code
bdb Debugger framework Lib/bdb.py
faulthandler Dump the Python traceback  
pdb The Python Debugger Lib/pdb.py
cprofile
profile
The Python Profilers Lib/profile.py
Lib/pstats.py
timeit Measure execution time of small code snippets Lib/timeit.py
trace Trace or track Python statement execution Lib/trace.py
tracemalloc Trace memory allocations Lib/tracemalloc.py

Software Packaging and Distribution

Modules Description CPython_Code
distutils Building and installing Python modules  
ensurepip Bootstrapping the pip installer  
venv Creation of virtual environments Lib/venv/
zipapp Manage executable python zip archives Lib/zipapp.py

Python Runtime Services

Modules Description CPython_Code
sys System-specific parameters and functions  
sysconfig Provide access to Python’s configuration information Lib/sysconfig.py
builtins Built-in objects  
__main__ Top-level script environment  
warnings Warning control Lib/warnings.py
contextlib Utilities for with-statement contexts Lib/contextlib.py
abc Abstract Base Classes Lib/abc.py
atexit Exit handlers  
traceback Print or retrieve a stack traceback Lib/traceback.py
__future__ Future statement definitions Lib/__future__.py
gc Garbage Collector interface  
inspect Inspect live objects Lib/inspect.py
site Site-specific configuration hook Lib/site.py
fpectl Floating point exception control  

Custom Python Interpreters

Modules Description CPython_Code
code Interpreter base classes Lib/code.py
codeop Compile Python code Lib/codeop.py

Importing Modules

Modules Description CPython_Code
zipimport Import modules from Zip archives  
pkgutil Package extension utility Lib/pkgutil.py
modulefinder Find modules used by a script Lib/modulefinder.py
runpy Locating and executing Python modules Lib/runpy.py
importlib The implementation of import Lib/importlib/__init__.py

Python Language Services

Modules Description CPython_Code
parser Access Python parse trees  
ast Abstract Syntax Trees Lib/ast.py
symtable Access to the compiler’s symbol tables Lib/symtable.py
symbol Constants used with Python parse trees Lib/symbol.py
token Constants used with Python parse trees Lib/token.py
keyword Testing for Python keywords Lib/keyword.py
tokenize Tokenizer for Python source Lib/tokenize.py
tabnanny Detection of ambiguous indentation Lib/tabnanny.py
pyclbr Python class browser support Lib/pyclbr.py
py_compile Compile Python source files Lib/py_compile.py
compileall Byte-compile Python libraries Lib/compileall.py
dis Disassembler for Python bytecode Lib/dis.py
pickletools Tools for pickle developers Lib/pickletools.py

Miscellaneous Services

Modules Description CPython_Code
formatter Generic output formatting  

MS Windows Specific Services

Modules Description CPython_Code
msilib Read and write Microsoft Installer files Lib/msilib/__init__.py
msvcrt Useful routines from the MS VC++ runtime  
winreg Windows registry access  
winsound Sound-playing interface for Windows  

Unix Specific Services

Modules Description CPython_Code
posix The most common POSIX system calls  
pwd The password database  
spwd The shadow password database  
grp The group database  
crypt Function to check Unix passwords Lib/crypt.py
termios POSIX style tty control  
tty Terminal control functions Lib/tty.py
pty Pseudo-terminal utilities Lib/pty.py
fcntl The fcntl and ioctl system calls  
pipes Interface to shell pipelines Lib/pipes.py
resource Resource usage information  
nis Interface to Sun’s NIS (Yellow Pages)  
syslog Unix syslog library routines  

Superseded Modules

The following modules are deprecated and only kept for backwards compatibility. They have been superseded by other modules.

Modules Description CPython_Code
optparse Parser for command line options Lib/optparse.py
imp Access the import internals Lib/imp.py

Undocumented Modules

These modules are used to implement the os.path module:

Modules Description CPython_Code
ntpath Implementation of os.path on Win32 and Win64 platforms.  
posixpath Implementation of os.path on POSIX.  

Other Python Libraries

Refer to the Python Package Index (PyPI) for packages used by the Python programming language.

Here is 20 Python libraries you can’t live without:

Libraries Description
Requests The most famous HTTP library written by kenneth reitz. It’s a must have for every python developer.
Scrapy If you are involved in WebScraping then this is a must have library for you. After using this library you won’t use any other.
wxPython A GUI toolkit for Python. I have primarily used it in place of Tkinter. You will really love it.
Pillow A friendly fork of PIL (Python Imaging Library). It is more user friendly than PIL and is a must have for anyone who works with images.
SQLAlchemy A database library. Many love it and many hate it. The choice is yours.
BeautifulSoup I know it’s slow but this XML and HTML parsing library is very useful for beginners.
Twisted The most important tool for any network application developer. It has a very beautiful API and is used by a lot of famous python developers.
NumPy How can we leave this very important library? It provides some advance math functionalities to python.
SciPy When we talk about NumPy then we have to talk about SciPy. It is a library of algorithms and mathematical tools for python and has caused many scientists to switch from ruby to python. Also see Scientific Computing via SciPy.
matplotlib A numerical plotting library. It is very useful for any data scientist or any data analyzer.
Pygame Which developer does not like to play games and develop them? This library will help you achieve your goal of 2D game development.
Pyglet A 3D animation and game creation engine. This is the engine in which the famous python port of Minecraft was made.
pyQT A GUI toolkit for python. It is my second choice after wxPython for developing GUI’s for my python scripts.
pyGtk Another python GUI library. It is the same library in which the famous Bittorrent client is created.
Scapy A packet sniffer and analyzer for Python made in Python.
PyWin32 A Python library which provides some useful methods and classes for interacting with windows.
NLTK Natural Language Toolkit – I realize most people won’t be using this one, but it’s generic enough. It is a very useful library if you want to manipulate strings. But it’s capacity is beyond that. Do check it out.
nose A testing framework for Python. It is used by millions of Python developers. It is a must have if you do test driven development.
SymPy SymPy can do algebraic evaluation, differentiation, expansion, complex numbers, etc. It is contained in a pure Python distribution.
IPython I just can’t stress enough how useful this tool is. It is a Python prompt on steroids. It has completion, history, shell capabilities, and a lot more. Make sure that you take a look at it.

How to Install Python Packages

The pip is the PyPA recommended tool for installing Python packages. It is the package installer for Python. You can use pip to install packages from the Python Package Index and other indexes. Refer to:

Install pip

Install pip on Python 2.x:

chenwx@chenwx:~ $ sudo apt install python-pip
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  python-pip-whl
Recommended packages:
  build-essential python-all-dev python-setuptools python-wheel
The following NEW packages will be installed:
  python-pip python-pip-whl
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 1803 kB of archives.
After this operation, 2526 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://mirrors.aliyun.com/ubuntu bionic-updates/universe amd64 python-pip-whl all 9.0.1-2.3~ubuntu1 [1652 kB]
Get:2 http://mirrors.aliyun.com/ubuntu bionic-updates/universe amd64 python-pip all 9.0.1-2.3~ubuntu1 [151 kB]
Fetched 1803 kB in 6s (292 kB/s)
Selecting previously unselected package python-pip-whl.
(Reading database ... 294441 files and directories currently installed.)
Preparing to unpack .../python-pip-whl_9.0.1-2.3~ubuntu1_all.deb ...
Unpacking python-pip-whl (9.0.1-2.3~ubuntu1) ...
Selecting previously unselected package python-pip.
Preparing to unpack .../python-pip_9.0.1-2.3~ubuntu1_all.deb ...
Unpacking python-pip (9.0.1-2.3~ubuntu1) ...
Setting up python-pip-whl (9.0.1-2.3~ubuntu1) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Setting up python-pip (9.0.1-2.3~ubuntu1) ...

chenwx@chenwx:~ $ python -m pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)

chenwx@chenwx:~ $ pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)

Install pip on Python 3.x:

chenwx@chenwx:~ $ sudo apt install python3-pip
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  python3-distutils python3-lib2to3
Recommended packages:
  build-essential python3-dev python3-setuptools python3-wheel
The following NEW packages will be installed:
  python3-distutils python3-lib2to3 python3-pip
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 331 kB of archives.
After this operation, 2714 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 python3-lib2to3 all 3.6.7-1~18.04 [76.5 kB]
Get:2 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 python3-distutils all 3.6.7-1~18.04 [141 kB]
Get:3 http://mirrors.aliyun.com/ubuntu bionic-updates/universe amd64 python3-pip all 9.0.1-2.3~ubuntu1 [114 kB]
Fetched 331 kB in 3s (113 kB/s)  
Selecting previously unselected package python3-lib2to3.
(Reading database ... 294575 files and directories currently installed.)
Preparing to unpack .../python3-lib2to3_3.6.7-1~18.04_all.deb ...
Unpacking python3-lib2to3 (3.6.7-1~18.04) ...
Selecting previously unselected package python3-distutils.
Preparing to unpack .../python3-distutils_3.6.7-1~18.04_all.deb ...
Unpacking python3-distutils (3.6.7-1~18.04) ...
Selecting previously unselected package python3-pip.
Preparing to unpack .../python3-pip_9.0.1-2.3~ubuntu1_all.deb ...
Unpacking python3-pip (9.0.1-2.3~ubuntu1) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Setting up python3-lib2to3 (3.6.7-1~18.04) ...
Setting up python3-distutils (3.6.7-1~18.04) ...
Setting up python3-pip (9.0.1-2.3~ubuntu1) ...

chenwx@chenwx:~ $ python3 -m pip --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)

chenwx@chenwx:~ $ pip3 --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)

Here is a list of pip’s sub-commands:

chenwx@chenwx:~ $ python3 -m pip
Usage:   
  /usr/bin/python3 -m pip <command> [options]

Commands:
  install                     Install packages.
  download                    Download packages.
  uninstall                   Uninstall packages.
  freeze                      Output installed packages in requirements format.
  list                        List installed packages.
  show                        Show information about installed packages.
  check                       Verify installed packages have compatible dependencies.
  search                      Search PyPI for packages.
  wheel                       Build wheels from your requirements.
  hash                        Compute hashes of package archives.
  completion                  A helper command used for command completion.
  help                        Show help for commands.

General Options:
  -h, --help                  Show help.
  --isolated                  Run pip in an isolated mode, ignoring environment variables and user configuration.
  -v, --verbose               Give more output. Option is additive, and can be used up to 3 times.
  -V, --version               Show version and exit.
  -q, --quiet                 Give less output. Option is additive, and can be used up to 3 times (corresponding to
                              WARNING, ERROR, and CRITICAL logging levels).
  --log <path>                Path to a verbose appending log.
  --proxy <proxy>             Specify a proxy in the form [user:passwd@]proxy.server:port.
  --retries <retries>         Maximum number of retries each connection should attempt (default 5 times).
  --timeout <sec>             Set the socket timeout (default 15 seconds).
  --exists-action <action>    Default action when a path already exists: (s)witch, (i)gnore, (w)ipe, (b)ackup,
                              (a)bort.
  --trusted-host <hostname>   Mark this host as trusted, even though it does not have valid or any HTTPS.
  --cert <path>               Path to alternate CA bundle.
  --client-cert <path>        Path to SSL client certificate, a single file containing the private key and the
                              certificate in PEM format.
  --cache-dir <dir>           Store the cache data in <dir>.
  --no-cache-dir              Disable the cache.
  --disable-pip-version-check
                              Don't periodically check PyPI to determine whether a new version of pip is available for
                              download. Implied with --no-index.

Install Python Packages via pip

Take installing package numPy for instance:

chenwx@chenwx:~ $ python3 -m pip --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)

chenwx@chenwx:~ $ python3 -m pip list --format=columns
Package               Version  
--------------------- ---------
apt-clone             0.2.1    
apturl                0.5.2    
attrs                 19.1.0   
backcall              0.1.0    
beautifulsoup4        4.6.0    
bleach                3.1.0    
Brlapi                0.6.6    
certifi               2018.1.18
chardet               3.0.4    
command-not-found     0.3      
configobj             5.0.6    
cupshelpers           1.0      
cycler                0.10.0   
decorator             4.4.0    
defer                 1.0.6    
defusedxml            0.6.0    
entrypoints           0.3      
httplib2              0.9.2    
idna                  2.6      
ipykernel             5.1.1    
ipython               7.5.0    
ipython-genutils      0.2.0    
ipywidgets            7.4.2    
jedi                  0.13.3   
Jinja2                2.10.1   
jsonschema            3.0.1    
jupyter               1.0.0    
jupyter-client        5.2.4    
jupyter-console       6.0.0    
jupyter-core          4.4.0    
kiwisolver            1.1.0    
louis                 3.5.0    
macaroonbakery        1.1.3    
Mako                  1.0.7    
MarkupSafe            1.1.1    
matplotlib            3.0.3    
meld                  3.18.0   
mistune               0.8.4    
mpmath                1.1.0    
nbconvert             5.5.0    
nbformat              4.4.0    
nemo-emblems          4.0.2    
netifaces             0.10.4   
nose                  1.3.7    
notebook              5.7.8    
numpy                 1.16.3   
onboard               1.4.1    
PAM                   0.4.2    
pandas                0.24.2   
pandocfilters         1.4.2    
parso                 0.4.0    
pexpect               4.7.0    
pickleshare           0.7.5    
Pillow                5.1.0    
pip                   9.0.1    
prometheus-client     0.6.0    
prompt-toolkit        2.0.9    
protobuf              3.0.0    
psutil                5.4.2    
ptyprocess            0.6.0    
pycairo               1.16.2   
pycrypto              2.6.1    
pycups                1.9.73   
pycurl                7.43.0.1
Pygments              2.4.0    
pygobject             3.26.1   
PyICU                 1.9.8    
pyinotify             0.9.6    
pymacaroons           0.13.0   
PyNaCl                1.1.2    
pyparsing             2.4.0    
pyRFC3339             1.0      
pyrsistent            0.15.2   
python-apt            1.6.4    
python-dateutil       2.8.0    
python-debian         0.1.32   
python-xapp           1.4.0    
python-xlib           0.20     
pytz                  2019.1   
pyxdg                 0.25     
PyYAML                3.12     
pyzmq                 18.0.1   
qtconsole             4.4.4    
reportlab             3.4.0    
requests              2.18.4   
requests-unixsocket   0.1.5    
scipy                 1.2.1    
Send2Trash            1.5.0    
sessioninstaller      0.0.0    
setproctitle          1.1.10   
setuptools            41.0.1   
six                   1.12.0   
sympy                 1.4      
system-service        0.3      
systemd-python        234      
terminado             0.8.2    
testpath              0.4.2    
tornado               6.0.2    
traitlets             4.3.2    
ubuntu-drivers-common 0.0.0    
ufw                   0.36     
urllib3               1.22     
wcwidth               0.1.7    
webencodings          0.5.1    
widgetsnbextension    3.4.2    
xkit                  0.0.0    

chenwx@chenwx:~ $ python3 -m pip search numpy
numpy (1.16.3)                         - NumPy is the fundamental package for array computing with Python.
  INSTALLED: 1.16.3 (latest)
numpy-utils (0.1.3)                    - NumPy utilities.
numpy-cloud (0.0.5)                    - Numpy in the cloud
numpy-turtle (0.1)                     - Turtle graphics with NumPy
numpy-sugar (1.5.0)                    - Missing NumPy functionalities
root-numpy (4.8.0)                     - The interface between ROOT and NumPy
msgpack-numpy (0.4.4.3)                - Numpy data serialization using msgpack
numpy-quaternion (2019.3.21.14.22.55)  - Add built-in support for quaternions to numpy
numpy-partition (1.18.9)               - SQL PARTITION BY and window functions for NumPy
mapchete-numpy (0.1)                   - Mapchete NumPy read/write extension
numpy-posit (1.15.2.0.0.1.dev2)        - posit (unum type III) integrated NumPy.
intel-numpy (1.15.1)                   - NumPy optimized with Intel(R) MKL library
numpy-mkl (1.10.2)                     - NumPy: array processing for numbers, strings, records, and objects.
ccv-numpy (0.0.2)                      - Wrapper module for ccv using numpy arrays interface
django-numpy (1.0.1)                   - Application for Django projects that adds some utilities and integration tools with Numpy.
BSON-NumPy (0.1)                       - Module for converting directly from BSON to NumPy ndarrays and vice versa
dicom-numpy (0.1.6)                    - Extract image data into a 3D numpy array from a set of DICOM files.
dl-with-numpy (0.0.3)                  - Simple deep learning with numpy
numpy-html (0.0.3)                     - A simple table renderer for numpy arrays. Provides a rich display hook for use with Jupyter Lab / Notebook.
numpy-indexed (0.3.5)                  - This package contains functionality for indexed operations on numpy ndarrays, providing efficient vectorized functionality such as grouping and set operations.
numpy-image-widget (2019.1.6)          - An easy-to-use Jupyter widget for displaying images from Numpy data arrays
django-numpy-json-encoder (0.1.3)      - Django JSON encoder with numpy support.
faster-numpy (0.1.2)                   -
numpy-stl (2.10.1)                     - Library to make reading, writing and modifying both binary and ascii STL files easy.
hypothesis-numpy (2.0.0)               - Adds support for generating datetime to Hypothesis
numpy-groupies (0.9.9)                 - Optimised tools for group-indexing operations: aggregated sum and more.
npplus (0.9.8)                         - Enhancements to Numpy
o243kgmvs924mtvd034mg (0.01)           - NumPy Testing
np (1.0.2)                             - np = numpy++: numpy with added convenience functionality
maskedarray (0.1)                      - NumPy Masked Array
jumpy (0.2.4)                          - Numpy and nd4j interop
nparray (1.0.0)                        - Numpy Array Helpers
oldnumeric (1.0.4)                     - The oldnumeric numpy package
nptyping (0.2.0)                       - Type hints for Numpy
match-arrays (0.0.1)                   - Matching of Numpy arrays
vidsrc (2019.1.1)                      - Video Frameserver for Numpy
ndarray (0.8)                          - Tools for numpy ndarray
wrapnumpy3 (0.0.3)                     - Wrapper of Numpy in Python3.
sqlite3ndarray (0.3.1)                 - sqlite3 helpers for numpy ndarrays
axarray (0.1.1)                        - numpy array with labeled axes
somber (2.0.1)                         - Self-Organizing Maps in Numpy
gsn_numpy_util (0.1.0)                 - Toolbox for working with Numpy arrays.
homog (0.1.14)                         - Utilities for Homogeneous Coordinates with Numpy
npstreams (1.6)                        - Streaming operations on NumPy arrays
sym2num (0.1.dev2)                     - Sympy to numpy code generator.
blocked-matrix-utils (1.0.1)           - NumPy wrappers for blocked matrices
dynarray (0.1.3)                       - Dynamically growable numpy arrays.
zappy (0.2.0)                          - Distributed processing with NumPy and Zarr.
permanent (1.1.1)                      - Calculates the permanent of a Numpy matrix
pstar (0.1.6)                          - pstar: numpy for arbitrary data
sompy (0.1.1)                          - Numpy based SOM Library
spartan (0.06)                         - Distributed Numpy-like arrays.
fastjet (0.0.3)                        - The interface between FastJet and NumPy
npdoc (1.0.1)                          - In-notebook NumPy man pages
gumpy (0.0.1)                          - GPU-accelerated NumPy API
wendelin.core (0.12)                   - Out-of-core NumPy arrays
sampyl-mcmc (0.3)                      - MCMC Samplers in Python & Numpy
gnuplotlib (0.29)                      - Gnuplot-based plotting for numpy
multipolyfit (0.0.1)                   - Multivariate Polynomial fitting with NumPy
nppretty (1.0.0)                       - Pretty printer for numpy arrays.
numpythia (1.1.0)                      - The interface between PYTHIA and NumPy
PyUblas (2017.1)                       - Seamless Numpy-UBlas interoperability
numm (0.3)                             - numpy-based multimedia library
ntype (0.0.1)                          - Type Handler using NumPy
fastfunc (0.1.7)                       - Fast numpy ufunc operations
pyjet (1.5.0)                          - The interface between FastJet and NumPy
snuggs (1.4.6)                         - Snuggs are s-expressions for Numpy
array_collections (0.1.7)              - A collection of numpy ndarray subclasses.
pyache (0.1.0)                         - A simple numpy caching library
opt-einsum (2.3.2)                     - Optimizing numpys einsum function
arkouda (0.0.0)                        - A numpy replacement backed by Chapel
fastremap (1.5.2)                      - Remap and renumber numpy arrays.
fewerbytes (0.0.1)                     - Compression techniques for numpy arrays
numpy_ringbuffer (0.2.1)               - Ring buffer implementation for numpy
bohrium (0.11.0.post15)                - Bohrium Python/NumPy Backend
afnumpy (1.3)                          - A GPU-ready drop-in replacement for numpy
ComStats (0.1.0)                       - Do combinatorial statistics on numpy ndarrays
xcy-Zscore (0.0.5)                     - A small tool of Zscore based on numpy
marshmallow-numerical (0.0.2)          - Marshmallow Schema generator for pandas and numpy
numpyx (0.3.0)                         - Utility functions for numpy, written in cython
sortednp (0.2.1)                       - Merge and intersect sorted numpy arrays.
numphy (0.0.2)                         - Physics objects backed by NumPy and/or TensorFlow.
vector-shortcuts (0.2.0)               - Vector and linear algebra toolbelt for NumPy
zmqnumpy (0.1)                         - numpy array over zmq sockets
pynetcdf (0.7)                         - Standalone Scientific.IO.NetCDF (2.4.11) built for NumPy
npcb (0.0.1)                           - Copy NumPy arrays via the clipboard
spacegrids (1.9)                       - numpy array with grids and associated operations
ndf (0.1.4)                            - NumPy based deep learning framework
multipletau (0.3.3)                    - A multiple-tau algorithm for Python/NumPy
seabred (0.0.1)                        - Seaborn-like plots for numpy arrays
numexpr (2.6.9)                        - Fast numerical expression evaluator for NumPy
tensorfont (0.0.2)                     - Turn font glyphs into numpy arrays
numpy_display (1.2.0)                  - Formats numpy matrices in an IPython Notebook
nnlib (0.2)                            - numpy only neural network library
matplottery (1.0.4)                    - Nicer histograms with numpy and plotting with matplotlib
geog (0.0.2)                           - Numpy-based vectorized geospatial functions
stft (0.5.2)                           - Short Time Fourier transform for NumPy.
hdrpy (0.3.3)                          - HDR histogram implementation based on numpy
np2d (0.1.1)                           - Common 2-D NumPy operations.
pySLALIB (1.0.4)                       - f2py and numpy based wrappers for SLALIB

chenwx@chenwx:~ $ python3 -m pip install numpy
Collecting numpy
  Downloading https://files.pythonhosted.org/packages/e0/b5/63b79fe426433fa1cd110eb04a94ec0c6967e56e5f57c98caf455a5fb6e2/numpy-1.16.1-cp27-cp27mu-manylinux1_x86_64.whl (17.0MB)
    100% |################################| 17.0MB 51kB/s
Installing collected packages: numpy
Successfully installed numpy-1.16.1

chenwx@chenwx:~ $ python3 -m pip show numpy
Name: numpy
Version: 1.16.1
Summary: NumPy is the fundamental package for array computing with Python.
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: None
License: BSD
Location: /home/chenwx/.local/lib/python2.7/site-packages
Requires:

chenwx@chenwx:~ $ python3
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy  
>>> numpy.eye(4)
array([[1., 0., 0., 0.],
       [0., 1., 0., 0.],
       [0., 0., 1., 0.],
       [0., 0., 0., 1.]])

Python Quick Reference

Python Code Conventions

PEP 8: Style Guide for Python Code

For Python, PEP 8: Style Guide for Python Code has emerged as the style guide that most projects adhere to; it promotes a very readable and eye-pleasing coding style. Every Python developer should read it at some point; here are the most important points extracted for you:

  • Use 4-space indentation, and no tabs. 4 spaces are a good compromise between small indentation (allows greater nesting depth) and large indentation (easier to read). Tabs introduce confusion, and are best left out.

  • Wrap lines so that they don’t exceed 79 characters. This helps users with small displays and makes it possible to have several code files side-by-side on larger displays.

  • Use blank lines to separate functions and classes, and larger blocks of code inside functions.

  • When possible, put comments on a line of their own.

  • Use docstrings.

  • Use spaces around operators and after commas, but not directly inside bracketing constructs: a = f(1, 2) + g(3, 4).

  • Name your classes and functions consistently; the convention is to use CamelCase for classes and lower_case_with_underscores for functions and methods. Always use self as the name for the first method argument.

  • Don’t use fancy encodings if your code is meant to be used in international environments. Python’s default, UTF-8, or even plain ASCII work best in any case. Likewise, don’t use non-ASCII characters in identifiers if there is only the slightest chance people speaking a different language will read or maintain the code.

autopep8

The tool autopep8 is used to format the Python code to follow rules defined in PEP 8: Style Guide for Python Code:

$ pip install --upgrade autopep8

$ autopep8 --in-place --aggressive <filename>

Google Python Style Guide

Python Open Source Projects

Python Developers Survey

Python FAQ

Find the location of your user site-packages directory:

>>> import site
>>> site.getusersitepackages()
'/home/ewaadex/.local/lib/python3.5/site-packages'

Python IDEs

Python Books

References