Python
This article introduces the Python programming language.
Overview
Python is a widely used high-level programming language for general-purpose programming, created by Guido van Rossum and first released in 1991. As an interpreted language, Python has a design philosophy which emphasizes code readability (notably using whitespace indentation to delimit code blocks rather than curly braces or keywords), and a syntax which allows programmers to express concepts in fewer lines of code than possible in languages such as C++ or Java. The language provides constructs intended to enable writing clear programs on both a small and large scale.
Python features a dynamic type system and automatic memory management and supports multiple programming paradigms, including object-oriented, imperative, functional programming, and procedural styles. It has a large and comprehensive standard library.
Python interpreters are available for many operating systems, allowing Python code to run on a wide variety of systems. CPython, the reference implementation of Python, is open source software and has a community-based development model, as do nearly all of its variant implementations. CPython is managed by the non-profit Python Software Foundation (PSF).
Python Releases
Refer to Python downloads page for Python releases, and What’s New in Python for release notes.
Refer to Should I use Python 2 or Python 3 for my development activity for the critical differences between Python 2.x and 3.x. Also refer to Porting Python 2 Code to Python 3.
According to PEP 373: Python 2.7 Release Schedule, the End Of Life date (EOL, sunset date) for Python 2.7 has been moved five years into the future, to 2020. This decision was made to clarify the status of Python 2.7 and relieve worries for those users who cannot yet migrate to Python 3. See also PEP 466: Network Security Enhancements for Python 2.7.x. This declaration does not guarantee that bugfix releases will be made on a regular basis, but it should enable volunteers who want to contribute bugfixes for Python 2.7 and it should satisfy vendors who still have to support Python 2 for years to come.
According to PEP 404: Python 2.8 Un-release Schedule:
- There is no official Python 2.8 release. There never will be an official Python 2.8 release. It is an ex-release. Python 2.7 is the end of the Python 2 line of development.
- The official upgrade path from Python 2.7 is to Python 3.
And the Python 2.7 Countdown shows a down counter of Python 2.7 retirement. The moving to require Python 3 shows the projects that have pledged to drop support for Python 2.7 no later than 2020.
Python 2
Python 2.0.x
Release | Date |
---|---|
2.0.1 | 2001-06-22 |
Python 2.1.x
Release | Date |
---|---|
2.1.3 | 2002-04-09 |
Python 2.2.x
Release | Date |
---|---|
2.2.0 | 2001-12-21 |
2.2.1 | 2002-04-10 |
2.2.2 | 2002-10-14 |
2.2.3 | 2003-05-30 |
Python 2.3.x
Release | Date |
---|---|
2.3.0 | 2003-07-29 |
2.3.1 | 2003-09-23 |
2.3.2 | 2003-10-03 |
2.3.3 | 2003-12-19 |
2.3.4 | 2004-05-27 |
2.3.5 | 2005-02-08 |
2.3.6 | 2006-11-01 |
2.3.7 | 2008-03-11 |
Python 2.4.x
Release | Date |
---|---|
2.4.0 | 2004-11-30 |
2.4.1 | 2005-03-30 |
2.4.2 | 2005-09-27 |
2.4.3 | 2006-04-15 |
2.4.4 | 2006-10-18 |
2.4.5 | 2008-03-11 |
2.4.6 | 2008-12-19 |
Python 2.5.x
Release | Date |
---|---|
2.5.0 | 2006-09-19 |
2.5.1 | 2007-04-19 |
2.5.2 | 2008-02-21 |
2.5.3 | 2008-12-19 |
2.5.4 | 2008-12-23 |
2.5.5 | 2010-01-31 |
2.5.6 | 2011-05-26 |
Python 2.6.x
Release | Date |
---|---|
2.6.0 | 2008-10-02 |
2.6.1 | 2008-12-04 |
2.6.2 | 2009-04-14 |
2.6.3 | 2009-10-02 |
2.6.4 | 2009-10-26 |
2.6.5 | 2010-03-18 |
2.6.6 | 2010-08-24 |
2.6.7 | 2011-06-03 |
2.6.8 | 2012-04-10 |
2.6.9 | 2013-10-29 |
Python 2.7.x
Release | Date |
---|---|
2.7.0 | 2010-07-03 |
2.7.1 | 2010-11-27 |
2.7.2 | 2011-06-11 |
2.7.3 | 2012-04-09 |
2.7.4 | 2013-04-06 |
2.7.5 | 2013-05-12 |
2.7.6 | 2013-11-10 |
2.7.7 | 2014-06-01 |
2.7.8 | 2014-07-02 |
2.7.9 | 2014-12-10 |
2.7.10 | 2015-05-23 |
2.7.11 | 2015-12-05 |
2.7.12 | 2016-06-25 |
2.7.13 | 2016-12-17 |
2.7.14 | 2017-09-16 |
2.7.15 | 2018-05-01 |
2.7.16 | 2019-03-04 |
2.7.17 | 2019-10-19 |
2.7.18 | 2020-04-20 |
Python 3
Python 3.0.x
Release | Date |
---|---|
3.0.0 | 2008-12-03 |
3.0.1 | 2009-02-13 |
Python 3.1.x
Release | Date |
---|---|
3.1.0 | 2009-06-26 |
3.1.1 | 2009-08-17 |
3.1.2 | 2010-03-20 |
3.1.3 | 2010-11-27 |
3.1.4 | 2011-06-11 |
3.1.5 | 2012-04-09 |
Python 3.2.x
Release | Date |
---|---|
3.2.0 | 2011-02-20 |
3.2.1 | 2011-07-09 |
3.2.2 | 2011-09-03 |
3.2.3 | 2012-04-10 |
3.2.4 | 2013-04-06 |
3.2.5 | 2013-05-15 |
3.2.6 | 2014-10-12 |
Python 3.3.x
Release | Date |
---|---|
3.3.0 | 2012-09-29 |
3.3.1 | 2013-04-06 |
3.3.2 | 2013-05-15 |
3.3.3 | 2013-11-17 |
3.3.4 | 2014-02-09 |
3.3.5 | 2014-03-09 |
3.3.6 | 2014-10-12 |
3.3.7 | 2017-09-19 |
Python 3.4.x
Release | Date |
---|---|
3.4.0 | 2014-03-17 |
3.4.1 | 2014-05-19 |
3.4.2 | 2014-10-13 |
3.4.3 | 2015-02-25 |
3.4.4 | 2015-12-21 |
3.4.5 | 2016-06-27 |
3.4.6 | 2017-01-17 |
3.4.7 | 2017-08-09 |
3.4.8 | 2018-02-05 |
3.4.9 | 2018-08-02 |
3.4.10 | 2019-03-18 |
Python 3.5.x
Release | Date |
---|---|
3.5.0 | 2015-09-13 |
3.5.1 | 2015-12-07 |
3.5.2 | 2016-06-27 |
3.5.3 | 2017-01-17 |
3.5.4 | 2017-08-08 |
3.5.5 | 2018-02-05 |
3.5.6 | 2018-08-02 |
3.5.7 | 2019-03-18 |
3.5.8 | 2019-10-29 |
3.5.9 | 2019-11-02 |
Python 3.6.x
Release | Date |
---|---|
3.6.0 | 2016-12-23 |
3.6.1 | 2017-03-21 |
3.6.2 | 2017-07-17 |
3.6.3 | 2017-10-03 |
3.6.4 | 2017-12-19 |
3.6.5 | 2018-03-28 |
3.6.6 | 2018-06-27 |
3.6.7 | 2018-10-20 |
3.6.8 | 2018-12-24 |
3.6.9 | 2019-07-02 |
3.6.10 | 2019-12-18 |
Python 3.7.x
Release | Date |
---|---|
3.7.0 | 2018-06-27 |
3.7.1 | 2018-10-20 |
3.7.2 | 2018-12-24 |
3.7.3 | 2019-03-25 |
3.7.4 | 2019-07-08 |
3.7.5 | 2019-10-15 |
3.7.6 | 2019-12-18 |
3.7.7 | 2020-03-10 |
Python 3.8.x
Release | Date |
---|---|
3.8.0 | 2019-10-14 |
3.8.1 | 2019-12-18 |
3.8.2 | 2020-02-24 |
Python Software Foundation (PSF)
The Python Software Foundation (PSF) is an independent non-profit organization that holds the copyright on Python versions 2.1 and newer. The PSF’s mission is to advance open source technology related to the Python programming language and to publicize the use of Python.
Python Enhancement Proposals (PEPs)
Python’s development is conducted largely through the Python Enhancement Proposal (PEP) process. The PEP process is the primary mechanism for proposing major new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. Outstanding PEPs are reviewed and commented upon by the Python community and by Van Rossum, the Python project’s benevolent dictator for life. Refer to Index of Python Enhancement Proposals (PEPs) and Python Developer’s Guide.
Python Implementations
There are some alternate implementations which are of particular interest to different audiences. Refer to Alternative Python Implementations for more information. The known implementations include:
-
CPython
This is the original and most-maintained implementation of Python, written in C. New language features generally appear here first. Refer to CPython for more information.
-
Jython
Python implemented in Java. This implementation can be used as a scripting language for Java applications, or can be used to create applications using the Java class libraries. It is also often used to create tests for Java libraries. More information can be found at the Jython website.
-
Python for .NET
This implementation actually uses the CPython implementation, but is a managed .NET application and makes .NET libraries available. It was created by Brian Lloyd. For more information, see the Python for .NET website.
-
IronPython
An alternate Python for .NET. Unlike Python.NET, this is a complete Python implementation that generates IL, and compiles Python code directly to .NET assemblies. It was created by Jim Hugunin, the original creator of Jython. For more information, see the IronPython website.
-
PyPy
An implementation of Python written completely in Python. It supports several advanced features not found in other implementations like stackless support and a Just in Time compiler (JIT). One of the goals of the project is to encourage experimentation with the language itself by making it easier to modify the interpreter (since it is written in Python). Additional information is available on the PyPy website.
CPython
The latest development sources of Python can be obtained at Git Repo:
chenwx@chenwx ~ $ git clone https://github.com/python/cpython.git
Directory structure of CPython source tree is listed below:
chenwx@chenwx ~ $ cd cpython/
chenwx@chenwx ~/cpython $ ls
Doc Mac Modules Programs config.guess configure.ac
Grammar Makefile Objects Python config.log install-sh
Include Makefile.pre PC README.rst config.status pyconfig.h
LICENSE Makefile.pre.in PCbuild Tools config.sub pyconfig.h.in
Lib Misc Parser aclocal.m4 configure setup.py
-
Doc
The official documentation. This is what https://docs.python.org/ uses. See also Building the documentation.
-
Grammar
Contains the Extended Backus-Naur Form (EBNF) grammar file for Python.
-
Include
Contains all interpreter-wide header files.
-
Lib
The part of the standard library implemented in pure Python.
-
Mac
Mac-specific code (e.g., using IDLE as an OS X application).
-
Misc
Things that do not belong elsewhere. Typically this is varying kinds of developer-specific documentation.
-
Modules
The part of the standard library (plus some other code) that is implemented in C.
-
Objects
Code for all built-in types.
-
PC
Windows-specific code.
-
PCbuild
Build files for the version of MSVC currently used for the Windows installers provided on python.org.
-
Parser
Code related to the parser. The definition of the AST nodes is also kept here.
-
Programs
Source code for C executables, including the main function for the CPython interpreter (in versions prior to Python 3.5, these files are in the Modules directory).
-
Python
The code that makes up the core CPython runtime. This includes the compiler, eval loop and various built-in modules.
-
Tools
Various tools that are (or have been) used to maintain Python.
Building CPython
On Debian, Ubuntu, and other apt based systems, try to get the dependencies for the Python version that you’re working on. If that package is not available for your system, try reducing the minor version until you find a package that is available.
chenwx@chenwx ~ $ sudo apt-get build-dep python3.5
Run the following command to build Python from source code, refer to Getting Started section of Python Developer’s Guide for details:
chenwx@chenwx ~ $ cd cpython/
chenwx@chenwx ~/cpython $ git co v3.6.1
HEAD is now at 69c0db5050f6... Update docs and patchlevel for 3.6.1 final
chenwx@chenwx ~/cpython $ ./configure --with-pydebug
...
configure: creating ./config.status
config.status: creating Makefile.pre
config.status: creating Modules/Setup.config
config.status: creating Misc/python.pc
config.status: creating Misc/python-config.sh
config.status: creating Modules/ld_so_aix
config.status: creating pyconfig.h
config.status: pyconfig.h is unchanged
creating Modules/Setup
creating Modules/Setup.local
creating Makefile
chenwx@chenwx ~/cpython $ make -s -j2
...
Python build finished successfully!
The necessary bits to build these optional modules were not found:
_bz2 _lzma
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
atexit pwd time
Failed to build these modules:
_tkinter
chenwx@chenwx ~/cpython $ ll python
-rwxrwxr-x 1 chenwx chenwx 11M Mar 29 17:33 python
chenwx@chenwx ~/cpython $ ./python -V
Python 3.6.1
chenwx@chenwx ~/cpython $ ./python -VV
Python 3.6.1 (v3.6.1:69c0db5050f6, Mar 29 2017, 17:33:44)
[GCC 5.4.0 20160609]
There is normally no need to install your built copy of Python because which is already installed by Linux distributions, refer to Python Installation.
Python Installation
Refer to PEP 394: The “python” Command on Unix-Like Systems, the Python is already installed on my LinuxMint:
chenwx@chenwx ~ $ which python
/usr/bin/python
chenwx@chenwx ~ $ ll /usr/bin/python
lrwxrwxrwx 1 root root 9 Dec 10 2015 /usr/bin/python -> python2.7
chenwx@chenwx ~ $ which python2
/usr/bin/python2
chenwx@chenwx ~ $ ll /usr/bin/python2
lrwxrwxrwx 1 root root 9 Dec 10 2015 /usr/bin/python2 -> python2.7
chenwx@chenwx ~ $ which python3
/usr/bin/python3
chenwx@chenwx ~ $ ll /usr/bin/python3
lrwxrwxrwx 1 root root 9 Mar 23 2016 /usr/bin/python3 -> python3.5
chenwx@chenwx ~ $ which python3m
/usr/bin/python3m
chenwx@chenwx ~ $ ll /usr/bin/python3m
lrwxrwxrwx 1 root root 10 Mar 23 2016 /usr/bin/python3m -> python3.5m
chenwx@chenwx ~ $ diff /usr/bin/python3.5 /usr/bin/python3.5m
chenwx@chenwx ~ $ ll /usr/bin/python*
lrwxrwxrwx 1 root root 9 Dec 10 2015 /usr/bin/python -> python2.7
lrwxrwxrwx 1 root root 16 Dec 10 2015 /usr/bin/python-config -> python2.7-config
lrwxrwxrwx 1 root root 9 Dec 10 2015 /usr/bin/python2 -> python2.7
lrwxrwxrwx 1 root root 16 Dec 10 2015 /usr/bin/python2-config -> python2.7-config
-rwxr-xr-x 1 root root 3.4M Nov 19 17:35 /usr/bin/python2.7
lrwxrwxrwx 1 root root 33 Nov 19 17:35 /usr/bin/python2.7-config -> x86_64-linux-gnu-python2.7-config
lrwxrwxrwx 1 root root 9 Mar 23 2016 /usr/bin/python3 -> python3.5
-rwxr-xr-x 2 root root 4.3M Nov 18 03:23 /usr/bin/python3.5
-rwxr-xr-x 2 root root 4.3M Nov 18 03:23 /usr/bin/python3.5m
lrwxrwxrwx 1 root root 10 Mar 23 2016 /usr/bin/python3m -> python3.5m
Alternatives to Python Interpreter
One alternative enhanced interactive interpreter that has been around for quite some time is IPython, which features tab completion, object exploration and advanced history management. It can also be thoroughly customized and embedded into other applications.
Another similar enhanced interactive environment is bpython.
Python Commands
Command Options
When invoking Python, you may specify any of these options, refer to command line and environment:
python2 [ -bBdEiOQRsStuUvVWxX3? ] [ -c command | -m module-name | script | - ] [ args ]
python3 [ -bBdEhiIOqsSuvVWx? ] [ -c command | -m module-name | script | - ] [ args ]
Options | Description |
---|---|
-? -h --help |
Print a short description of all command line options. |
-V --version -VV |
Print the Python version number and exit. When given twice (-VV ), print more information about the build. |
-v -vv |
Print a message each time a module is initialized, showing the place (filename or built-in module) from which it is loaded. When given twice (-vv ), print a message for each file that is checked for when searching for a module. Also provides information on module cleanup at exit. See also PYTHONVERBOSE. |
-b -bb |
Issue a warning when comparing bytes or bytearray with str , or bytes with int . Issue an error when the option is given twice (-bb ). Changed in version 3.5: Affects comparisons of bytes with int . |
-B |
If given, Python won’t try to write .pyc files on the import of source modules. See also PYTHONDONTWRITEBYTECODE. |
-d |
Turn on parser debugging output (for wizards only, depending on compilation options). See also PYTHONDEBUG. |
-E |
Ignore all PYTHON* environment variables, e.g. PYTHONPATH and PYTHONHOME, that might be set, see Environment Variables. |
-i |
When a script is passed as first argument or the -c option is used, enter interactive mode after executing the script or the command, even when sys.stdin does not appear to be a terminal. The PYTHONSTARTUP file is not read. This can be useful to inspect global variables or a stack trace when a script raises an exception. See also PYTHONINSPECT. |
-I |
Run Python in isolated mode. This also implies -E and -s. In isolated mode sys.path contains neither the script’s directory nor the user’s site-packages directory. All PYTHON* environment variables are ignored too. Further restrictions may be imposed to prevent the user from injecting malicious code. New in version 3.4. |
-O |
Turn on basic optimizations. See also PYTHONOPTIMIZE. |
-OO |
Discard docstrings in addition to the -O optimizations. |
-q |
Don’t display the copyright and version messages even in interactive mode. New in version 3.2. |
-R |
Kept for compatibility. On Python 3.3 and greater, hash randomization is turned on by default. On previous versions of Python, this option turns on hash randomization, so that the __hash__() values of str , bytes and datetime are salted with an unpredictable random value. Although they remain constant within an individual Python process, they are not predictable between repeated invocations of Python. New in version 3.2.3. Changed in version 3.7: The option is no longer ignored. |
-s |
Don’t add the user site-packages directory to sys.path, see Python FAQ and PEP 370: Per user site-packages directory. |
-S |
Disable the import of the module site and the site-dependent manipulations of sys.path that it entails. Also disable these manipulations if site is explicitly imported later (call site.main() if you want them to be triggered). |
-t |
Issue a warning when a source file mixes tabs and spaces for indentation in a way that makes it depend on the worth of a tab expressed in spaces. Issue an error when the option is given twice (-tt ). |
-u |
Force the binary layer of the stdout and stderr streams (which is available as their buffer attribute) to be unbuffered. The text I/O layer will still be line-buffered if writing to the console, or block-buffered if redirected to a non-interactive file. See also PYTHONUNBUFFERED. |
-W arg |
Warning control. Python’s warning machinery by default prints warning messages to sys.stderr. |
-x |
Skip the first line of the source, allowing use of non-Unix forms of #!cmd . This is intended for a DOS specific hack only. |
-X |
Reserved for various implementation-specific options. |
-J |
Reserved for use by Jython. You shouldn’t use this option. |
-3 |
Warn about Python 3.x possible incompatibilities by emitting a DeprecationWarning for features that are removed or significantly changed in Python 3 and can’t be detected using static code analysis. New in version 2.6. See Porting Python 2 Code to Python 3 for more details. |
args |
Arguments passed to program in sys.argv[1:]. |
Run the following command to print a short description of all command line options:
chenwx@chenwx ~ $ python -V
Python 2.7.12
chenwx@chenwx ~ $ python -h
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
-B : don't write .py[co] files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d : debug output from parser; also PYTHONDEBUG=x
-E : ignore PYTHON* environment variables (such as PYTHONPATH)
-h : print this help message and exit (also --help)
-i : inspect interactively after running script; forces a prompt even
if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-m mod : run library module as a script (terminates option list)
-O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x
-OO : remove doc-strings in addition to the -O optimizations
-R : use a pseudo-random salt to make hash() values of various types be
unpredictable between separate invocations of the interpreter, as
a defense against denial-of-service attacks
-Q arg : division options: -Qold (default), -Qwarn, -Qwarnall, -Qnew
-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S : don't imply 'import site' on initialization
-t : issue warnings about inconsistent tab usage (-tt: issue errors)
-u : unbuffered binary stdout and stderr; also PYTHONUNBUFFERED=x
see man page for details on internal buffering relating to '-u'
-v : verbose (trace import statements); also PYTHONVERBOSE=x
can be supplied multiple times to increase verbosity
-V : print the Python version number and exit (also --version)
-W arg : warning control; arg is action:message:category:module:lineno
also PYTHONWARNINGS=arg
-x : skip first line of source, allowing use of non-Unix forms of #!cmd
-3 : warn about Python 3.x incompatibilities that 2to3 cannot trivially fix
file : program read from script file
- : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]
Other environment variables:
PYTHONSTARTUP: file executed on interactive startup (no default)
PYTHONPATH : ':'-separated list of directories prefixed to the
default module search path. The result is sys.path.
PYTHONHOME : alternate <prefix> directory (or <prefix>:<exec_prefix>).
The default module search path uses <prefix>/pythonX.X.
PYTHONCASEOK : ignore case in 'import' statements (Windows).
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONHASHSEED: if this variable is set to 'random', the effect is the same
as specifying the -R option: a random value is used to seed the hashes of
str, bytes and datetime objects. It can also be set to an integer
in the range [0,4294967295] to get hash values with a predictable seed.
chenwx@chenwx:~ $ python3 -V
Python 3.6.7
chenwx@chenwx:~ $ python3 -h
usage: python3 [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
-b : issue warnings about str(bytes_instance), str(bytearray_instance)
and comparing bytes/bytearray with str. (-bb: issue errors)
-B : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d : debug output from parser; also PYTHONDEBUG=x
-E : ignore PYTHON* environment variables (such as PYTHONPATH)
-h : print this help message and exit (also --help)
-i : inspect interactively after running script; forces a prompt even
if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O : remove assert and __debug__-dependent statements; add .opt-1 before
.pyc extension; also PYTHONOPTIMIZE=x
-OO : do -O changes and also discard docstrings; add .opt-2 before
.pyc extension
-q : don't print version and copyright messages on interactive startup
-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S : don't imply 'import site' on initialization
-u : force the binary I/O layers of stdout and stderr to be unbuffered;
stdin is always buffered; text I/O layer will be line-buffered;
also PYTHONUNBUFFERED=x
-v : verbose (trace import statements); also PYTHONVERBOSE=x
can be supplied multiple times to increase verbosity
-V : print the Python version number and exit (also --version)
when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
also PYTHONWARNINGS=arg
-x : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
file : program read from script file
- : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]
Other environment variables:
PYTHONSTARTUP: file executed on interactive startup (no default)
PYTHONPATH : ':'-separated list of directories prefixed to the
default module search path. The result is sys.path.
PYTHONHOME : alternate <prefix> directory (or <prefix>:<exec_prefix>).
The default module search path uses <prefix>/lib/pythonX.X.
PYTHONCASEOK : ignore case in 'import' statements (Windows).
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONFAULTHANDLER: dump the Python traceback on fatal errors.
PYTHONHASHSEED: if this variable is set to 'random', a random value is used
to seed the hashes of str, bytes and datetime objects. It can also be
set to an integer in the range [0,4294967295] to get hash values with a
predictable seed.
PYTHONMALLOC: set the Python memory allocators and/or install debug hooks
on Python memory allocators. Use PYTHONMALLOC=debug to install debug
hooks.
Or, you can find descriptions of those options in man page:
chenwx@chenwx ~ $ man python
chenwx@chenwx ~ $ man python3
Or, run the following command to print a message each time a module is initialized, showing the place (filename or built-in module) from which it is loaded:
chenwx@chenwx ~ $ python -v
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# /usr/lib/python2.7/site.pyc matches /usr/lib/python2.7/site.py
import site # precompiled from /usr/lib/python2.7/site.pyc
# /usr/lib/python2.7/os.pyc matches /usr/lib/python2.7/os.py
import os # precompiled from /usr/lib/python2.7/os.pyc
import errno # builtin
import posix # builtin
# /usr/lib/python2.7/posixpath.pyc matches /usr/lib/python2.7/posixpath.py
import posixpath # precompiled from /usr/lib/python2.7/posixpath.pyc
# /usr/lib/python2.7/stat.pyc matches /usr/lib/python2.7/stat.py
import stat # precompiled from /usr/lib/python2.7/stat.pyc
# /usr/lib/python2.7/genericpath.pyc matches /usr/lib/python2.7/genericpath.py
import genericpath # precompiled from /usr/lib/python2.7/genericpath.pyc
# /usr/lib/python2.7/warnings.pyc matches /usr/lib/python2.7/warnings.py
import warnings # precompiled from /usr/lib/python2.7/warnings.pyc
# /usr/lib/python2.7/linecache.pyc matches /usr/lib/python2.7/linecache.py
import linecache # precompiled from /usr/lib/python2.7/linecache.pyc
# /usr/lib/python2.7/types.pyc matches /usr/lib/python2.7/types.py
import types # precompiled from /usr/lib/python2.7/types.pyc
# /usr/lib/python2.7/UserDict.pyc matches /usr/lib/python2.7/UserDict.py
import UserDict # precompiled from /usr/lib/python2.7/UserDict.pyc
# /usr/lib/python2.7/_abcoll.pyc matches /usr/lib/python2.7/_abcoll.py
import _abcoll # precompiled from /usr/lib/python2.7/_abcoll.pyc
# /usr/lib/python2.7/abc.pyc matches /usr/lib/python2.7/abc.py
import abc # precompiled from /usr/lib/python2.7/abc.pyc
# /usr/lib/python2.7/_weakrefset.pyc matches /usr/lib/python2.7/_weakrefset.py
import _weakrefset # precompiled from /usr/lib/python2.7/_weakrefset.pyc
import _weakref # builtin
# /usr/lib/python2.7/copy_reg.pyc matches /usr/lib/python2.7/copy_reg.py
import copy_reg # precompiled from /usr/lib/python2.7/copy_reg.pyc
# /usr/lib/python2.7/traceback.pyc matches /usr/lib/python2.7/traceback.py
import traceback # precompiled from /usr/lib/python2.7/traceback.pyc
# /usr/lib/python2.7/sysconfig.pyc matches /usr/lib/python2.7/sysconfig.py
import sysconfig # precompiled from /usr/lib/python2.7/sysconfig.pyc
# /usr/lib/python2.7/re.pyc matches /usr/lib/python2.7/re.py
import re # precompiled from /usr/lib/python2.7/re.pyc
# /usr/lib/python2.7/sre_compile.pyc matches /usr/lib/python2.7/sre_compile.py
import sre_compile # precompiled from /usr/lib/python2.7/sre_compile.pyc
import _sre # builtin
# /usr/lib/python2.7/sre_parse.pyc matches /usr/lib/python2.7/sre_parse.py
import sre_parse # precompiled from /usr/lib/python2.7/sre_parse.pyc
# /usr/lib/python2.7/sre_constants.pyc matches /usr/lib/python2.7/sre_constants.py
import sre_constants # precompiled from /usr/lib/python2.7/sre_constants.pyc
import _locale # builtin
# /usr/lib/python2.7/_sysconfigdata.pyc matches /usr/lib/python2.7/_sysconfigdata.py
import _sysconfigdata # precompiled from /usr/lib/python2.7/_sysconfigdata.pyc
# /usr/lib/python2.7/plat-x86_64-linux-gnu/_sysconfigdata_nd.pyc matches /usr/lib/python2.7/plat-x86_64-linux-gnu/_sysconfigdata_nd.py
import _sysconfigdata_nd # precompiled from /usr/lib/python2.7/plat-x86_64-linux-gnu/_sysconfigdata_nd.pyc
# /usr/lib/python2.7/sitecustomize.pyc matches /usr/lib/python2.7/sitecustomize.py
import sitecustomize # precompiled from /usr/lib/python2.7/sitecustomize.pyc
import encodings # directory /usr/lib/python2.7/encodings
# /usr/lib/python2.7/encodings/__init__.pyc matches /usr/lib/python2.7/encodings/__init__.py
import encodings # precompiled from /usr/lib/python2.7/encodings/__init__.pyc
# /usr/lib/python2.7/codecs.pyc matches /usr/lib/python2.7/codecs.py
import codecs # precompiled from /usr/lib/python2.7/codecs.pyc
import _codecs # builtin
# /usr/lib/python2.7/encodings/aliases.pyc matches /usr/lib/python2.7/encodings/aliases.py
import encodings.aliases # precompiled from /usr/lib/python2.7/encodings/aliases.pyc
# /usr/lib/python2.7/encodings/ascii.pyc matches /usr/lib/python2.7/encodings/ascii.py
import encodings.ascii # precompiled from /usr/lib/python2.7/encodings/ascii.pyc
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
dlopen("/usr/lib/python2.7/lib-dynload/readline.x86_64-linux-gnu.so", 2);
import readline # dynamically loaded from /usr/lib/python2.7/lib-dynload/readline.x86_64-linux-gnu.so
>>> exit()
# clear __builtin__._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.exitfunc
# clear sys.exc_type
# clear sys.exc_value
# clear sys.exc_traceback
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup __main__
# cleanup[1] encodings
# cleanup[1] site
# cleanup[1] sysconfig
# cleanup[1] abc
# cleanup[1] _weakrefset
# cleanup[1] sre_constants
# cleanup[1] re
# cleanup[1] _codecs
# cleanup[1] _warnings
# cleanup[1] zipimport
# cleanup[1] _sysconfigdata
# cleanup[1] encodings.ascii
# cleanup[1] codecs
# cleanup[1] readline
# cleanup[1] _sysconfigdata_nd
# cleanup[1] _locale
# cleanup[1] sitecustomize
# cleanup[1] signal
# cleanup[1] traceback
# cleanup[1] posix
# cleanup[1] encodings.aliases
# cleanup[1] exceptions
# cleanup[1] _weakref
# cleanup[1] sre_compile
# cleanup[1] _sre
# cleanup[1] sre_parse
# cleanup[2] copy_reg
# cleanup[2] posixpath
# cleanup[2] errno
# cleanup[2] _abcoll
# cleanup[2] types
# cleanup[2] genericpath
# cleanup[2] stat
# cleanup[2] warnings
# cleanup[2] UserDict
# cleanup[2] os.path
# cleanup[2] linecache
# cleanup[2] os
# cleanup sys
# cleanup __builtin__
# cleanup ints: 19 unfreed ints
# cleanup floats
chenwx@chenwx ~ $ python3 -v
import _frozen_importlib # frozen
import _imp # builtin
import sys # builtin
import '_warnings' # <class '_frozen_importlib.BuiltinImporter'>
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'>
import '_io' # <class '_frozen_importlib.BuiltinImporter'>
import 'marshal' # <class '_frozen_importlib.BuiltinImporter'>
import 'posix' # <class '_frozen_importlib.BuiltinImporter'>
import _thread # previously loaded ('_thread')
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import _weakref # previously loaded ('_weakref')
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
# installing zipimport hook
import 'zipimport' # <class '_frozen_importlib.BuiltinImporter'>
# installed zipimport hook
# /usr/lib/python3.5/encodings/__pycache__/__init__.cpython-35.pyc matches /usr/lib/python3.5/encodings/__init__.py
# code object from '/usr/lib/python3.5/encodings/__pycache__/__init__.cpython-35.pyc'
# /usr/lib/python3.5/__pycache__/codecs.cpython-35.pyc matches /usr/lib/python3.5/codecs.py
# code object from '/usr/lib/python3.5/__pycache__/codecs.cpython-35.pyc'
import '_codecs' # <class '_frozen_importlib.BuiltinImporter'>
import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8456d128>
# /usr/lib/python3.5/encodings/__pycache__/aliases.cpython-35.pyc matches /usr/lib/python3.5/encodings/aliases.py
# code object from '/usr/lib/python3.5/encodings/__pycache__/aliases.cpython-35.pyc'
import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d844fcbe0>
import 'encodings' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d84563cf8>
# /usr/lib/python3.5/encodings/__pycache__/ascii.cpython-35.pyc matches /usr/lib/python3.5/encodings/ascii.py
# code object from '/usr/lib/python3.5/encodings/__pycache__/ascii.cpython-35.pyc'
import 'encodings.ascii' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8450aa90>
import '_signal' # <class '_frozen_importlib.BuiltinImporter'>
# /usr/lib/python3.5/encodings/__pycache__/utf_8.cpython-35.pyc matches /usr/lib/python3.5/encodings/utf_8.py
# code object from '/usr/lib/python3.5/encodings/__pycache__/utf_8.cpython-35.pyc'
import 'encodings.utf_8' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8450d588>
# /usr/lib/python3.5/encodings/__pycache__/latin_1.cpython-35.pyc matches /usr/lib/python3.5/encodings/latin_1.py
# code object from '/usr/lib/python3.5/encodings/__pycache__/latin_1.cpython-35.pyc'
import 'encodings.latin_1' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8450d828>
# /usr/lib/python3.5/__pycache__/io.cpython-35.pyc matches /usr/lib/python3.5/io.py
# code object from '/usr/lib/python3.5/__pycache__/io.cpython-35.pyc'
# /usr/lib/python3.5/__pycache__/abc.cpython-35.pyc matches /usr/lib/python3.5/abc.py
# code object from '/usr/lib/python3.5/__pycache__/abc.cpython-35.pyc'
# /usr/lib/python3.5/__pycache__/_weakrefset.cpython-35.pyc matches /usr/lib/python3.5/_weakrefset.py
# code object from '/usr/lib/python3.5/__pycache__/_weakrefset.cpython-35.pyc'
import '_weakrefset' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d845124e0>
import 'abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8450dcc0>
import 'io' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8450da58>
# /usr/lib/python3.5/__pycache__/site.cpython-35.pyc matches /usr/lib/python3.5/site.py
# code object from '/usr/lib/python3.5/__pycache__/site.cpython-35.pyc'
# /usr/lib/python3.5/__pycache__/os.cpython-35.pyc matches /usr/lib/python3.5/os.py
# code object from '/usr/lib/python3.5/__pycache__/os.cpython-35.pyc'
import 'errno' # <class '_frozen_importlib.BuiltinImporter'>
# /usr/lib/python3.5/__pycache__/stat.cpython-35.pyc matches /usr/lib/python3.5/stat.py
# code object from '/usr/lib/python3.5/__pycache__/stat.cpython-35.pyc'
import '_stat' # <class '_frozen_importlib.BuiltinImporter'>
import 'stat' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d844b89e8>
# /usr/lib/python3.5/__pycache__/posixpath.cpython-35.pyc matches /usr/lib/python3.5/posixpath.py
# code object from '/usr/lib/python3.5/__pycache__/posixpath.cpython-35.pyc'
# /usr/lib/python3.5/__pycache__/genericpath.cpython-35.pyc matches /usr/lib/python3.5/genericpath.py
# code object from '/usr/lib/python3.5/__pycache__/genericpath.cpython-35.pyc'
import 'genericpath' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d844c1278>
import 'posixpath' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d844b8d30>
# /usr/lib/python3.5/__pycache__/_collections_abc.cpython-35.pyc matches /usr/lib/python3.5/_collections_abc.py
# code object from '/usr/lib/python3.5/__pycache__/_collections_abc.cpython-35.pyc'
import '_collections_abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d844c1908>
import 'os' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d845271d0>
# /usr/lib/python3.5/__pycache__/_sitebuiltins.cpython-35.pyc matches /usr/lib/python3.5/_sitebuiltins.py
# code object from '/usr/lib/python3.5/__pycache__/_sitebuiltins.cpython-35.pyc'
import '_sitebuiltins' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d845274a8>
# /usr/lib/python3.5/__pycache__/sysconfig.cpython-35.pyc matches /usr/lib/python3.5/sysconfig.py
# code object from '/usr/lib/python3.5/__pycache__/sysconfig.cpython-35.pyc'
import 'sysconfig' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8447c0b8>
# /usr/lib/python3.5/__pycache__/_sysconfigdata.cpython-35.pyc matches /usr/lib/python3.5/_sysconfigdata.py
# code object from '/usr/lib/python3.5/__pycache__/_sysconfigdata.cpython-35.pyc'
# /usr/lib/python3.5/plat-x86_64-linux-gnu/__pycache__/_sysconfigdata_m.cpython-35.pyc matches /usr/lib/python3.5/plat-x86_64-linux-gnu/_sysconfigdata_m.py
# code object from '/usr/lib/python3.5/plat-x86_64-linux-gnu/__pycache__/_sysconfigdata_m.cpython-35.pyc'
import '_sysconfigdata_m' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d84481b70>
import '_sysconfigdata' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d84481978>
# /usr/lib/python3.5/__pycache__/sitecustomize.cpython-35.pyc matches /usr/lib/python3.5/sitecustomize.py
# code object from '/usr/lib/python3.5/__pycache__/sitecustomize.cpython-35.pyc'
import 'sitecustomize' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8448fcf8>
import 'site' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d84519cc0>
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
# extension module 'readline' loaded from '/usr/lib/python3.5/lib-dynload/readline.cpython-35m-x86_64-linux-gnu.so'
# extension module 'readline' executed from '/usr/lib/python3.5/lib-dynload/readline.cpython-35m-x86_64-linux-gnu.so'
import 'readline' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7f3d8449c208>
import 'atexit' # <class '_frozen_importlib.BuiltinImporter'>
# /usr/lib/python3.5/__pycache__/rlcompleter.cpython-35.pyc matches /usr/lib/python3.5/rlcompleter.py
# code object from '/usr/lib/python3.5/__pycache__/rlcompleter.cpython-35.pyc'
import 'rlcompleter' # <_frozen_importlib_external.SourceFileLoader object at 0x7f3d8449c320>
>>> exit()
# clear builtins._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.__interactivehook__
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup[2] removing encodings.utf_8
# cleanup[2] removing codecs
# cleanup[2] removing encodings.ascii
# cleanup[2] removing os
# cleanup[2] removing site
# destroy site
# cleanup[2] removing posixpath
# cleanup[2] removing encodings.latin_1
# cleanup[2] removing builtins
# cleanup[2] removing _codecs
# cleanup[2] removing _sysconfigdata_m
# destroy _sysconfigdata_m
# cleanup[2] removing io
# destroy io
# cleanup[2] removing _weakrefset
# destroy _weakrefset
# cleanup[2] removing sysconfig
# destroy sysconfig
# cleanup[2] removing _stat
# cleanup[2] removing zipimport
# cleanup[2] removing readline
# cleanup[2] removing sitecustomize
# destroy sitecustomize
# cleanup[2] removing _imp
# cleanup[2] removing marshal
# cleanup[2] removing stat
# cleanup[2] removing _io
# cleanup[2] removing atexit
# cleanup[2] removing genericpath
# cleanup[2] removing _warnings
# cleanup[2] removing _weakref
# cleanup[2] removing os.path
# cleanup[2] removing errno
# cleanup[2] removing _sysconfigdata
# destroy _sysconfigdata
# cleanup[2] removing _frozen_importlib_external
# cleanup[2] removing sys
# cleanup[2] removing __main__
# cleanup[2] removing encodings.aliases
# cleanup[2] removing _thread
# cleanup[2] removing _frozen_importlib
# cleanup[2] removing rlcompleter
# destroy rlcompleter
# cleanup[2] removing encodings
# destroy encodings
# cleanup[2] removing _signal
# cleanup[2] removing abc
# destroy abc
# cleanup[2] removing _sitebuiltins
# cleanup[2] removing _collections_abc
# destroy _collections_abc
# cleanup[2] removing posix
# destroy zipimport
# destroy _signal
# destroy _sitebuiltins
# destroy posixpath
# destroy errno
# destroy _stat
# destroy genericpath
# destroy stat
# destroy os
# destroy __main__
# destroy readline
# destroy atexit
# cleanup[3] wiping encodings.utf_8
# cleanup[3] wiping codecs
# cleanup[3] wiping encodings.ascii
# cleanup[3] wiping encodings.latin_1
# cleanup[3] wiping _codecs
# cleanup[3] wiping _imp
# cleanup[3] wiping marshal
# cleanup[3] wiping _io
# cleanup[3] wiping _warnings
# cleanup[3] wiping _weakref
# cleanup[3] wiping _frozen_importlib_external
# destroy io
# destroy marshal
# cleanup[3] wiping encodings.aliases
# cleanup[3] wiping _thread
# cleanup[3] wiping _frozen_importlib
# destroy _weakref
# destroy _thread
# destroy _imp
# destroy _warnings
# destroy _frozen_importlib_external
# cleanup[3] wiping posix
# destroy posix
# cleanup[3] wiping sys
# cleanup[3] wiping builtins
# destroy _frozen_importlib
Also you can run command python -vv
or python3 -vv
to print a message for each file that is checked for when searching for a module.
Environment Variables
The environment variables influence Python’s behavior, they are processed before the command-line switches other than -E
or -I
. It is customary that command-line switches override environmental variables where there is a conflict. Refer to command line and environment for details.
-
PYTHONHOME
Change the location of the standard Python libraries. By default, the libraries are searched in prefix/lib/pythonversion and exec_prefix/lib/pythonversion, where prefix and exec_prefix are installation-dependent directories, both defaulting to /usr/local.
When PYTHONHOME is set to a single directory, its value replaces both prefix and exec_prefix. To specify different values for these, set PYTHONHOME to prefix:exec_prefix.
-
PYTHONPATH
Augment the default search path for module files. The format is the same as the shell’s PATH: one or more directory pathnames separated by
os.pathsep
, e.g. colons on Unix or semicolons on Windows. Non-existent directories are silently ignored.In addition to normal directories, individual PYTHONPATH entries may refer to zipfiles containing pure Python modules (in either source or compiled form). Extension modules cannot be imported from zipfiles.
The default search path is installation dependent, but generally begins with prefix/lib/pythonversion, see PYTHONHOME above. It is always appended to PYTHONPATH.
An additional directory will be inserted in the search path in front of PYTHONPATH as described above under Interface options. The search path can be manipulated from within a Python program as the variable
sys.path
. -
PYTHONSTARTUP
If this is the name of a readable file, the Python commands in that file are executed before the first prompt is displayed in interactive mode. The file is executed in the same namespace where interactive commands are executed so that objects defined or imported in it can be used without qualification in the interactive session. You can also change the prompts
sys.ps1
andsys.ps2
and the hooksys.__interactivehook__
in this file. -
PYTHONOPTIMIZE
If this is set to a non-empty string it is equivalent to specifying the
-O
option. If set to an integer, it is equivalent to specifying-O
multiple times. -
PYTHONDEBUG
If this is set to a non-empty string it is equivalent to specifying the
-d
option. If set to an integer, it is equivalent to specifying-d
multiple times. -
PYTHONINSPECT
If this is set to a non-empty string it is equivalent to specifying the
-i
option. This variable can also be modified by Python code usingos.environ
to force inspect mode on program termination. -
PYTHONUNBUFFERED
If this is set to a non-empty string it is equivalent to specifying the
-u
option. -
PYTHONVERBOSE
If this is set to a non-empty string it is equivalent to specifying the
-v
option. If set to an integer, it is equivalent to specifying-v
multiple times. -
PYTHONCASEOK
If this is set, Python ignores case in
import
statements. This only works on Windows and OS X. -
PYTHONDONTWRITEBYTECODE
If this is set to a non-empty string, Python won’t try to write .pyc files on the import of source modules. This is equivalent to specifying the
-B
option. -
PYTHONHASHSEED
If this variable is not set or set to
random
, a random value is used to seed the hashes of str, bytes and datetime objects.New in version 3.2.3.
-
PYTHONIOENCODING
If this is set before running the interpreter, it overrides the encoding used for stdin, stdout, stderr, in the syntax encodingname:errorhandler. Both the encodingname and the :errorhandler parts are optional and have the same meaning as in
str.encode()
. -
PYTHONNOUSERSITE
If this is set, Python won’t add the user site-packages directory to
sys.path
, refer to PEP 370: Per user site-packages directory. -
PYTHONUSERBASE
Defines the user base directory, which is used to compute the path of the user site-packages directory and Distutils installation paths for
python setup.py install --user
. -
PYTHONEXECUTABLE
If this environment variable is set,
sys.argv[0]
will be set to its value instead of the value got through the C runtime. Only works on Mac OS X. -
PYTHONWARNINGS
This is equivalent to the
-W
option. If set to a comma separated string, it is equivalent to specifying-W
multiple times. -
PYTHONFAULTHANDLER
If this environment variable is set to a non-empty string,
faulthandler.enable()
is called at startup: install a handler for SIGSEGV, SIGFPE, SIGABRT, SIGBUS and SIGILL signals to dump the Python traceback. This is equivalent to-X faulthandler
option.New in version 3.3.
-
PYTHONTRACEMALLOC
If this environment variable is set to a non-empty string, start tracing Python memory allocations using the tracemalloc module. The value of the variable is the maximum number of frames stored in a traceback of a trace. For example,
PYTHONTRACEMALLOC=1
stores only the most recent frame. See thetracemalloc.start()
for more information.New in version 3.4.
-
PYTHONASYNCIODEBUG
If this environment variable is set to a non-empty string, enable the debug mode of the asyncio module.
New in version 3.4.
-
PYTHONMALLOC
Set the Python memory allocators and/or install debug hooks.
New in version 3.6.
-
PYTHONMALLOCSTATS
If set to a non-empty string, Python will print statistics of the pymalloc memory allocator every time a new pymalloc object arena is created, and on shutdown.
-
PYTHONLEGACYWINDOWSFSENCODING
If set to a non-empty string, the default filesystem encoding and errors mode will revert to their pre-3.6 values of mbcs and replace, respectively. Otherwise, the new defaults utf-8 and surrogatepass are used. This may also be enabled at runtime with
sys._enablelegacywindowsfsencoding()
.Availability: Windows
-
PYTHONLEGACYWINDOWSIOENCODING
If set to a non-empty string, does not use the new console reader and writer. This means that Unicode characters will be encoded according to the active console code page, rather than using utf-8. This variable is ignored if the standard streams are redirected (to files or pipes) rather than referring to console buffers.
Availability: Windows
New in version 3.6.
-
PYTHONTHREADDEBUG
Setting this variable only has an effect in a debug build of Python, that is, if Python was configured with the
--with-pydebug
build option.If set, Python will print threading debug info.
-
PYTHONDUMPREFS
Setting this variable only has an effect in a debug build of Python, that is, if Python was configured with the
--with-pydebug
build option.If set, Python will dump objects and reference counts still alive after shutting down the interpreter.
Python Language
Python documentation:
For C or C++ programmers, two additional manuals exist:
- Extending and Embedding the Python Interpreter describes the high-level picture of how to write a Python extension module
- Python/C API Reference Manual describes the interfaces available to C/C++ programmers in detail
And you can find the index of all functions, classes, terms on:
Keywords
The following identifiers are used as reserved words, or keywords of the language, and cannot be used as ordinary identifiers. They must be spelled exactly as written here. More information can be found at Simple statements and Compound statements in Python 3.x Language Reference.
False class finally is return
None continue for lambda try
True def from nonlocal while
and del global not with
as elif if or yield
assert else import pass
break except in raise
The keyword module allows a Python program to determine if a string is a keyword. Its source code is located in Lib/keyword.py:
chenwx@chenwx ~/cpython $ ll Lib/keyword.py
-rwxrwxr-x 1 chenwx chenwx 2.2K Mar 26 18:40 Lib/keyword.py
chenwx@chenwx ~/cpython $ python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import keyword
>>> keyword.kwlist
['False', 'None', 'True', 'and', 'as', 'assert', 'break', 'class', 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', 'not', 'or', 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield']
>>> print(*keyword.kwlist)
False None True and as assert break class continue def del elif else except finally for from global if import in is lambda nonlocal not or pass raise return try while with yield
>>> keyword.iskeyword("for")
True
Operators
The following tokens are operators:
+ - * ** / // % @
<< >> & | ^ ~
< > <= >= == !=
Python operator precedence:
The operator module exports a set of efficient functions corresponding to the intrinsic operators of Python. Its source code is located in Lib/operator.py:
chenwx@chenwx ~/cpython $ python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import operator
>>> dir(operator)
['__abs__', '__add__', '__all__', '__and__', '__builtins__', '__cached__', '__concat__', '__contains__', '__delitem__', '__doc__', '__eq__', '__file__', '__floordiv__', '__ge__', '__getitem__', '__gt__', '__iadd__', '__iand__', '__iconcat__', '__ifloordiv__', '__ilshift__', '__imatmul__', '__imod__', '__imul__', '__index__', '__inv__', '__invert__', '__ior__', '__ipow__', '__irshift__', '__isub__', '__itruediv__', '__ixor__', '__le__', '__loader__', '__lshift__', '__lt__', '__matmul__', '__mod__', '__mul__', '__name__', '__ne__', '__neg__', '__not__', '__or__', '__package__', '__pos__', '__pow__', '__rshift__', '__setitem__', '__spec__', '__sub__', '__truediv__', '__xor__', '_abs', 'abs', 'add', 'and_', 'attrgetter', 'concat', 'contains', 'countOf', 'delitem', 'eq', 'floordiv', 'ge', 'getitem', 'gt', 'iadd', 'iand', 'iconcat', 'ifloordiv', 'ilshift', 'imatmul', 'imod', 'imul', 'index', 'indexOf', 'inv', 'invert', 'ior', 'ipow', 'irshift', 'is_', 'is_not', 'isub', 'itemgetter', 'itruediv', 'ixor', 'le', 'length_hint', 'lshift', 'lt', 'matmul', 'methodcaller', 'mod', 'mul', 'ne', 'neg', 'not_', 'or_', 'pos', 'pow', 'rshift', 'setitem', 'sub', 'truediv', 'truth', 'xor']
>>> print(*dir(operator))
__abs__ __add__ __all__ __and__ __builtins__ __cached__ __concat__ __contains__ __delitem__ __doc__ __eq__ __file__ __floordiv__ __ge__ __getitem__ __gt__ __iadd__ __iand__ __iconcat__ __ifloordiv__ __ilshift__ __imatmul__ __imod__ __imul__ __index__ __inv__ __invert__ __ior__ __ipow__ __irshift__ __isub__ __itruediv__ __ixor__ __le__ __loader__ __lshift__ __lt__ __matmul__ __mod__ __mul__ __name__ __ne__ __neg__ __not__ __or__ __package__ __pos__ __pow__ __rshift__ __setitem__ __spec__ __sub__ __truediv__ __xor__ _abs abs add and_ attrgetter concat contains countOf delitem eq floordiv ge getitem gt iadd iand iconcat ifloordiv ilshift imatmul imod imul index indexOf inv invert ior ipow irshift is_ is_not isub itemgetter itruediv ixor le length_hint lshift lt matmul methodcaller mod mul ne neg not_ or_ pos pow rshift setitem sub truediv truth xor
Delimiters
The following tokens serve as delimiters in the grammar:
( ) [ ] { }
, : . ; @ = ->
+= -= *= /= //= %= @=
&= |= ^= >>= <<= **=
The period (.
) can also occur in floating-point and imaginary literals. A sequence of three periods (...
) has a special meaning as an ellipsis literal. The second half of the list, the augmented assignment operators, serve lexically as delimiters, but also perform an operation.
The following printing ASCII characters have special meaning as part of other tokens or are otherwise significant to the lexical analyzer:
' " # \
The following printing ASCII characters are not used in Python. Their occurrence outside string literals and comments is an unconditional error:
$ ? `
Comments
One Line Comments
Typically, you just use the hash character (#
) to comment out everything that follows it on that line. A comment may appear at the start of a line or following whitespace or code, but not within a string literal. A hash character (#
) within a string literal is just a hash character.
# this is the first comment
spam = 1 # and this is the second comment
# ... and now a third!
text = "# This is not a comment because it's inside quotes."
Multiple Line Comments
Multiple line comments simply use 3 single quotes ('''
) before and after the part you want commented:
'''
print("We are in a comment")
print("We are still in a comment")
'''
print("We are out of the comment")
Encoding Declaration
If a comment in the first or second line of the Python script matches the regular expression (see PEP 263: Defining Python Source Code Encodings):
^[ \t\v]*#.*?coding[:=][ \t]*([-_.a-zA-Z0-9]+)
or, the regular expression (see encoding declarations):
coding[=:]\s*([-\w.]+)
this comment is processed as an encoding declaration. The first group of this expression names the encoding of the source code file. If the encoding is unknown to Python, an error is raised during compilation. More supported encoding can be found at:
- Python 2.x Standard Encodings and Python Specific Encodings
- Python 3.x Standard Encodings and Python Specific Encodings
There must not be any Python statement on the line that contains the encoding declaration, that’s the encoding declaration must appear on a line of its own. If the first line matches the second line is ignored; if it is the second line, the first line must also be a comment-only line, for instance:
#!/usr/bin/python
# -*- coding: <encoding-name> -*-
or,
#!/usr/bin/python
# vim: set fileencoding=<encoding name> :
The recommended forms of an encoding expression are
# -*- coding: <encoding-name> -*-
which is recognized also by GNU Emacs, and
# vim: set fileencoding=<encoding name> :
which is recognized by Bram Moolenaar’s VIM.
If no encoding declaration is found, the default encoding is UTF-8, refer to PEP 3120: Using UTF-8 as the default source encoding. If an encoding is declared, the encoding name must be recognized by Python. The encoding is used for all lexical analysis, including string literals, comments and identifiers. Refer to PEP 263: Defining Python Source Code Encodings.
Indentation
Indentation is Python’s way of grouping statements. Each line within a basic block must be indented by the same amount.
Leading whitespace (spaces and tabs) at the beginning of a logical line is used to compute the indentation level of the line, which in turn is used to determine the grouping of statements.
Tabs are replaced (from left to right) by one to eight spaces such that the total number of characters up to and including the replacement is a multiple of eight (this is intended to be the same rule as used by Unix). The total number of spaces preceding the first non-blank character then determines the line’s indentation. Indentation cannot be split over multiple physical lines using backslashes; the whitespace up to the first backslash determines the indentation.
Indentation is rejected as inconsistent if a source file mixes tabs and spaces in a way that makes the meaning dependent on the worth of a tab in spaces; a TabError is raised in that case.
Tab or spaces?
According to PEP 8: Style Guide for Python Code, spaces are the preferred indentation method. Tabs should be used solely to remain consistent with code that is already indented with tabs.
Python 3 disallows mixing the use of tabs and spaces for indentation. Python 2 code indented with a mixture of tabs and spaces should be converted to using spaces exclusively. When invoking the Python 2 command line interpreter with the -t
option, it issues warnings about code that illegally mixes tabs and spaces. When using -tt
these warnings become errors. These options are highly recommended!
Data Structures
Lists
Run the following command to check help information of list:
>>> help(list)
Create a list:
>>> squares = []
>>> for x in range(10):
... squares.append(x**2)
...
>>> squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> type(squares)
<class 'list'>
or,
>>> squares = [x**2 for x in range(10)]
>>> squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
or,
>>> squares = list(map(lambda x: x**2, range(10)))
>>> squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Lists are mutable:
>>> squares[0] = 10
>>> squares[1] = 11
>>> squares
[10, 11, 4, 9, 16, 25, 36, 49, 64, 81]
There is a way to remove an item from a list given its index instead of its value: the del
statement. This differs from the pop()
method which returns a value. The del
statement can also be used to remove slices from a list or clear the entire list.
>>> squares.pop()
81
>>> squares
[10, 11, 4, 9, 16, 25, 36, 49, 64]
>>> squares.pop()
64
>>> squares
[10, 11, 4, 9, 16, 25, 36, 49]
>>> del squares[0]
>>> squares
[11, 4, 9, 16, 25, 36, 49]
>>> del squares[2:4]
>>> squares
[11, 4, 25, 36, 49]
>>> del squares[:]
>>> squares
[]
The del
statement can also be used to delete entire variables:
>>> squares = [x**2 for x in range(10)]
>>> squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> del squares
>>> squares
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'squares' is not defined
Referencing the name squares
hereafter is an error, at least until another value is assigned to it.
Tuples
Run the following command to check help information of tuple:
>>> help(tuple)
A tuple consists of a number of values separated by commas:
>>> t = (12345, 54321, 'hello!') # or, t = 12345, 54321, 'hello!'
>>> t
(12345, 54321, 'hello!')
>>> t[0]
12345
>>> type(t)
<class 'tuple'>
Tuples may be nested:
>>> u = (t, (1, 2, 3, 4, 5)) # or, u = t, (1, 2, 3, 4, 5)
>>> u
((12345, 54321, 'hello!'), (1, 2, 3, 4, 5))
Tuples are immutable. It is not possible to assign to the individual items of a tuple, however it is possible to create tuples which contain mutable objects, such as lists:
>>> v = ([1, 2, 3], [3, 2, 1]) # or, v = [1, 2, 3], [3, 2, 1]
>>> v
([1, 2, 3], [3, 2, 1])
>>> v[0] = [11, 22, 33]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> v[0][1] = 22
>>> v
([1, 22, 3], [3, 2, 1])
A special problem is the construction of tuples containing 0 or 1 items: the syntax has some extra quirks to accommodate these. Empty tuples are constructed by an empty pair of parentheses; a tuple with one item is constructed by following a value with a comma (it is not sufficient to enclose a single value in parentheses). Ugly, but effective.
>>> empty = ()
>>> len(empty)
0
>>> empty
()
>>> singleton = ('hello',) # or, singleton = 'hello', <-- note trailing comma
>>> len(singleton)
1
>>> singleton
('hello',)
This is sequence unpacking and works for any sequence on the right-hand side. Sequence unpacking requires that there are as many variables on the left side of the equals sign as there are elements in the sequence. Note that multiple assignment is really just a combination of tuple packing and sequence unpacking.
>>> t = (12345, 54321, 'hello!') # or, t = 12345, 54321, 'hello!'
>>> t
(12345, 54321, 'hello!')
>>> x, y, z = t
>>> x
12345
>>> y
54321
>>> z
'hello!'
Sets
Run the following command to check help information of set:
>>> help(set)
A set is an unordered collection with no duplicate elements. Basic uses include membership testing and eliminating duplicate entries. Set objects also support mathematical operations like union, intersection, difference, and symmetric difference.
Curly braces ({}
) or the set()
function can be used to create sets. Note that in order to create an empty set you have to use set()
, not {}
; the latter creates an empty dictionary.
>>> basket = {'apple', 'orange', 'apple', 'pear', 'orange', 'banana'}
>>> print(basket) # show that duplicates have been removed
{'orange', 'banana', 'pear', 'apple'}
>>> basket
{'orange', 'pear', 'banana', 'apple'}
>>> type(basket)
<class 'set'>
>>> 'orange' in basket # fast membership testing
True
>>> 'crabgrass' in basket
False
Demonstrate set()
operations on unique letters from two words:
>>> a = set('abracadabra')
>>> a # unique letters in a
{'a', 'r', 'b', 'c', 'd'}
>>> b = set('alacazam')
>>> b # unique letters in b
{'z', 'a', 'm', 'c', 'l'}
>>> a - b # letters in a but not in b
{'r', 'd', 'b'}
>>> a | b # letters in either a or b
{'a', 'c', 'r', 'd', 'b', 'm', 'z', 'l'}
>>> a & b # letters in both a and b
{'a', 'c'}
>>> a ^ b # letters in a or b but not both
{'r', 'd', 'b', 'm', 'z', 'l'}
Dictionaries
Run the following command to check help information of dict:
>>> help(dict)
Unlike sequences, which are indexed by a range of numbers, dictionaries are indexed by keys, which can be any immutable type; strings and numbers can always be keys.
It is best to think of a dictionary as an unordered set of key:value pairs, with the requirement that the keys are unique (within one dictionary). A pair of braces creates an empty dictionary: {}
. Placing a comma-separated list of key:value pairs within the braces adds initial key:value pairs to the dictionary; this is also the way dictionaries are written on output. If you store using a key that is already in use, the old value associated with that key is forgotten.
>>> tel = {}
>>> tel
{}
>>> type(tel)
<class 'dict'>
>>> tel = {'jack': 4098, 'sape': 4140}
>>> tel
{'sape': 4140, 'jack': 4098}
>>> tel = {'jack': 4098, 'sape': 4139, 'sape': 4141}
>>> tel
{'sape': 4141, 'jack': 4098}
The main operations on a dictionary are storing a value with some key and extracting the value given the key. It is also possible to delete a key:value pair with del
statement. It is an error to extract a value using a non-existent key. To check whether a single key is in the dictionary, use the in
keyword.
>>> tel = {'jack': 4098, 'sape': 4139}
>>> tel
{'sape': 4139, 'jack': 4098}
>>> tel['guido'] = 4127
>>> tel
{'sape': 4139, 'guido': 4127, 'jack': 4098}
>>> tel['jack']
4098
>>> del tel['sape']
{'guido': 4127, 'jack': 4098}
>>> tel['irv'] = 4127
>>> tel
{'guido': 4127, 'irv': 4127, 'jack': 4098}
>>> list(tel.keys())
['irv', 'guido', 'jack']
>>> sorted(tel.keys())
['guido', 'irv', 'jack']
>>> 'guido' in tel
True
>>> 'jack' not in tel
False
The dict()
constructor builds dictionaries directly from sequences of key:value pairs:
>>> dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])
{'sape': 4139, 'jack': 4098, 'guido': 4127}
In addition, dict comprehensions can be used to create dictionaries from arbitrary key and value expressions:
>>> {x: x**2 for x in (2, 4, 6)}
{2: 4, 4: 16, 6: 36}
When the keys are simple strings, it is sometimes easier to specify pairs using keyword arguments:
>>> dict(sape=4139, guido=4127, jack=4098)
{'sape': 4139, 'jack': 4098, 'guido': 4127}
When looping through dictionaries, the key and corresponding value can be retrieved at the same time using the items()
method.
>>> knights = {'gallahad': 'the pure', 'robin': 'the brave'}
>>> for k, v in knights.items():
... print(k, v)
...
gallahad the pure
robin the brave
Statements
Multiple Assignment
The expressions on the right-hand side are all evaluated first before any of the assignments take place. The right-hand side expressions are evaluated from the left to the right.
# Fibonacci series: the sum of two elements defines the next
a, b = 0, 1
while (b < 10)
print(b)
a, b = b, a+b
assert
The assert
statements are a convenient way to insert debugging assertions into a program:
assert <expression> [, <expression> ]
The simple form, assert <expression>
, is equivalent to
if __debug__:
if not <expression>: raise AssertionError
The extended form, assert <expression1>, <expression2>
, is equivalent to
if __debug__:
if not <expression1>: raise AssertionError(<expression2>)
These equivalences assume that __debug__
and AssertionError
refer to the built-in variables with those names. In the current implementation, the built-in variable __debug__
is True under normal circumstances, False when optimization is requested (command line option -O
). The current code generator emits no code for an assert statement when optimization is requested at compile time. Note that it is unnecessary to include the source code for the expression that failed in the error message; it will be displayed as part of the stack trace.
Assignments to __debug__
are illegal. The value for the built-in variable is determined when the interpreter starts.
pass
The pass
statement does nothing. It can be used when a statement is required syntactically but the program requires no action.
>>> def initlog(*args):
... pass # Remember to implement this!
...
del
del <target_list>
Deletion is recursively defined very similar to the way assignment is defined. Rather than spelling it out in full details, here are some hints.
Deletion of a target list recursively deletes each target, from left to right.
Deletion of a name removes the binding of that name from the local or global namespace, depending on whether the name occurs in a global statement in the same code block. If the name is unbound, a NameError exception will be raised.
Deletion of attribute references, subscriptions and slicings is passed to the primary object involved; deletion of a slicing is in general equivalent to assignment of an empty slice of the right type (but even this is determined by the sliced object).
Changed in version 3.2: Previously it was illegal to delete a name from the local namespace if it occurs as a free variable in a nested block.
return
return <expression_list>
The return
statement may only occur syntactically nested in a function definition, not within a nested class definition.
If an expression_list is present, it is evaluated, else None is substituted.
The return
statement leaves the current function call with the expression_list or None as return value.
When return
passes control out of a try
statement with a finally
clause, that finally
clause is executed before really leaving the function.
In a generator function, the return
statement indicates that the generator is done and will cause StopIteration to be raised. The returned value (if any) is used as an argument to construct StopIteration and becomes the StopIteration.value attribute.
In an asynchronous generator function, an empty return statement indicates that the asynchronous generator is done and will cause **StopAsyncIteration×× to be raised. A non-empty return statement is a syntax error in an asynchronous generator function.
yield
A yield
statement is semantically equivalent to a yield expression. The yield
statement can be used to omit the parentheses that would otherwise be required in the equivalent yield expression statement. For example, the yield statements
yield <expr>
yield from <expr>
are equivalent to the yield expression statements
(yield <expr>)
(yield from <expr>)
The yield
expressions and statements are only used when defining a generator function, and are only used in the body of the generator function. Using yield in a function definition is sufficient to cause that definition to create a generator function instead of a normal function. Refer to yield expressions.
raise
raise [<expression] [from <expression>]]
If no expressions are present, the raise
statement re-raises the last exception that was active in the current scope. If no exception is active in the current scope, a RuntimeError exception is raised indicating that this is an error.
Otherwise, the raise
statement evaluates the first expression as the exception object. It must be either a subclass or an instance of BaseException. If it is a class, the exception instance will be obtained when needed by instantiating the class with no arguments.
The type of the exception is the exception instance’s class, the value is the instance itself.
Changed in version 3.3: None
is now permitted as Y in raise X from Y
.
break
The break
statement may only occur syntactically nested in a for
or while
loop, but not nested in a function or class definition within that loop.
It terminates the nearest enclosing loop, skipping the optional else
clause if the loop has one.
- Loop statements (
for
andwhile
) may have anelse
clause; it is executed when the loop terminates through exhaustion of the list (withfor
) or when the condition becomes false (withwhile
), but not when the loop is terminated by abreak
statement.
If a for
loop is terminated by break
, the loop control target keeps its current value.
When break
passes control out of a try
statement with a finally
clause, that finally
clause is executed before really leaving the loop.
>>> for n in range(2, 10):
... for x in range(2, n):
... if n % x == 0:
... print(n, 'equals', x, '*', n//x)
... break
... else:
... # loop fell through without finding a factor
... print(n, 'is a prime number')
...
2 is a prime number
3 is a prime number
4 equals 2 * 2
5 is a prime number
6 equals 2 * 3
7 is a prime number
8 equals 2 * 4
9 equals 3 * 3
continue
The continue
statment may only occur syntactically nested in a for
or while
loop, but not nested in a function or class definition or finally
clause within that loop. It continues with the next cycle of the nearest enclosing loop.
When continue
statement passes control out of a try
statement with a finally
clause, that finally
clause is executed before really starting the next loop cycle.
>>> for num in range(2, 10):
... if num % 2 == 0:
... print("Found an even number", num)
... continue
... print("Found a number", num)
Found an even number 2
Found a number 3
Found an even number 4
Found a number 5
Found an even number 6
Found a number 7
Found an even number 8
Found a number 9
import
According to import, Python code in one module gains access to the code in another module by the process of importing it. The import statement is the most common way of invoking the import machinery, but it is not the only way. Functions such as importlib.import_module()
and built-in __import__()
can also be used to invoke the import machinery.
The import statement combines two operations; it searches for the named module, then it binds the results of that search to a name in the local scope. The search operation of the import statement is defined as a call to the __import__()
function, with the appropriate arguments. The return value of import() is used to perform the name binding operation of the import statement.
import <module1> [as <name1>] [, <moduleN> [as <nameN>]]
from <relative_module> import <identifier1> [as <name1>] [, <identifierN> [as <nameN>]]
from <relative_module> import (<identifier1> [as <name1>] [, <identifierN> [as <nameN>]])
from <module> import *
global
global <identifier1> [, <identifier2> [, <identifierN>]]
The global
statement is a declaration which holds for the entire current code block. It means that the listed identifiers are to be interpreted as globals. It would be impossible to assign to a global variable without global
, although free variables may refer to globals without being declared global
.
Names listed in a global
statement must not be used in the same code block textually preceding that global
statement.
Names listed in a global
statement must not be defined as formal parameters or in a for
loop control target, class definition, function definition, import
statement, or variable annotation.
nonlocal
nonlocal <identifier1> [, <identifier2> [, <identifierN>]]
The nonlocal
statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals. This is important because the default behavior for binding is to search the local namespace first. The statement allows encapsulated code to rebind variables outside of the local scope besides the global (module) scope.
Names listed in a nonlocal
statement, unlike those listed in a global
statement, must refer to pre-existing bindings in an enclosing scope (the scope in which a new binding should be created cannot be determined unambiguously).
Names listed in a nonlocal
statement must not collide with pre-existing bindings in the local scope.
if..elif..else
The if
statement is used for conditional execution. There can be zero or more elif
parts, and the else
part is optional:
if <expression1>:
<suite1>
elif <expression2>:
<suite2>
elif <expressionM>:
<suiteM>
else:
<suiteN>
It selects exactly one of the suites by evaluating the expressions one by one until one is found to be true; then that suite is executed (and no other part of the if
statement is executed or evaluated). If all expressions are false, the suite of the else
clause, if present, is executed. For instance:
>>> x = int(input("Please enter an integer: "))
Please enter an integer: 42
>>> if x < 0:
... x = 0
... print('Negative changed to zero')
... elif x == 0:
... print('Zero')
... elif x == 1:
... print('Single')
... else:
... print('More')
...
More
for
The for
statement is used to iterate over the elements of a sequence (such as a string, tuple or list) or other iterable object:
for <target_list> in <expression_list>:
<suite>
else:
<suite>
The expression_list is evaluated once; it should yield an iterable object. An iterator is created for the result of the expression_list. The suite is then executed once for each item provided by the iterator, in the order returned by the iterator. Each item in turn is assigned to the target list using the standard rules for assignments, and then the suite is executed. When the items are exhausted (which is immediately when the sequence is empty or an iterator raises a StopIteration exception), the suite in the else
clause, if present, is executed, and the loop terminates.
A break
statement executed in the first suite terminates the loop without executing the else
clause’s suite. A continue
statement executed in the first suite skips the rest of the suite and continues with the next item, or with the else
clause if there is no next item.
Names in the target_list are not deleted when the loop is finished, but if the sequence is empty, they will not have been assigned to at all by the loop. Hint: the built-in function range()
returns an iterator of integers suitable to emulate the effect of Pascal’s for i := a to b do
; e.g., list(range(3))
returns the list [0, 1, 2]
.
>>> for i in range(5):
... print(i)
...
0
1
2
3
4
>>> words = ['cat', 'window', 'defenestrate']
>>> for w in words:
... print(w, len(w))
...
('cat', 3)
('window', 6)
('defenestrate', 12)
If you need to modify the sequence you are iterating over while inside the loop (for example to duplicate selected items), it is recommended that you first make a copy. Iterating over a sequence does not implicitly make a copy. The slice notation makes this especially convenient:
>>> for w in words[:]: # Loop over a slice copy of the entire list.
... if len(w) > 6:
... words.insert(0, w)
...
>>> words
['defenestrate', 'cat', 'window', 'defenestrate']
while
In Python, like in C, any non-zero integer value is true; zero is false. The condition may also be a string or list value, in fact any sequence; anything with a non-zero length is true, empty sequences are false.
The while
statement is used for repeated execution as long as an expression is true:
while <expression>:
<suite>
else:
<suite>
This repeatedly tests the expression and, if it is true, executes the first suite; if the expression is false (which may be the first time it is tested) the suite of the else
clause, if present, is executed and the loop terminates.
A break
statement executed in the first suite terminates the loop without executing the else
clause’s suite. A continue
statement executed in the first suite skips the rest of the suite and goes back to testing the expression.
>>> # Fibonacci series: the sum of two elements defines the next
... a, b = 0, 1
>>> while b < 10:
... print(b)
... a, b = b, a+b
... else:
... print(100)
...
1
1
2
3
5
8
100
Functions
The keyword def
introduces a function definition. It must be followed by the function name and the parenthesized list of formal parameters. The statements that form the body of the function start at the next line, and must be indented.
>>> def fib(n): # write Fibonacci series up to n
... """Print a Fibonacci series up to n."""
... a, b = 0, 1
... while a < n:
... print(a, end=' ')
... a, b = b, a+b
... print()
...
>>> # Now call the function we just defined:
... fib(2000)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597
A function definition introduces the function name in the current symbol table. The value of the function name has a type that is recognized by the interpreter as a user-defined function. This value can be assigned to another name which can then also be used as a function. This serves as a general renaming mechanism:
>>> fib
<function fib at 10042ed0>
>>> f = fib
>>> f(100)
0 1 1 2 3 5 8 13 21 34 55 89
The return
statement returns with a value from a function. return
without an expression argument returns None
. Falling off the end of a function also returns None
.
>>> def fib2(n): # return Fibonacci series up to n
... """Return a list containing the Fibonacci series up to n."""
... result = []
... a, b = 0, 1
... while a < n:
... result.append(a) # see below
... a, b = b, a+b
... return result
...
>>> f100 = fib2(100) # call it
>>> f100 # write the result
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
docstring
The first statement of the function body can optionally be a string literal; this string literal is the function’s documentation string, or docstring.
The first line should always be a short, concise summary of the object’s purpose. For brevity, it should not explicitly state the object’s name or type, since these are available by other means (except if the name happens to be a verb describing a function’s operation). This line should begin with a capital letter and end with a period.
If there are more lines in the documentation string, the second line should be blank, visually separating the summary from the rest of the description. The following lines should be one or more paragraphs describing the object’s calling conventions, its side effects, etc.
The Python parser does not strip indentation from multi-line string literals in Python, so tools that process documentation have to strip indentation if desired. This is done using the following convention. The first non-blank line after the first line of the string determines the amount of indentation for the entire documentation string. (We can’t use the first line since it is generally adjacent to the string’s opening quotes so its indentation is not apparent in the string literal.) Whitespace equivalent to this indentation is then stripped from the start of all lines of the string. Lines that are indented less should not occur, but if they occur all their leading whitespace should be stripped. Equivalence of whitespace should be tested after expansion of tabs (to 8 spaces, normally).
>>> def my_function():
... """Do nothing, but document it.
...
... No, really, it doesn't do anything.
... """
... pass
...
>>> print(my_function.__doc__)
Do nothing, but document it.
No, really, it doesn't do anything.
Default Argument Values
The default values are evaluated at the point of function definition in the defining scope:
>>> i = 5
>>> def f(arg = i):
... print(arg)
...
>>> i = 6
>>> f()
5
The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes. The following function accumulates the arguments passed to it on subsequent calls:
>>> def f(a, L = []):
... L.append(a)
... return L
...
>>> print(f(1))
[1]
>>> print(f(2))
[1, 2]
>>> print(f(3))
[1, 2, 3]
If you don’t want the default to be shared between subsequent calls, you can write the function like this instead:
>>> def f(a, L=None):
... if L is None:
... L = []
... L.append(a)
... return L
...
>>> print(f(1))
[1]
>>> print(f(2))
[2]
>>> print(f(3))
[3]
Keyword Arguments
Functions can also be called using keyword arguments of the form kwarg=value
. The function:
>>> def parrot(voltage, state='a stiff', action='voom', type='Norwegian Blue'):
... print("-- This parrot wouldn't", action, end=' ')
... print("if you put", voltage, "volts through it.")
... print("-- Lovely plumage, the", type)
... print("-- It's", state, "!")
...
can be called in any of the following ways:
parrot(1000) # 1 positional argument
parrot(voltage=1000) # 1 keyword argument
parrot(voltage=1000000, action='VOOOOOM') # 2 keyword arguments
parrot(action='VOOOOOM', voltage=1000000) # 2 keyword arguments
parrot('a million', 'bereft of life', 'jump') # 3 positional arguments
parrot('a thousand', state='pushing up the daisies') # 1 positional, 1 keyword
Note:
- In a function call, keyword arguments must follow positional arguments.
- All the keyword arguments passed must match one of the arguments accepted by the function and their order is not important.
- No argument may receive a value more than once.
When a final formal parameter of the form **name
is present, it receives a dictionary containing all keyword arguments except for those corresponding to a formal parameter. This may be combined with a formal parameter of the form *name
which receives a tuple containing the positional arguments beyond the formal parameter list. *name
must occur before **name
.
Arbitrary Argument Lists
A function can be called with an arbitrary number of arguments. These arguments will be wrapped up in a tuple. Before the variable number of arguments, zero or more normal arguments may occur.
def write_multiple_items(file, separator, *args):
file.write(separator.join(args))
Normally, these variadic arguments will be last in the list of formal parameters, because they scoop up all remaining input arguments that are passed to the function. Any formal parameters which occur after the *args
parameter are keyword-only arguments, meaning that they can only be used as keywords rather than positional arguments.
>>> def concat(*args, sep="/"):
... return sep.join(args)
...
>>> concat("earth", "mars", "venus") # sep = "/" in this case
'earth/mars/venus'
>>> concat("earth", "mars", "venus", sep=".") # sep = "." in this case
'earth.mars.venus'
Unpacking Argument Lists
The reverse situation occurs when the arguments are already in a list or tuple but need to be unpacked for a function call requiring separate positional arguments. If they are not available separately, write the function call with the *
-operator to unpack the arguments out of a list or tuple:
>>> list(range(3, 6)) # normal call with separate arguments
[3, 4, 5]
>>> args = [3, 6]
>>> list(range(*args)) # call with arguments unpacked from a list
[3, 4, 5]
In the same fashion, dictionaries can deliver keyword arguments with the **
-operator:
>>> def parrot(voltage, state='a stiff', action='voom'):
... print("-- This parrot wouldn't", action, end=' ')
... print("if you put", voltage, "volts through it.", end=' ')
... print("E's", state, "!")
...
>>> d = {"voltage": "four million", "state": "bleedin' demised", "action": "VOOM"}
>>> parrot(**d)
-- This parrot wouldn't VOOM if you put four million volts through it. E's bleedin' demised !
Lambda Expressions
Small anonymous functions can be created with the lambda
keyword. This function returns the sum of its two arguments: lambda a, b: a+b
. Lambda functions can be used wherever function objects are required. They are syntactically restricted to a single expression. Semantically, they are just syntactic sugar for a normal function definition. Like nested function definitions, lambda functions can reference variables from the containing scope:
>>> def make_incrementor(n):
... return lambda x: x + n
...
>>> f = make_incrementor(42)
>>> f(0)
42
>>> f(1)
43
The above example uses a lambda expression to return a function. Another use is to pass a small function as an argument:
>>> pairs = [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')]
>>> pairs.sort(key=lambda pair: pair[1])
>>> pairs
[(4, 'four'), (1, 'one'), (3, 'three'), (2, 'two')]
Function Annotations
Function annotations are completely optional metadata information about the types used by user-defined functions, see PEP 484: Type Hints for more information.
Annotations are stored in the __annotations__
attribute of the function as a dictionary and have no effect on any other part of the function.
Parameter annotations are defined by a colon after the parameter name, followed by an expression evaluating to the value of the annotation.
Return annotations are defined by a literal ->
, followed by an expression, between the parameter list and the colon denoting the end of the def
statement. The following example has a positional argument, a keyword argument, and the return value annotated:
>>> def f(ham: str, eggs: str = 'eggs') -> str:
... print("Annotations:", f.__annotations__)
... print("Arguments:", ham, eggs)
... return ham + ' and ' + eggs
...
>>> f('spam')
Annotations: {'ham': <class 'str'>, 'return': <class 'str'>, 'eggs': <class 'str'>}
Arguments: spam eggs
'spam and eggs'
Modules
If you quit from the Python interpreter and enter it again, the definitions you have made (functions and variables) are lost. Therefore, if you want to write a somewhat longer program, you are better off using a text editor to prepare the input for the interpreter and running it with that file as input instead.
To support this, Python has a way to put definitions in a file and use them in a script or in an interactive instance of the interpreter. Such a file is called a module; definitions from a module can be imported into other modules or into the main module (the collection of variables that you have access to in a script executed at the top level and in calculator mode).
A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. Within a module, the module’s name (as a string) is available as the value of the global variable __name__. For instance, use your favorite text editor to create a file called fibo.py in the current directory with the following contents:
# Fibonacci numbers module
def fib(n): # write Fibonacci series up to n
a, b = 0, 1
while b < n:
print(b, end=' ')
a, b = b, a+b
print()
def fib2(n): # return Fibonacci series up to n
result = []
a, b = 0, 1
while b < n:
result.append(b)
a, b = b, a+b
return result
Now enter the Python interpreter and import this module and access the functions with the following command:
>>> import fibo
>>> fibo.fib(1000)
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
>>> fibo.fib2(100)
[1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
>>> fibo.__name__
'fibo'
>>> fib = fibo.fib
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377
A module can contain executable statements as well as function definitions. These statements are intended to initialize the module. They are executed only the first time the module name is encountered in an import statement.
Each module has its own private symbol table, which is used as the global symbol table by all functions defined in the module. Thus, the author of a module can use global variables in the module without worrying about accidental clashes with a user’s global variables. On the other hand, if you know what you are doing you can touch a module’s global variables with the same notation used to refer to its functions, modname.itemname
.
Modules can import other modules. It is customary but not required to place all import
statements at the beginning of a module (or script, for that matter). The imported module names are placed in the importing module’s global symbol table.
There is a variant of the import
statement that imports names from a module directly into the importing module’s symbol table. This does not introduce the module name from which the imports are taken in the local symbol table (so in the example, fibo is not defined):
>>> from fibo import fib, fib2
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377
There is even a variant to import all names that a module defines. This imports all names except those beginning with an underscore _
:
>>> from fibo import *
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377
Executing modules as scripts
When you run a Python module with following command, the code in the module will be executed, just as if you imported it, but with the __name__
set to __main__
.
python fibo.py <arguments>
or, by adding this code at the end of your module:
if __name__ == "__main__":
import sys
fib(int(sys.argv[1]))
you can make the file usable as a script as well as an importable module, because the code that parses the command line only runs if the module is executed as the main file:
$ python fibo.py 50
1 1 2 3 5 8 13 21 34
Module Search Path
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path
, which is initialized from these locations:
- The directory containing the input script (or the current directory when no file is specified).
- PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
- The installation-dependent default.
Note: On file systems which support symlinks, the directory containing the input script is calculated after the symlink is followed. In other words the directory containing the symlink is not added to the module search path.
After initialization, Python programs can modify sys.path
. The directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path. This means that scripts in that directory will be loaded instead of modules of the same name in the library directory. This is an error unless the replacement is intended.
Compiled Python Files
To speed up loading modules, Python caches the compiled version of each module in the __pycache__
directory under the name module.version.pyc, where the version encodes the format of the compiled file; it generally contains the Python version number, such as __pycache__/spam.cpython-33.pyc
. This naming convention allows compiled modules from different releases and different versions of Python to coexist.
Python checks the modification date of the source against the compiled version to see if it’s out of date and needs to be recompiled.
Python does not check the cache in two circumstances:
- It always recompiles and does not store the result for the module that’s loaded directly from the command line.
- It does not check the cache if there is no source module. To support a non-source (compiled only) distribution, the compiled module must be in the source directory, and there must not be a source module.
Packages
Packages are a way of structuring Python’s module namespace by using dotted module names. For example, the module name A.B designates a submodule named B in a package named A.
sound/ # Top-level package
__init__.py # Initialize the sound package
formats/ # Subpackage for file format conversions
__init__.py
wavread.py
wavwrite.py
aiffread.py
aiffwrite.py
auread.py
auwrite.py
...
effects/ # Subpackage for sound effects
__init__.py
echo.py
surround.py
reverse.py
...
filters/ # Subpackage for filters
__init__.py
equalizer.py
vocoder.py
karaoke.py
...
When importing the package, Python searches through the directories on sys.path looking for the package subdirectory.
The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__
variable.
Users of the package can import individual modules from the package:
import sound.effects.echo
This loads the submodule sound.effects.echo. It must be referenced with its full name.
sound.effects.echo.echofilter(input, output, delay=0.7, atten=4)
An alternative way of importing the submodule is:
from sound.effects import echo
This also loads the submodule echo, and makes it available without its package prefix, so it can be used as follows:
echo.echofilter(input, output, delay=0.7, atten=4)
Yet another variation is to import the desired function or variable directly:
from sound.effects.echo import echofilter
Again, this loads the submodule echo, but this makes its function echofilter()
directly available:
echofilter(input, output, delay=0.7, atten=4)
__init__.py
The import
statement uses the following convention:
If a package’s __init__.py
code defines a list named __all__
, it is taken to be the list of module names that should be imported when from package import *
is encountered. The following code would mean that from sound.effects import * would import the three named submodules of the sound package:
__all__ = ["echo", "surround", "reverse"]
If __all__
is not defined, the statement from sound.effects import *
does not import all submodules from the package sound.effects into the current namespace; it only ensures that the package sound.effects has been imported (possibly running any initialization code in __init__.py
) and then imports whatever names are defined in the package. This includes any names defined (and submodules explicitly loaded) by __init__.py
. It also includes any submodules of the package that were explicitly loaded by previous import
statements.
dir() Function
The built-in function dir()
is used to find out which names a module defines (including all types of names: variables, modules, functions, etc). It returns a sorted list of strings. Without arguments, dir()
lists the names you have defined currently.
>>> dir()
['__builtins__', '__doc__', '__name__', '__package__']
>>>
>>> import sys
>>> dir(sys)
['__displayhook__', '__doc__', '__egginsert', '__excepthook__', '__name__', '__package__', '__plen', '__stderr__', '__stdin__', '__stdout__', '_clear_type_cache', '_current_frames', '_getframe', '_mercurial', 'api_version', 'argv', 'builtin_module_names', 'byteorder', 'call_tracing', 'callstats', 'copyright', 'displayhook', 'dont_write_bytecode', 'exc_clear', 'exc_info', 'exc_type', 'excepthook', 'exec_prefix', 'executable', 'exit', 'exitfunc', 'flags', 'float_info', 'float_repr_style', 'getcheckinterval', 'getdefaultencoding', 'getdlopenflags', 'getfilesystemencoding', 'getprofile', 'getrecursionlimit', 'getrefcount', 'getsizeof', 'gettrace', 'hexversion', 'long_info', 'maxint', 'maxsize', 'maxunicode', 'meta_path', 'modules', 'path', 'path_hooks', 'path_importer_cache', 'platform', 'prefix', 'ps1', 'ps2', 'py3kwarning', 'setcheckinterval', 'setdlopenflags', 'setprofile', 'setrecursionlimit', 'settrace', 'stderr', 'stdin', 'stdout', 'subversion', 'version', 'version_info', 'warnoptions']
>>>
>>> print(sys.version)
3.5.0 (default, Dec 21 2015, 22:50:16)
[GCC 4.8.2]
The dir()
does not list the names of built-in functions and variables. If you want a list of those, they are defined in the standard module builtins:
>>> import builtins
>>> dir(builtins)
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']
Input and Output
Errors and Exceptions
There are (at least) two distinguishable kinds of errors: syntax errors and exceptions.
Syntax Errors
Syntax errors, also known as parsing errors, are perhaps the most common kind of complaint you get while you are still learning Python:
>>> while True print('Hello world')
File "<stdin>", line 1
while True print('Hello world')
^
SyntaxError: invalid syntax
Exceptions
Errors detected during execution are called exceptions and are not unconditionally fatal: you will soon learn how to handle them in Python programs. Most exceptions are not handled by programs, however, and result in error messages as shown here:
>>> 1/0
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero
Exceptions come in different types, and the type is printed as part of the message: ZeroDivisionError. Standard exception names are built-in identifiers (not reserved keywords). Built-in Exceptions lists the built-in exceptions and their meanings.
Handling Exceptions
It is possible to write programs that handle selected exceptions:
>>> while True:
... try:
... x = int(input("Please enter a number: "))
... break
... except ValueError:
... print("Oops! That was no valid number. Try again...")
...
Please enter a number: abc
Oops! That was no valid number. Try again...
Please enter a number: 100
The try
statement works as follows:
- First, the
try
clause (the statement(s) between thetry
andexcept
keywords) is executed. - If no exception occurs, the
except
clause is skipped and execution of thetry
statement is finished. - If an exception occurs during execution of the
try
clause, the rest of the clause is skipped. Then if its type matches the exception named after theexcept
keyword, theexcept
clause is executed, and then execution continues after thetry
statement. - If an exception occurs which does not match the exception named in the
except
clause, it is passed on to outertry
statements; if no handler is found, it is an unhandled exception and execution stops with a message as shown above.
A try
statement may have more than one except
clause, to specify handlers for different exceptions. At most one handler will be executed. Handlers only handle exceptions that occur in the corresponding try
clause, not in other handlers of the same try
statement. An except
clause may name multiple exceptions as a parenthesized tuple:
... except (RuntimeError, TypeError, NameError):
... pass
The last except clause may omit the exception name(s), to serve as a wildcard. Use this with extreme caution, since it is easy to mask a real programming error in this way! It can also be used to print an error message and then re-raise the exception (allowing a caller to handle the exception as well):
import sys
try:
f = open('myfile.txt')
s = f.readline()
i = int(s.strip())
except OSError as err: # the except clause may specify a variable after the exception name
print("OS error: {0}".format(err))
except ValueError:
print("Could not convert data to an integer.")
except:
print("Unexpected error:", sys.exc_info()[0])
raise
The try...except
statement has an optional else
clause, which, when present, must follow all except
clauses. It is useful for code that must be executed if the try clause does not raise an exception:
for arg in sys.argv[1:]:
try:
f = open(arg, 'r')
except OSError:
print('cannot open', arg)
else:
print(arg, 'has', len(f.readlines()), 'lines')
f.close()
Exception handlers don’t just handle exceptions if they occur immediately in the try clause, but also if they occur inside functions that are called (even indirectly) in the try clause:
def this_fails():
x = 1/0
try:
this_fails()
except ZeroDivisionError as err:
print('Handling run-time error:', err)
The try
statement has another optional clause which is intended to define clean-up actions that must be executed under all circumstances:
>>> try:
... raise KeyboardInterrupt
... finally:
... print('Goodbye, world!')
...
Goodbye, world!
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
KeyboardInterrupt
A finally
clause is always executed before leaving the try
statement, whether an exception has occurred or not. When an exception has occurred in the try
clause and has not been handled by an except
clause (or it has occurred in an except
or else
clause), it is re-raised after the finally
clause has been executed. The finally
clause is also executed on the way out when any other clause of the try
statement is left via a break
, continue
or return
statement.
>>> def divide(x, y):
... try:
... result = x / y
... except ZeroDivisionError:
... print("division by zero!")
... else:
... print("result is", result)
... finally:
... print("executing finally clause")
...
>>> divide(2, 1)
result is 2.0
executing finally clause
>>>
>>> divide(2, 0)
division by zero!
executing finally clause
>>>
>>> divide("2", "1")
executing finally clause
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in divide
TypeError: unsupported operand type(s) for /: 'str' and 'str'
In real world applications, the finally
clause is useful for releasing external resources (such as files or network connections), regardless of whether the use of the resource was successful.
Raising Exceptions
The raise
statement allows the programmer to force a specified exception to occur:
try:
raise NameError('HiThere')
except NameError:
print('An exception flew by!')
raise
User-defined Exceptions
Programs may name their own exceptions by creating a new exception class. Exceptions should typically be derived from the Exception
class, either directly or indirectly.
>>> help(Exception)
Help on class Exception in module builtins:
class Exception(BaseException)
| Common base class for all non-exit exceptions.
|
| Method resolution order:
| Exception
| BaseException
| object
|
| Methods defined here:
|
| __init__(self, /, *args, **kwargs)
| Initialize self. See help(type(self)) for accurate signature.
|
| __new__(*args, **kwargs) from builtins.type
| Create and return a new object. See help(type) for accurate signature.
|
| ----------------------------------------------------------------------
| Methods inherited from BaseException:
|
| __delattr__(self, name, /)
| Implement delattr(self, name).
|
| __getattribute__(self, name, /)
| Return getattr(self, name).
|
| __reduce__(...)
| helper for pickle
|
| __repr__(self, /)
| Return repr(self).
|
| __setattr__(self, name, value, /)
| Implement setattr(self, name, value).
|
| __setstate__(...)
|
| __str__(self, /)
| Return str(self).
|
| with_traceback(...)
| Exception.with_traceback(tb) --
| set self.__traceback__ to tb and return self.
|
| ----------------------------------------------------------------------
| Data descriptors inherited from BaseException:
|
| __cause__
| exception cause
|
| __context__
| exception context
|
| __dict__
|
| __suppress_context__
|
| __traceback__
|
| args
Exception classes can be defined which do anything any other class can do, but are usually kept simple, often only offering a number of attributes that allow information about the error to be extracted by handlers for the exception. When creating a module that can raise several distinct errors, a common practice is to create a base class for exceptions defined by that module, and subclass that to create specific exception classes for different error conditions.
Most exceptions are defined with names that end in Error, similar to the naming of the standard exceptions.
Many standard modules define their own exceptions to report errors that may occur in functions they define.
Classes
Compared with other programming languages, Python’s class mechanism adds classes with a minimum of new syntax and semantics. It is a mixture of the class mechanisms found in C++ and Modula-3. Python classes provide all the standard features of Object Oriented Programming: the class inheritance mechanism allows multiple base classes, a derived class can override any methods of its base class or classes, and a method can call the method of a base class with the same name. Objects can contain arbitrary amounts and kinds of data. As is true for modules, classes partake of the dynamic nature of Python: they are created at runtime, and can be modified further after creation.
Scopes and Namespaces
A namespace is a mapping from names to objects. Namespaces are created at different moments and have different lifetimes:
- The namespace containing the built-in names is created when the Python interpreter starts up, and is never deleted.
- The global namespace for a module is created when the module definition is read in; normally, module namespaces also last until the interpreter quits.
- The statements executed by the top-level invocation of the interpreter, either read from a script file or interactively, are considered part of a module called
__main__
, so they have their own global namespace. - The local namespace for a function is created when the function is called, and deleted when the function returns or raises an exception that is not handled within the function.
The built-in names actually also live in a module named builtins:
>>> import builtins
>>> dir(builtins)
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']
A scope is a textual region of a Python program where a namespace is directly accessible. Although scopes are determined statically, they are used dynamically. At any time during execution, there are at least three nested scopes whose namespaces are directly accessible:
- the innermost scope, which is searched first, contains the local names
- the scopes of any enclosing functions, which are searched starting with the nearest enclosing scope, contains non-local, but also non-global names
- the next-to-last scope contains the current module’s global names
- the outermost scope (searched last) is the namespace containing built-in names
The global
statement can be used to indicate that particular variables live in the global scope and should be rebound there; the nonlocal
statement indicates that particular variables live in an enclosing scope and should be rebound there.
Class Definition Syntax
class ClassName:
<statement-1>
...
<statement-N>
Class definitions, like function definitions (def
statements) must be executed before they have any effect. You could conceivably place a class definition in a branch of an if
statement, or inside a function.
In practice, the statements inside a class definition will usually be function definitions, but other statements are allowed, and sometimes useful.
When a class definition is entered, a new namespace is created, and used as the local scope.
When a class definition is left normally (via the end), a class object is created.
>>> class MyClass:
... '''A simple example class'''
... def __init__(self, var):
... self.i = var
... def f(self):
... return "Hello world"
... i = 123
...
>>>
>>> help(MyClass)
Help on class MyClass in module __main__:
class MyClass(builtins.object)
| A simple example class
|
| Methods defined here:
|
| __init__(self, var)
| Initialize self. See help(type(self)) for accurate signature.
|
| f(self)
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __dict__
| dictionary for instance variables (if defined)
|
| __weakref__
| list of weak references to the object (if defined)
|
| ----------------------------------------------------------------------
| Data and other attributes defined here:
|
| i = 123
The special thing about methods is that the instance object is passed as the first argument of the function, that’s self
. In general, calling a method with a list of n arguments is equivalent to calling the corresponding function with an argument list that is created by inserting the method’s instance object before the first argument.
Class Objects
Class objects support two kinds of operations: attribute references and instantiation.
- Attribute references use the standard syntax used for all attribute references in Python:
obj.name
. Valid attribute names are all the names that were in the class’s namespace when the class object was created. - Class instantiation uses function notation. Just pretend that the class object is a parameterless function that returns a new instance of the class:
x = MyClass()
. When a class defines an__init__()
method, class instantiation automatically invokes__init__()
for the newly-created class instance.
Class and Instance Variables
Instance variables are for data unique to each instance and class variables are for attributes and methods shared by all instances of the class:
>>> class Dog:
... kind = 'canine' # class variable shared by all instances
... def __init__(self, name):
... self.name = name # instance variable unique to each instance
...
>>> d = Dog('Fido')
>>> e = Dog('Buddy')
>>> d.kind # shared by all dogs
'canine'
>>> e.kind # shared by all dogs
'canine'
>>> d.name # unique to d
'Fido'
>>> e.name # unique to e
'Buddy'
Random Remarks
Data attributes override method attributes with the same name; to avoid accidental name conflicts, which may cause hard-to-find bugs in large programs, it is wise to use some kind of convention that minimizes the chance of conflicts. Possible conventions include capitalizing method names, prefixing data attribute names with a small unique string (perhaps just an underscore), or using verbs for methods and nouns for data attributes.
Inheritance
The syntax for a derived class definition looks like this:
class DerivedClassName(BaseClassName):
<statement-1>
...
<statement-N>
Derived classes may override methods of their base classes. For C++ programmers: all methods in Python are effectively virtual.
An overriding method in a derived class may in fact want to extend rather than simply replace the base class method of the same name. There is a simple way to call the base class method directly: just call BaseClassName.methodName(self, arguments)
.
Python has two built-in functions that work with inheritance:
- Use
isinstance()
to check an instance’s type:isinstance(obj, int)
will be True only ifobj.__class__
isint
or some class derived fromint
. - Use
issubclass()
to check class inheritance:issubclass(bool, int)
is True sincebool
is a subclass ofint
. However,issubclass(float, int)
is False sincefloat
is not a subclass ofint
.
Python supports a form of multiple inheritance as well. A class definition with multiple base classes looks like this:
class DerivedClassName(Base1, Base2, Base3):
<statement-1>
...
<statement-N>
Private Variables
Private instance variables that cannot be accessed except from inside an object don’t exist in Python. However, there is a convention that is followed by most Python code: a name prefixed with an underscore (e.g. _spam
) should be treated as a non-public part of the API (whether it is a function, a method or a data member). It should be considered an implementation detail and subject to change without notice.
Since there is a valid use-case for class-private members (namely to avoid name clashes of names with names defined by subclasses), there is limited support for such a mechanism, called name mangling. Any identifier of the form __spam
(at least two leading underscores, at most one trailing underscore) is textually replaced with _classname__spam
, where classname is the current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, as long as it occurs within the definition of a class.
Name mangling is helpful for letting subclasses override methods without breaking intraclass method calls.
class Mapping:
def __init__(self, iterable):
self.items_list = []
self.__update(iterable)
def update(self, iterable):
for item in iterable:
self.items_list.append(item)
__update = update # private copy of original update() method
class MappingSubclass(Mapping):
def update(self, keys, values):
# provides new signature for update()
# but does not break __init__()
for item in zip(keys, values):
self.items_list.append(item)
Iterators
The most container objects can be looped over using a for statement:
for element in [1, 2, 3]:
print(element)
for element in (1, 2, 3):
print(element)
for key in {'one':1, 'two':2}:
print(key)
for char in "123":
print(char)
for line in open("myfile.txt"):
print(line, end='')
The use of iterators pervades and unifies Python. Behind the scenes, the for statement calls iter()
on the container object. The function returns an iterator object that defines the method __next__()
which accesses elements in the container one at a time. When there are no more elements, __next__()
raises a StopIteration
exception which tells the for loop to terminate. You can call the __next__()
method using the next()
built-in function; this example shows how it all works:
>>> s = 'abc'
>>> it = iter(s)
>>> it
<iterator object at 0x00A1DB50>
>>> next(it)
'a'
>>> next(it)
'b'
>>> next(it)
'c'
>>> next(it)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
next(it)
StopIteration
Having seen the mechanics behind the iterator protocol, it is easy to add iterator behavior to your classes. Define an __iter__()
method which returns an object with a __next__()
method. If the class defines __next__()
, then __iter__()
can just return self
:
class Reverse:
"""Iterator for looping over a sequence backwards."""
def __init__(self, data):
self.data = data
self.index = len(data)
def __iter__(self):
return self
def __next__(self):
if self.index == 0:
raise StopIteration
self.index = self.index - 1
return self.data[self.index]
>>> rev = Reverse('spam')
>>> iter(rev)
<__main__.Reverse object at 0x00A1DB50>
>>> for char in rev:
... print(char)
...
m
a
p
s
Help information of method iter()
:
Help on built-in function iter in module builtins:
iter(...)
iter(iterable) -> iterator
iter(callable, sentinel) -> iterator
Get an iterator from an object. In the first form, the argument must
supply its own iterator, or be a sequence.
In the second form, the callable is called until it returns the sentinel.
Help information of method next()
:
Help on built-in function next in module builtins:
next(...)
next(iterator[, default])
Return the next item from the iterator. If default is given and the iterator
is exhausted, it is returned instead of raising StopIteration.
Generators
Generators are a simple and powerful tool for creating iterators. They are written like regular functions but use the yield
statement whenever they want to return data. Each time next()
is called on it, the generator resumes where it left off (it remembers all the data values and which statement was last executed). An example shows that generators can be trivially easy to create:
def reverse(data):
for index in range(len(data)-1, -1, -1):
yield data[index]
>>> for char in reverse('golf'):
... print(char)
...
f
l
o
g
Special Methods
Virtual Environments
Refer to Virtual Environments and Packages
Python Libraries
Python Standard Library
The Python Standard Library:
The Python Module Index:
In addition to the Python standard library, there is a growing collection of several thousand components (from individual programs and modules to packages and entire application development frameworks), available from the Python Package Index (PyPI).
The Python standard library contains:
- Data types that would normally be considered part of the core of a language, such as numbers and lists.
- Built-in functions and exceptions can be used by all Python code without the need of an
import
statement. Some of these are defined by the core language, but many are not essential for the core semantics. - Collection of modules. Some modules are written in C and built in to the Python interpreter; others are written in Python and imported in source form. Some modules provide interfaces that are highly specific to Python, like printing a stack trace; some provide interfaces that are specific to particular operating systems, such as access to specific hardware; others provide interfaces that are specific to a particular application domain, like the World Wide Web. Some modules are available in all versions and ports of Python; others are only available when the underlying system supports or requires them; yet others are available only when a particular configuration option was chosen at the time when Python was compiled and installed.
Here is a list of Python 3.x standard library:
Modules | Description |
---|---|
os | provides functions for interacting with the operating system |
glob | provides functions for making file lists from directory wildcard searches |
sys | command line arguments |
getopt | processes sys.argv using the conventions of the Unix getopt() function |
argparse | more powerful and flexible command line processing |
re | provides regular expression tools for advanced string processing |
math | gives access to the underlying C library functions for floating point math |
random | provides tools for making random selections |
statistics | calculates basic statistical properties (the mean, median, variance, etc.) of numeric data |
datetime | supplies classes for manipulating dates and times in both simple and complex ways |
zlib | support common data archiving and compression formats |
gzip | support common data archiving and compression formats |
bz2 | support common data archiving and compression formats |
lzma | support common data archiving and compression formats |
zipfile | support common data archiving and compression formats |
tarfile | support common data archiving and compression formats |
timeit | demonstrates a modest performance advantage |
profile | provide tools for identifying time critical sections in larger blocks of code |
doctest | provides a tool for scanning a module and validating tests embedded in a program’s docstrings |
unittest | allows a more comprehensive set of tests to be maintained in a separate file |
The built-in functions, constants, types and exceptions can be listed by dir(__builtins__)
or dir(builtins)
:
chenwx@chenwx:~ $ python3
Python 3.6.8 (default, Jan 14 2019, 11:02:34)
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> dir(__builtins__)
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'ModuleNotFoundError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']
>>> import builtins
>>> dir(builtins)
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'ModuleNotFoundError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '_', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']
Built-in Functions
The Python interpreter has a number of functions and types built into it that are always available, refer to Python 3.x Built-in Functions:
abs()
all()
any()
ascii()
bin()
bool()
bytearray()
bytes()
callable()
chr()
classmethod()
compile()
complex()
delattr()
dict()
dir()
divmod()
enumerate()
eval()
exec()
filter()
float()
format()
frozenset()
getattr()
globals()
hasattr()
hash()
help()
hex()
id()
input()
int()
isinstance()
issubclass()
iter()
len()
list()
locals()
map()
max()
memoryview()
min()
next()
object()
oct()
open()
ord()
pow()
print()
property()
range()
repr()
reversed()
round()
set()
setattr()
slice()
sorted()
staticmethod()
str()
sum()
super()
tuple()
type()
vars()
zip()
__import__()
You can get help information of the built-in functions by help()
, such as:
>>> help(abs)
Help on built-in function abs in module builtins:
abs(x, /)
Return the absolute value of the argument.
Built-in Constants
A small number of constants live in the built-in namespace. They are:
Constants | Description |
---|---|
False |
The false value of the bool type. Assignments to False are illegal and raise a SyntaxError . |
True |
The true value of the bool type. Assignments to True are illegal and raise a SyntaxError . |
None |
The sole value of the type NoneType . None is frequently used to represent the absence of a value, as when default arguments are not passed to a function. Assignments to None are illegal and raise a SyntaxError . |
NotImplemented |
Special value which should be returned by the binary special methods (e.g. __eq__() , __lt__() , __add__() , __rsub__() , etc.) to indicate that the operation is not implemented with respect to the other type; may be returned by the in-place binary special methods (e.g. __imul__() , __iand__() , etc.) for the same purpose. Its truth value is true. |
Ellipsis |
The same as ... . Special value used mostly in conjunction with extended slicing syntax for user-defined container data types. |
__debug__ |
This constant is true if Python was not started with an -O option. Assignments to __debug__ are illegal and raise a SyntaxError . |
The site module (which is imported automatically during startup, except if the -S
command-line option is given) adds several constants to the built-in namespace. They are useful for the interactive interpreter shell and should not be used in programs. Constants added by the site module are:
Constants | Description |
---|---|
quit(code=None) exit(code=None) |
Objects that when printed, print a message like Use quit() or Ctrl-D (i.e. EOF) to exit, and when called, raise SystemExit with the specified exit code. |
copyright license credits |
Objects that when printed, print a message like Type license() to see the full license text, and when called, display the corresponding text in a pager-like fashion (one screen at a time). |
Built-in Types
Refer to Built-in Types for more information.
Built-in Exceptions
The class hierarchy for built-in exceptions is:
BaseException
+-- SystemExit
+-- KeyboardInterrupt
+-- GeneratorExit
+-- Exception
+-- StopIteration
+-- StopAsyncIteration
+-- ArithmeticError
| +-- FloatingPointError
| +-- OverflowError
| +-- ZeroDivisionError
+-- AssertionError
+-- AttributeError
+-- BufferError
+-- EOFError
+-- ImportError
+-- ModuleNotFoundError
+-- LookupError
| +-- IndexError
| +-- KeyError
+-- MemoryError
+-- NameError
| +-- UnboundLocalError
+-- OSError
| +-- BlockingIOError
| +-- ChildProcessError
| +-- ConnectionError
| | +-- BrokenPipeError
| | +-- ConnectionAbortedError
| | +-- ConnectionRefusedError
| | +-- ConnectionResetError
| +-- FileExistsError
| +-- FileNotFoundError
| +-- InterruptedError
| +-- IsADirectoryError
| +-- NotADirectoryError
| +-- PermissionError
| +-- ProcessLookupError
| +-- TimeoutError
+-- ReferenceError
+-- RuntimeError
| +-- NotImplementedError
| +-- RecursionError
+-- SyntaxError
| +-- IndentationError
| +-- TabError
+-- SystemError
+-- TypeError
+-- ValueError
| +-- UnicodeError
| +-- UnicodeDecodeError
| +-- UnicodeEncodeError
| +-- UnicodeTranslateError
+-- Warning
+-- DeprecationWarning
+-- PendingDeprecationWarning
+-- RuntimeWarning
+-- SyntaxWarning
+-- UserWarning
+-- FutureWarning
+-- ImportWarning
+-- UnicodeWarning
+-- BytesWarning
+-- ResourceWarning
Collection of Modules
Refer to Python Module Index for the index of all modules.
Text Processing Services
Modules | Description | CPython_Code |
---|---|---|
string | Common string operations | Lib/string.py |
re | Regular expression operations | Lib/re.py |
difflib | Helpers for computing deltas | Lib/difflib.py |
textwrap | Text wrapping and filling | Lib/textwrap.py |
unicodedata | Unicode Database | |
stringprep | Internet String Preparation | Lib/stringprep.py |
readline | GNU readline interface | |
rlcompleter | Completion function for GNU readline | Lib/rlcompleter.py |
Binary Data Services
Modules | Description | CPython_Code |
---|---|---|
struct | Interpret bytes as packed binary data | Lib/struct.py |
codecs | Codec registry and base classes | Lib/codecs.py |
Data Types
Modules | Description | CPython_Code |
---|---|---|
datetime | Basic date and time types | Lib/datetime.py |
calendar | General calendar-related functions | Lib/calendar.py |
collections | Container datatypes | Lib/collections/__init__.py |
collections.abc | Abstract Base Classes for Containers | Lib/_collections_abc.py |
heapq | Heap queue algorithm | Lib/heapq.py |
bisect | Array bisection algorithm | Lib/bisect.py |
array | Efficient arrays of numeric values | |
weakref | Weak references | Lib/weakref.py |
types | Dynamic type creation and names for built-in types | Lib/types.py |
copy | Shallow and deep copy operations | Lib/copy.py |
pprint | Data pretty printer | Lib/pprint.py |
reprlib | Alternate repr() implementation | Lib/reprlib.py |
enum | Support for enumerations | Lib/enum.py |
Numeric and Mathematical Modules
Modules | Description | CPython_Code |
---|---|---|
numbers | Numeric abstract base classes | Lib/numbers.py |
math | Mathematical functions | |
cmath | Mathematical functions for complex numbers | |
decimal | Decimal fixed point and floating point arithmetic | Lib/decimal.py |
fractions | Rational numbers | Lib/fractions.py |
random | Generate pseudo-random numbers | Lib/random.py |
statistics | Mathematical statistics functions | Lib/statistics.py |
Functional Programming Modules
Modules | Description | CPython_Code |
---|---|---|
itertools | Functions creating iterators for efficient looping | |
functools | Higher-order functions and operations on callable objects | Lib/functools.py |
operator | Standard operators as functions | Lib/operator.py |
File and Directory Access
Modules | Description | CPython_Code |
---|---|---|
pathlib | Object-oriented filesystem paths | Lib/pathlib.py |
os.path | Common pathname manipulations | Lib/posixpath.py (for POSIX) Lib/ntpath.py (for Windows NT) Lib/macpath.py (for Macintosh) |
fileinput | Iterate over lines from multiple input streams | Lib/fileinput.py |
stat | Interpreting stat() results | Lib/stat.py |
filecmp | File and Directory Comparisons | Lib/filecmp.py |
tempfile | Generate temporary files and directories | Lib/tempfile.py |
glob | Unix style pathname pattern expansion | Lib/glob.py |
fnmatch | Unix filename pattern matching | Lib/fnmatch.py |
linecache | Random access to text lines | Lib/linecache.py |
shutil | High-level file operations | Lib/shutil.py |
macpath | Mac OS 9 path manipulation functions | Lib/macpath.py |
Data Persistence
Modules | Description | CPython_Code |
---|---|---|
pickle | Python object serialization | Lib/pickle.py |
copyreg | Register pickle support functions | Lib/copyreg.py |
shelve | Python object persistence | Lib/shelve.py |
marshal | Internal Python object serialization | |
dbm | Interfaces to Unix databases | Lib/dbm/__init__.py |
sqlite3 | DB-API 2.0 interface for SQLite databases | Lib/sqlite3/ |
Data Compression and Archiving
Modules | Description | CPython_Code |
---|---|---|
zlib | Compression compatible with gzip | |
gzip | Support for gzip files | Lib/gzip.py |
bz2 | Support for bzip2 compression | Lib/bz2.py |
lzma | Compression using the LZMA algorithm | Lib/lzma.py |
zipfile | Work with ZIP archives | Lib/zipfile.py |
tarfile | Read and write tar archive files | Lib/tarfile.py |
File Formats
Modules | Description | CPython_Code |
---|---|---|
csv | CSV File Reading and Writing | Lib/csv.py |
configparser | Configuration file parser | Lib/configparser.py |
netrc | netrc file processing | Lib/netrc.py |
xdrlib | Encode and decode XDR data | Lib/xdrlib.py |
plistlib | Generate and parse Mac OS X .plist files | Lib/plistlib.py |
Cryptographic Services
Modules | Description | CPython_Code |
---|---|---|
hashlib | Secure hashes and message digests | Lib/hashlib.py |
hmac | Keyed-Hashing for Message Authentication | Lib/hmac.py |
secrets | Generate secure random numbers for managing secrets | Lib/secrets.py |
Generic Operating System Services
Modules | Description | CPython_Code |
---|---|---|
os | Miscellaneous operating system interfaces | Lib/os.py |
io | Core tools for working with streams | Lib/io.py |
time | Time access and conversions | |
argparse | Parser for command-line options, arguments and sub-commands | Lib/argparse.py |
getopt | C-style parser for command line options | Lib/getopt.py |
logging | Logging facility for Python | Lib/logging/__init__.py |
logging.config | Logging configuration | Lib/logging/config.py |
logging.handlers | Logging handlers | Lib/logging/handlers.py |
getpass | Portable password input | Lib/getpass.py |
curses | Terminal handling for character-cell displays | |
curses.textpad | Text input widget for curses programs | |
curses.ascii | Utilities for ASCII characters | |
curses.panel | A panel stack extension for curses | |
platform | Access to underlying platform’s identifying data | Lib/platform.py |
errno | Standard errno system symbols | |
ctypes | A foreign function library for Python |
Concurrent Execution
Modules | Description | CPython_Code |
---|---|---|
threading | Thread-based parallelism | Lib/threading.py |
multiprocessing | Process-based parallelism | Lib/multiprocessing/ |
concurrent | ||
concurrent.futures | Launching parallel tasks | Lib/concurrent/futures/thread.py Lib/concurrent/futures/process.py |
subprocess | Subprocess management | Lib/subprocess.py |
sched | Event scheduler | Lib/sched.py |
queue | A synchronized queue class | Lib/queue.py |
dummy_threading | Drop-in replacement for the threading module | Lib/dummy_threading.py |
_thread | Low-level threading API | |
_dummy_thread | Drop-in replacement for the _thread module | Lib/_dummy_thread.py |
Interprocess Communication and Networking
Modules | Description | CPython_Code |
---|---|---|
socket | Low-level networking interface | Lib/socket.py |
ssl | TLS/SSL wrapper for socket objects | Lib/ssl.py |
select | Waiting for I/O completion | |
selectors | High-level I/O multiplexing | Lib/selectors.py |
asyncio | Asynchronous I/O, event loop, coroutines and tasks | Lib/asyncio/ |
asyncore | Asynchronous socket handler | Lib/asyncore.py |
asynchat | Asynchronous socket command/response handler | Lib/asynchat.py |
signal | Set handlers for asynchronous events | |
mmap | Memory-mapped file support |
Internet Data Handling
Modules | Description | CPython_Code |
---|---|---|
An email and MIME handling package | Lib/email/__init__.py | |
json | JSON encoder and decoder | Lib/json/__init__.py |
mailcap | Mailcap file handling | Lib/mailcap.py |
mailbox | Manipulate mailboxes in various formats | Lib/mailbox.py |
mimetypes | Map filenames to MIME types | Lib/mimetypes.py |
base64 | Base16, Base32, Base64, Base85 Data Encodings | Lib/base64.py |
binhex | Encode and decode binhex4 files | Lib/binhex.py |
binascii | Convert between binary and ASCII | |
quopri | Encode and decode MIME quoted-printable data | Lib/quopri.py |
uu | Encode and decode uuencode files | Lib/uu.py |
Structured Markup Processing Tools
Modules | Description | CPython_Code |
---|---|---|
html | HyperText Markup Language support | Lib/html/__init__.py |
html.parser | Simple HTML and XHTML parser | Lib/html/parser.py |
html.entities | Definitions of HTML general entities | Lib/html/entities.py |
xml | XML Processing Modules | Lib/xml/ |
xml.etree.ElementTree | The ElementTree XML API | Lib/xml/etree/ElementTree.py |
xml.dom | The Document Object Model API | Lib/xml/dom/__init__.py |
xml.dom.minidom | Minimal DOM implementation | Lib/xml/dom/minidom.py |
xml.dom.pulldom | Support for building partial DOM trees | Lib/xml/dom/pulldom.py |
xml.sax | Support for SAX2 parsers | Lib/xml/sax/__init__.py |
xml.sax.handler | Base classes for SAX handlers | Lib/xml/sax/handler.py |
xml.sax.saxutils | SAX Utilities | Lib/xml/sax/saxutils.py |
xml.sax.xmlreader | Interface for XML parsers | Lib/xml/sax/xmlreader.py |
xml.parsers.expat | Fast XML parsing using Expat |
Internet Protocols and Support
Modules | Description | CPython_Code |
---|---|---|
webbrowser | Convenient Web-browser controller | Lib/webbrowser.py |
cgi | Common Gateway Interface support | Lib/cgi.py |
cgitb | Traceback manager for CGI scripts | Lib/cgitb.py |
wsgiref | WSGI Utilities and Reference Implementation | |
urllib | URL handling modules | Lib/urllib/ |
urllib.request | Extensible library for opening URLs | Lib/urllib/request.py |
urllib.response | Response classes used by urllib | |
urllib.parse | Parse URLs into components | Lib/urllib/parse.py |
urllib.error | Exception classes raised by urllib.request | Lib/urllib/error.py |
urllib.robotparser | Parser for robots.txt | Lib/urllib/robotparser.py |
http | HTTP modules | Lib/http/__init__.py |
http.client | HTTP protocol client | Lib/http/client.py |
ftplib | FTP protocol client | Lib/ftplib.py |
poplib | POP3 protocol client | Lib/poplib.py |
imaplib | IMAP4 protocol client | Lib/imaplib.py |
nntplib | NNTP protocol client | Lib/nntplib.py |
smtplib | SMTP protocol client | Lib/smtplib.py |
smtpd | SMTP Server | Lib/smtpd.py |
telnetlib | Telnet client | Lib/telnetlib.py |
uuid | UUID objects according to RFC 4122 | Lib/uuid.py |
socketserver | A framework for network servers | Lib/socketserver.py |
http.server | HTTP servers | Lib/http/server.py |
http.cookies | HTTP state management | Lib/http/cookies.py |
http.cookiejar | Cookie handling for HTTP clients | Lib/http/cookiejar.py |
xmlrpc | XMLRPC server and client modules | |
xmlrpc.client | XML-RPC client access | Lib/xmlrpc/client.py |
xmlrpc.server | Basic XML-RPC servers | Lib/xmlrpc/server.py |
ipaddress | IPv4/IPv6 manipulation library | Lib/ipaddress.py |
Multimedia Services
Modules | Description | CPython_Code |
---|---|---|
audioop | Manipulate raw audio data | |
aifc | Read and write AIFF and AIFC files | Lib/aifc.py |
sunau | Read and write Sun AU files | Lib/sunau.py |
wave | Read and write WAV files | Lib/wave.py |
chunk | Read IFF chunked data | Lib/chunk.py |
colorsys | Conversions between color systems | Lib/colorsys.py |
imghdr | Determine the type of an image | Lib/imghdr.py |
sndhdr | Determine type of sound file | Lib/sndhdr.py |
ossaudiodev | Access to OSS-compatible audio devices |
Internationalization
Modules | Description | CPython_Code |
---|---|---|
gettext | Multilingual internationalization services | Lib/gettext.py |
locale | Internationalization services | Lib/locale.py |
Program Frameworks
Modules | Description | CPython_Code |
---|---|---|
turtle | Turtle graphics | Lib/turtle.py |
cmd | Support for line-oriented command interpreters | Lib/cmd.py |
shlex | Simple lexical analysis | Lib/shlex.py |
Graphical User Interfaces with Tk
Modules | Description | CPython_Code |
---|---|---|
tkinter | Python interface to Tcl/Tk | Lib/tkinter/__init__.py |
tkinter.ttk | Tk themed widgets | Lib/tkinter/ttk.py |
tkinter.tix | Extension widgets for Tk | Lib/tkinter/tix.py |
tkinter.scrolledtext | Scrolled Text Widget | Lib/tkinter/scrolledtext.py |
Development Tools
Modules | Description | CPython_Code |
---|---|---|
typing | Support for type hints | Lib/typing.py |
pydoc | Documentation generator and online help system | Lib/pydoc.py |
doctest | Test interactive Python examples | Lib/doctest.py |
unittest | Unit testing framework | Lib/unittest/__init__.py |
unittest.mock | mock object library | Lib/unittest/mock.py |
2to3 | Automated Python 2 to 3 code translation | |
test | Regression tests package for Python | |
test.support | Utilities for the Python test suite |
Debugging and Profiling
Modules | Description | CPython_Code |
---|---|---|
bdb | Debugger framework | Lib/bdb.py |
faulthandler | Dump the Python traceback | |
pdb | The Python Debugger | Lib/pdb.py |
cprofile profile |
The Python Profilers | Lib/profile.py Lib/pstats.py |
timeit | Measure execution time of small code snippets | Lib/timeit.py |
trace | Trace or track Python statement execution | Lib/trace.py |
tracemalloc | Trace memory allocations | Lib/tracemalloc.py |
Software Packaging and Distribution
Modules | Description | CPython_Code |
---|---|---|
distutils | Building and installing Python modules | |
ensurepip | Bootstrapping the pip installer | |
venv | Creation of virtual environments | Lib/venv/ |
zipapp | Manage executable python zip archives | Lib/zipapp.py |
Python Runtime Services
Modules | Description | CPython_Code |
---|---|---|
sys | System-specific parameters and functions | |
sysconfig | Provide access to Python’s configuration information | Lib/sysconfig.py |
builtins | Built-in objects | |
__main__ | Top-level script environment | |
warnings | Warning control | Lib/warnings.py |
contextlib | Utilities for with-statement contexts | Lib/contextlib.py |
abc | Abstract Base Classes | Lib/abc.py |
atexit | Exit handlers | |
traceback | Print or retrieve a stack traceback | Lib/traceback.py |
__future__ | Future statement definitions | Lib/__future__.py |
gc | Garbage Collector interface | |
inspect | Inspect live objects | Lib/inspect.py |
site | Site-specific configuration hook | Lib/site.py |
fpectl | Floating point exception control |
Custom Python Interpreters
Modules | Description | CPython_Code |
---|---|---|
code | Interpreter base classes | Lib/code.py |
codeop | Compile Python code | Lib/codeop.py |
Importing Modules
Modules | Description | CPython_Code |
---|---|---|
zipimport | Import modules from Zip archives | |
pkgutil | Package extension utility | Lib/pkgutil.py |
modulefinder | Find modules used by a script | Lib/modulefinder.py |
runpy | Locating and executing Python modules | Lib/runpy.py |
importlib | The implementation of import | Lib/importlib/__init__.py |
Python Language Services
Modules | Description | CPython_Code |
---|---|---|
parser | Access Python parse trees | |
ast | Abstract Syntax Trees | Lib/ast.py |
symtable | Access to the compiler’s symbol tables | Lib/symtable.py |
symbol | Constants used with Python parse trees | Lib/symbol.py |
token | Constants used with Python parse trees | Lib/token.py |
keyword | Testing for Python keywords | Lib/keyword.py |
tokenize | Tokenizer for Python source | Lib/tokenize.py |
tabnanny | Detection of ambiguous indentation | Lib/tabnanny.py |
pyclbr | Python class browser support | Lib/pyclbr.py |
py_compile | Compile Python source files | Lib/py_compile.py |
compileall | Byte-compile Python libraries | Lib/compileall.py |
dis | Disassembler for Python bytecode | Lib/dis.py |
pickletools | Tools for pickle developers | Lib/pickletools.py |
Miscellaneous Services
Modules | Description | CPython_Code |
---|---|---|
formatter | Generic output formatting |
MS Windows Specific Services
Modules | Description | CPython_Code |
---|---|---|
msilib | Read and write Microsoft Installer files | Lib/msilib/__init__.py |
msvcrt | Useful routines from the MS VC++ runtime | |
winreg | Windows registry access | |
winsound | Sound-playing interface for Windows |
Unix Specific Services
Modules | Description | CPython_Code |
---|---|---|
posix | The most common POSIX system calls | |
pwd | The password database | |
spwd | The shadow password database | |
grp | The group database | |
crypt | Function to check Unix passwords | Lib/crypt.py |
termios | POSIX style tty control | |
tty | Terminal control functions | Lib/tty.py |
pty | Pseudo-terminal utilities | Lib/pty.py |
fcntl | The fcntl and ioctl system calls | |
pipes | Interface to shell pipelines | Lib/pipes.py |
resource | Resource usage information | |
nis | Interface to Sun’s NIS (Yellow Pages) | |
syslog | Unix syslog library routines |
Superseded Modules
The following modules are deprecated and only kept for backwards compatibility. They have been superseded by other modules.
Modules | Description | CPython_Code |
---|---|---|
optparse | Parser for command line options | Lib/optparse.py |
imp | Access the import internals | Lib/imp.py |
Undocumented Modules
These modules are used to implement the os.path
module:
Modules | Description | CPython_Code |
---|---|---|
ntpath | Implementation of os.path on Win32 and Win64 platforms. |
|
posixpath | Implementation of os.path on POSIX. |
Other Python Libraries
Refer to the Python Package Index (PyPI) for packages used by the Python programming language.
Here is 20 Python libraries you can’t live without:
Libraries | Description |
---|---|
Requests | The most famous HTTP library written by kenneth reitz. It’s a must have for every python developer. |
Scrapy | If you are involved in WebScraping then this is a must have library for you. After using this library you won’t use any other. |
wxPython | A GUI toolkit for Python. I have primarily used it in place of Tkinter. You will really love it. |
Pillow | A friendly fork of PIL (Python Imaging Library). It is more user friendly than PIL and is a must have for anyone who works with images. |
SQLAlchemy | A database library. Many love it and many hate it. The choice is yours. |
BeautifulSoup | I know it’s slow but this XML and HTML parsing library is very useful for beginners. |
Twisted | The most important tool for any network application developer. It has a very beautiful API and is used by a lot of famous python developers. |
NumPy | How can we leave this very important library? It provides some advance math functionalities to python. |
SciPy | When we talk about NumPy then we have to talk about SciPy. It is a library of algorithms and mathematical tools for python and has caused many scientists to switch from ruby to python. Also see Scientific Computing via SciPy. |
matplotlib | A numerical plotting library. It is very useful for any data scientist or any data analyzer. |
Pygame | Which developer does not like to play games and develop them? This library will help you achieve your goal of 2D game development. |
Pyglet | A 3D animation and game creation engine. This is the engine in which the famous python port of Minecraft was made. |
pyQT | A GUI toolkit for python. It is my second choice after wxPython for developing GUI’s for my python scripts. |
pyGtk | Another python GUI library. It is the same library in which the famous Bittorrent client is created. |
Scapy | A packet sniffer and analyzer for Python made in Python. |
PyWin32 | A Python library which provides some useful methods and classes for interacting with windows. |
NLTK | Natural Language Toolkit – I realize most people won’t be using this one, but it’s generic enough. It is a very useful library if you want to manipulate strings. But it’s capacity is beyond that. Do check it out. |
nose | A testing framework for Python. It is used by millions of Python developers. It is a must have if you do test driven development. |
SymPy | SymPy can do algebraic evaluation, differentiation, expansion, complex numbers, etc. It is contained in a pure Python distribution. |
IPython | I just can’t stress enough how useful this tool is. It is a Python prompt on steroids. It has completion, history, shell capabilities, and a lot more. Make sure that you take a look at it. |
How to Install Python Packages
The pip is the PyPA recommended tool for installing Python packages. It is the package installer for Python. You can use pip to install packages from the Python Package Index and other indexes. Refer to:
Install pip
Install pip on Python 2.x:
chenwx@chenwx:~ $ sudo apt install python-pip
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
python-pip-whl
Recommended packages:
build-essential python-all-dev python-setuptools python-wheel
The following NEW packages will be installed:
python-pip python-pip-whl
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 1803 kB of archives.
After this operation, 2526 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://mirrors.aliyun.com/ubuntu bionic-updates/universe amd64 python-pip-whl all 9.0.1-2.3~ubuntu1 [1652 kB]
Get:2 http://mirrors.aliyun.com/ubuntu bionic-updates/universe amd64 python-pip all 9.0.1-2.3~ubuntu1 [151 kB]
Fetched 1803 kB in 6s (292 kB/s)
Selecting previously unselected package python-pip-whl.
(Reading database ... 294441 files and directories currently installed.)
Preparing to unpack .../python-pip-whl_9.0.1-2.3~ubuntu1_all.deb ...
Unpacking python-pip-whl (9.0.1-2.3~ubuntu1) ...
Selecting previously unselected package python-pip.
Preparing to unpack .../python-pip_9.0.1-2.3~ubuntu1_all.deb ...
Unpacking python-pip (9.0.1-2.3~ubuntu1) ...
Setting up python-pip-whl (9.0.1-2.3~ubuntu1) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Setting up python-pip (9.0.1-2.3~ubuntu1) ...
chenwx@chenwx:~ $ python -m pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)
chenwx@chenwx:~ $ pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)
Install pip on Python 3.x:
chenwx@chenwx:~ $ sudo apt install python3-pip
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
python3-distutils python3-lib2to3
Recommended packages:
build-essential python3-dev python3-setuptools python3-wheel
The following NEW packages will be installed:
python3-distutils python3-lib2to3 python3-pip
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 331 kB of archives.
After this operation, 2714 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 python3-lib2to3 all 3.6.7-1~18.04 [76.5 kB]
Get:2 http://mirrors.aliyun.com/ubuntu bionic-updates/main amd64 python3-distutils all 3.6.7-1~18.04 [141 kB]
Get:3 http://mirrors.aliyun.com/ubuntu bionic-updates/universe amd64 python3-pip all 9.0.1-2.3~ubuntu1 [114 kB]
Fetched 331 kB in 3s (113 kB/s)
Selecting previously unselected package python3-lib2to3.
(Reading database ... 294575 files and directories currently installed.)
Preparing to unpack .../python3-lib2to3_3.6.7-1~18.04_all.deb ...
Unpacking python3-lib2to3 (3.6.7-1~18.04) ...
Selecting previously unselected package python3-distutils.
Preparing to unpack .../python3-distutils_3.6.7-1~18.04_all.deb ...
Unpacking python3-distutils (3.6.7-1~18.04) ...
Selecting previously unselected package python3-pip.
Preparing to unpack .../python3-pip_9.0.1-2.3~ubuntu1_all.deb ...
Unpacking python3-pip (9.0.1-2.3~ubuntu1) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Setting up python3-lib2to3 (3.6.7-1~18.04) ...
Setting up python3-distutils (3.6.7-1~18.04) ...
Setting up python3-pip (9.0.1-2.3~ubuntu1) ...
chenwx@chenwx:~ $ python3 -m pip --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)
chenwx@chenwx:~ $ pip3 --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)
Here is a list of pip’s sub-commands:
chenwx@chenwx:~ $ python3 -m pip
Usage:
/usr/bin/python3 -m pip <command> [options]
Commands:
install Install packages.
download Download packages.
uninstall Uninstall packages.
freeze Output installed packages in requirements format.
list List installed packages.
show Show information about installed packages.
check Verify installed packages have compatible dependencies.
search Search PyPI for packages.
wheel Build wheels from your requirements.
hash Compute hashes of package archives.
completion A helper command used for command completion.
help Show help for commands.
General Options:
-h, --help Show help.
--isolated Run pip in an isolated mode, ignoring environment variables and user configuration.
-v, --verbose Give more output. Option is additive, and can be used up to 3 times.
-V, --version Show version and exit.
-q, --quiet Give less output. Option is additive, and can be used up to 3 times (corresponding to
WARNING, ERROR, and CRITICAL logging levels).
--log <path> Path to a verbose appending log.
--proxy <proxy> Specify a proxy in the form [user:passwd@]proxy.server:port.
--retries <retries> Maximum number of retries each connection should attempt (default 5 times).
--timeout <sec> Set the socket timeout (default 15 seconds).
--exists-action <action> Default action when a path already exists: (s)witch, (i)gnore, (w)ipe, (b)ackup,
(a)bort.
--trusted-host <hostname> Mark this host as trusted, even though it does not have valid or any HTTPS.
--cert <path> Path to alternate CA bundle.
--client-cert <path> Path to SSL client certificate, a single file containing the private key and the
certificate in PEM format.
--cache-dir <dir> Store the cache data in <dir>.
--no-cache-dir Disable the cache.
--disable-pip-version-check
Don't periodically check PyPI to determine whether a new version of pip is available for
download. Implied with --no-index.
Install Python Packages via pip
Take installing package numPy for instance:
chenwx@chenwx:~ $ python3 -m pip --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)
chenwx@chenwx:~ $ python3 -m pip list --format=columns
Package Version
--------------------- ---------
apt-clone 0.2.1
apturl 0.5.2
attrs 19.1.0
backcall 0.1.0
beautifulsoup4 4.6.0
bleach 3.1.0
Brlapi 0.6.6
certifi 2018.1.18
chardet 3.0.4
command-not-found 0.3
configobj 5.0.6
cupshelpers 1.0
cycler 0.10.0
decorator 4.4.0
defer 1.0.6
defusedxml 0.6.0
entrypoints 0.3
httplib2 0.9.2
idna 2.6
ipykernel 5.1.1
ipython 7.5.0
ipython-genutils 0.2.0
ipywidgets 7.4.2
jedi 0.13.3
Jinja2 2.10.1
jsonschema 3.0.1
jupyter 1.0.0
jupyter-client 5.2.4
jupyter-console 6.0.0
jupyter-core 4.4.0
kiwisolver 1.1.0
louis 3.5.0
macaroonbakery 1.1.3
Mako 1.0.7
MarkupSafe 1.1.1
matplotlib 3.0.3
meld 3.18.0
mistune 0.8.4
mpmath 1.1.0
nbconvert 5.5.0
nbformat 4.4.0
nemo-emblems 4.0.2
netifaces 0.10.4
nose 1.3.7
notebook 5.7.8
numpy 1.16.3
onboard 1.4.1
PAM 0.4.2
pandas 0.24.2
pandocfilters 1.4.2
parso 0.4.0
pexpect 4.7.0
pickleshare 0.7.5
Pillow 5.1.0
pip 9.0.1
prometheus-client 0.6.0
prompt-toolkit 2.0.9
protobuf 3.0.0
psutil 5.4.2
ptyprocess 0.6.0
pycairo 1.16.2
pycrypto 2.6.1
pycups 1.9.73
pycurl 7.43.0.1
Pygments 2.4.0
pygobject 3.26.1
PyICU 1.9.8
pyinotify 0.9.6
pymacaroons 0.13.0
PyNaCl 1.1.2
pyparsing 2.4.0
pyRFC3339 1.0
pyrsistent 0.15.2
python-apt 1.6.4
python-dateutil 2.8.0
python-debian 0.1.32
python-xapp 1.4.0
python-xlib 0.20
pytz 2019.1
pyxdg 0.25
PyYAML 3.12
pyzmq 18.0.1
qtconsole 4.4.4
reportlab 3.4.0
requests 2.18.4
requests-unixsocket 0.1.5
scipy 1.2.1
Send2Trash 1.5.0
sessioninstaller 0.0.0
setproctitle 1.1.10
setuptools 41.0.1
six 1.12.0
sympy 1.4
system-service 0.3
systemd-python 234
terminado 0.8.2
testpath 0.4.2
tornado 6.0.2
traitlets 4.3.2
ubuntu-drivers-common 0.0.0
ufw 0.36
urllib3 1.22
wcwidth 0.1.7
webencodings 0.5.1
widgetsnbextension 3.4.2
xkit 0.0.0
chenwx@chenwx:~ $ python3 -m pip search numpy
numpy (1.16.3) - NumPy is the fundamental package for array computing with Python.
INSTALLED: 1.16.3 (latest)
numpy-utils (0.1.3) - NumPy utilities.
numpy-cloud (0.0.5) - Numpy in the cloud
numpy-turtle (0.1) - Turtle graphics with NumPy
numpy-sugar (1.5.0) - Missing NumPy functionalities
root-numpy (4.8.0) - The interface between ROOT and NumPy
msgpack-numpy (0.4.4.3) - Numpy data serialization using msgpack
numpy-quaternion (2019.3.21.14.22.55) - Add built-in support for quaternions to numpy
numpy-partition (1.18.9) - SQL PARTITION BY and window functions for NumPy
mapchete-numpy (0.1) - Mapchete NumPy read/write extension
numpy-posit (1.15.2.0.0.1.dev2) - posit (unum type III) integrated NumPy.
intel-numpy (1.15.1) - NumPy optimized with Intel(R) MKL library
numpy-mkl (1.10.2) - NumPy: array processing for numbers, strings, records, and objects.
ccv-numpy (0.0.2) - Wrapper module for ccv using numpy arrays interface
django-numpy (1.0.1) - Application for Django projects that adds some utilities and integration tools with Numpy.
BSON-NumPy (0.1) - Module for converting directly from BSON to NumPy ndarrays and vice versa
dicom-numpy (0.1.6) - Extract image data into a 3D numpy array from a set of DICOM files.
dl-with-numpy (0.0.3) - Simple deep learning with numpy
numpy-html (0.0.3) - A simple table renderer for numpy arrays. Provides a rich display hook for use with Jupyter Lab / Notebook.
numpy-indexed (0.3.5) - This package contains functionality for indexed operations on numpy ndarrays, providing efficient vectorized functionality such as grouping and set operations.
numpy-image-widget (2019.1.6) - An easy-to-use Jupyter widget for displaying images from Numpy data arrays
django-numpy-json-encoder (0.1.3) - Django JSON encoder with numpy support.
faster-numpy (0.1.2) -
numpy-stl (2.10.1) - Library to make reading, writing and modifying both binary and ascii STL files easy.
hypothesis-numpy (2.0.0) - Adds support for generating datetime to Hypothesis
numpy-groupies (0.9.9) - Optimised tools for group-indexing operations: aggregated sum and more.
npplus (0.9.8) - Enhancements to Numpy
o243kgmvs924mtvd034mg (0.01) - NumPy Testing
np (1.0.2) - np = numpy++: numpy with added convenience functionality
maskedarray (0.1) - NumPy Masked Array
jumpy (0.2.4) - Numpy and nd4j interop
nparray (1.0.0) - Numpy Array Helpers
oldnumeric (1.0.4) - The oldnumeric numpy package
nptyping (0.2.0) - Type hints for Numpy
match-arrays (0.0.1) - Matching of Numpy arrays
vidsrc (2019.1.1) - Video Frameserver for Numpy
ndarray (0.8) - Tools for numpy ndarray
wrapnumpy3 (0.0.3) - Wrapper of Numpy in Python3.
sqlite3ndarray (0.3.1) - sqlite3 helpers for numpy ndarrays
axarray (0.1.1) - numpy array with labeled axes
somber (2.0.1) - Self-Organizing Maps in Numpy
gsn_numpy_util (0.1.0) - Toolbox for working with Numpy arrays.
homog (0.1.14) - Utilities for Homogeneous Coordinates with Numpy
npstreams (1.6) - Streaming operations on NumPy arrays
sym2num (0.1.dev2) - Sympy to numpy code generator.
blocked-matrix-utils (1.0.1) - NumPy wrappers for blocked matrices
dynarray (0.1.3) - Dynamically growable numpy arrays.
zappy (0.2.0) - Distributed processing with NumPy and Zarr.
permanent (1.1.1) - Calculates the permanent of a Numpy matrix
pstar (0.1.6) - pstar: numpy for arbitrary data
sompy (0.1.1) - Numpy based SOM Library
spartan (0.06) - Distributed Numpy-like arrays.
fastjet (0.0.3) - The interface between FastJet and NumPy
npdoc (1.0.1) - In-notebook NumPy man pages
gumpy (0.0.1) - GPU-accelerated NumPy API
wendelin.core (0.12) - Out-of-core NumPy arrays
sampyl-mcmc (0.3) - MCMC Samplers in Python & Numpy
gnuplotlib (0.29) - Gnuplot-based plotting for numpy
multipolyfit (0.0.1) - Multivariate Polynomial fitting with NumPy
nppretty (1.0.0) - Pretty printer for numpy arrays.
numpythia (1.1.0) - The interface between PYTHIA and NumPy
PyUblas (2017.1) - Seamless Numpy-UBlas interoperability
numm (0.3) - numpy-based multimedia library
ntype (0.0.1) - Type Handler using NumPy
fastfunc (0.1.7) - Fast numpy ufunc operations
pyjet (1.5.0) - The interface between FastJet and NumPy
snuggs (1.4.6) - Snuggs are s-expressions for Numpy
array_collections (0.1.7) - A collection of numpy ndarray subclasses.
pyache (0.1.0) - A simple numpy caching library
opt-einsum (2.3.2) - Optimizing numpys einsum function
arkouda (0.0.0) - A numpy replacement backed by Chapel
fastremap (1.5.2) - Remap and renumber numpy arrays.
fewerbytes (0.0.1) - Compression techniques for numpy arrays
numpy_ringbuffer (0.2.1) - Ring buffer implementation for numpy
bohrium (0.11.0.post15) - Bohrium Python/NumPy Backend
afnumpy (1.3) - A GPU-ready drop-in replacement for numpy
ComStats (0.1.0) - Do combinatorial statistics on numpy ndarrays
xcy-Zscore (0.0.5) - A small tool of Zscore based on numpy
marshmallow-numerical (0.0.2) - Marshmallow Schema generator for pandas and numpy
numpyx (0.3.0) - Utility functions for numpy, written in cython
sortednp (0.2.1) - Merge and intersect sorted numpy arrays.
numphy (0.0.2) - Physics objects backed by NumPy and/or TensorFlow.
vector-shortcuts (0.2.0) - Vector and linear algebra toolbelt for NumPy
zmqnumpy (0.1) - numpy array over zmq sockets
pynetcdf (0.7) - Standalone Scientific.IO.NetCDF (2.4.11) built for NumPy
npcb (0.0.1) - Copy NumPy arrays via the clipboard
spacegrids (1.9) - numpy array with grids and associated operations
ndf (0.1.4) - NumPy based deep learning framework
multipletau (0.3.3) - A multiple-tau algorithm for Python/NumPy
seabred (0.0.1) - Seaborn-like plots for numpy arrays
numexpr (2.6.9) - Fast numerical expression evaluator for NumPy
tensorfont (0.0.2) - Turn font glyphs into numpy arrays
numpy_display (1.2.0) - Formats numpy matrices in an IPython Notebook
nnlib (0.2) - numpy only neural network library
matplottery (1.0.4) - Nicer histograms with numpy and plotting with matplotlib
geog (0.0.2) - Numpy-based vectorized geospatial functions
stft (0.5.2) - Short Time Fourier transform for NumPy.
hdrpy (0.3.3) - HDR histogram implementation based on numpy
np2d (0.1.1) - Common 2-D NumPy operations.
pySLALIB (1.0.4) - f2py and numpy based wrappers for SLALIB
chenwx@chenwx:~ $ python3 -m pip install numpy
Collecting numpy
Downloading https://files.pythonhosted.org/packages/e0/b5/63b79fe426433fa1cd110eb04a94ec0c6967e56e5f57c98caf455a5fb6e2/numpy-1.16.1-cp27-cp27mu-manylinux1_x86_64.whl (17.0MB)
100% |################################| 17.0MB 51kB/s
Installing collected packages: numpy
Successfully installed numpy-1.16.1
chenwx@chenwx:~ $ python3 -m pip show numpy
Name: numpy
Version: 1.16.1
Summary: NumPy is the fundamental package for array computing with Python.
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: None
License: BSD
Location: /home/chenwx/.local/lib/python2.7/site-packages
Requires:
chenwx@chenwx:~ $ python3
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.eye(4)
array([[1., 0., 0., 0.],
[0., 1., 0., 0.],
[0., 0., 1., 0.],
[0., 0., 0., 1.]])
Python Quick Reference
Python Code Conventions
PEP 8: Style Guide for Python Code
For Python, PEP 8: Style Guide for Python Code has emerged as the style guide that most projects adhere to; it promotes a very readable and eye-pleasing coding style. Every Python developer should read it at some point; here are the most important points extracted for you:
-
Use 4-space indentation, and no tabs. 4 spaces are a good compromise between small indentation (allows greater nesting depth) and large indentation (easier to read). Tabs introduce confusion, and are best left out.
-
Wrap lines so that they don’t exceed 79 characters. This helps users with small displays and makes it possible to have several code files side-by-side on larger displays.
-
Use blank lines to separate functions and classes, and larger blocks of code inside functions.
-
When possible, put comments on a line of their own.
-
Use docstrings.
-
Use spaces around operators and after commas, but not directly inside bracketing constructs:
a = f(1, 2) + g(3, 4)
. -
Name your classes and functions consistently; the convention is to use CamelCase for classes and lower_case_with_underscores for functions and methods. Always use
self
as the name for the first method argument. -
Don’t use fancy encodings if your code is meant to be used in international environments. Python’s default, UTF-8, or even plain ASCII work best in any case. Likewise, don’t use non-ASCII characters in identifiers if there is only the slightest chance people speaking a different language will read or maintain the code.
autopep8
The tool autopep8
is used to format the Python code to follow rules defined in PEP 8: Style Guide for Python Code:
$ pip install --upgrade autopep8
$ autopep8 --in-place --aggressive <filename>
Google Python Style Guide
Python Open Source Projects
Python Developers Survey
Python FAQ
Find the location of your user site-packages directory:
>>> import site
>>> site.getusersitepackages()
'/home/ewaadex/.local/lib/python3.5/site-packages'
Python IDEs
- Python IDEs
- PyCharm Community Edition, see PyCharm Usage
- Eclipse + Pydev
- VIM
- Wing Python IDE
- Spyder Python
- Komodo IDE
- Python Tools for Visual Studio (PTVS)
- The Eric Python IDE
- Sublime Text 3 / Anaconda Python IDE
- Python Programming In Emacs
- PyScripter
- Interactive Editor for Python
- UliPad
Python Books
- Programming Python, 4th Edition
- Python in a Nutshell
- Learning Python, 4th Edition
- Python Cookbook
- Hands-On Python: A Tutorial Introduction for Beginners
- Python Pocket Reference, 4th Edition
- Python Standard Library
-
Python Module of the Week
- Python on Windows documentation
- The Algorithms: Python
References
- Python Official Site
- Python Software Foundation (PSF)
- Index of Python Enhancement Proposals (PEPs)
- Beginner’s Guide to Python
- Python 2.x Tutorial
- Python 3.x Tutorial
- Python 2.x Documentation
- Python 3.x Documentation
- Python HOWTOs
- Python 2.x Standard Library
- Python 3.x Standard Library
- Python 2.x Module Index
- Python 3.x Module Index
- Python Package Index (PyPI)
- Python Packaging Authority
- Python Packaging User Guide
- PyCrumbs
- Python Reference on ShareTechnote
- Learn Python Programming