Installing Python Modules

Installing Python Modules

Email

[e-mail]

As a popular open source development project, Python has an active supporting community of contributors and users that also make their software available for other Python developers to use under open source license terms.

This allows Python users to share and collaborate effectively, benefiting from the solutions others have already created to common (and sometimes even rare!) problems, as well as potentially contributing their own solutions to the common pool.

This guide covers the installation part of the process. For a guide to creating and sharing your own Python projects, refer to the distribution guide.

Note

For corporate and other institutional users, be aware that many organisations have their own policies around using and contributing to open source software. Please take such policies into account when making use of the distribution and installation tools provided with Python.

Key terms

• pip is the preferred installer program. Starting with Python 3.4, it is included by default with the Python binary installers.

• A virtual environment is a semi-isolated Python environment that allows packages to be installed for use by a particular application, rather than being installed system wide.

• venv is the standard tool for creating virtual environments, and has been part of Python since Python 3.3. Starting with Python 3.4, it defaults to installing pip into all created virtual environments.

• virtualenv is a third party alternative (and predecessor) to venv. It allows virtual environments to be used on versions of Python prior to 3.4, which either don’t provide venv at all, or aren’t able to automatically install pip into created environments.

• The Python Packaging Index is a public repository of open source licensed packages made available for use by other Python users.

• the Python Packaging Authority is the group of developers and documentation authors responsible for the maintenance and evolution of the standard packaging tools and the associated metadata and file format standards. They maintain a variety of tools, documentation, and issue trackers on both GitHub and Bitbucket.

• distutils is the original build and distribution system first added to the Python standard library in 1998. While direct use of distutils is being phased out, it still laid the foundation for the current packaging and distribution infrastructure, and it not only remains part of the standard library, but its name lives on in other ways (such as the name of the mailing list used to coordinate Python packaging standards development).

Changed in version 3.5: The use of venv is now recommended for creating virtual environments.

See also

Python Packaging User Guide: Creating and using virtual environments

Basic usage

The standard packaging tools are all designed to be used from the command line.

The following command will install the latest version of a module and its dependencies from the Python Packaging Index:

python -m pip install SomePackage

Note

For POSIX users (including Mac OS X and Linux users), the examples in this guide assume the use of a virtual environment.

For Windows users, the examples in this guide assume that the option to adjust the system PATH environment variable was selected when installing Python.

It’s also possible to specify an exact or minimum version directly on the command line. When using comparator operators such as >, < or some other special character which get interpreted by shell, the package name and the version should be enclosed within double quotes:

python -m pip install SomePackage==1.0.4    # specific version
python -m pip install "SomePackage>=1.0.4"  # minimum version

Normally, if a suitable module is already installed, attempting to install it again will have no effect. Upgrading existing modules must be requested explicitly:

python -m pip install --upgrade SomePackage

More information and resources regarding pip and its capabilities can be found in the Python Packaging User Guide.

Creation of virtual environments is done through the venv module. Installing packages into an active virtual environment uses the commands shown above.

See also

Python Packaging User Guide: Installing Python Distribution Packages

How do I …?

These are quick answers or links for some common tasks.

… install pip in versions of Python prior to Python 3.4?

Python only started bundling pip with Python 3.4. For earlier versions, pip needs to be “bootstrapped” as described in the Python Packaging User Guide.

See also

Python Packaging User Guide: Requirements for Installing Packages

… install packages just for the current user?

Passing the --user option to python -m pip install will install a package just for the current user, rather than for all users of the system.

… install scientific Python packages?

A number of scientific Python packages have complex binary dependencies, and aren’t currently easy to install using pip directly. At this point in time, it will often be easier for users to install these packages by other means rather than attempting to install them with pip.

See also

Python Packaging User Guide: Installing Scientific Packages

… work with multiple versions of Python installed in parallel?

On Linux, Mac OS X, and other POSIX systems, use the versioned Python commands in combination with the -m switch to run the appropriate copy of pip:

python2   -m pip install SomePackage  # default Python 2
python2.7 -m pip install SomePackage  # specifically Python 2.7
python3   -m pip install SomePackage  # default Python 3
python3.4 -m pip install SomePackage  # specifically Python 3.4

Appropriately versioned pip commands may also be available.

On Windows, use the py Python launcher in combination with the -m switch:

py -2   -m pip install SomePackage  # default Python 2
py -2.7 -m pip install SomePackage  # specifically Python 2.7
py -3   -m pip install SomePackage  # default Python 3
py -3.4 -m pip install SomePackage  # specifically Python 3.4

Common installation issues

Installing into the system Python on Linux

On Linux systems, a Python installation will typically be included as part of the distribution. Installing into this Python installation requires root access to the system, and may interfere with the operation of the system package manager and other components of the system if a component is unexpectedly upgraded using pip.

On such systems, it is often better to use a virtual environment or a per-user installation when installing packages with pip.

Pip not installed

It is possible that pip does not get installed by default. One potential fix is:

python -m ensurepip --default-pip

There are also additional resources for installing pip.

Installing binary extensions

Python has typically relied heavily on source based distribution, with end users being expected to compile extension modules from source as part of the installation process.

With the introduction of support for the binary wheel format, and the ability to publish wheels for at least Windows and Mac OS X through the Python Packaging Index, this problem is expected to diminish over time, as users are more regularly able to install pre-built extensions rather than needing to build them themselves.

Some of the solutions for installing scientific software that are not yet available as pre-built wheel files may also help with obtaining other binary extensions without needing to build them locally.

The Python Standard Library¶

While The Python Language Reference describes the exact syntax and semantics of the Python language, this library reference manual describes the standard library that is distributed with Python. It also describes some of the optional components that are commonly included in Python distributions.

Python’s standard library is very extensive, offering a wide range of facilities as indicated by the long table of contents listed below. The library contains built-in modules (written in C) that provide access to system functionality such as file I/O that would otherwise be inaccessible to Python programmers, as well as modules written in Python that provide standardized solutions for many problems that occur in everyday programming. Some of these modules are explicitly designed to encourage and enhance the portability of Python programs by abstracting away platform-specifics into platform-neutral APIs.

The Python installers for the Windows platform usually include the entire standard library and often also include many additional components. For Unix-like operating systems Python is normally provided as a collection of packages, so it may be necessary to use the packaging tools provided with the operating system to obtain some or all of the optional components.

In addition to the standard library, there is a growing collection of several thousand components (from individual programs and modules to packages and entire application development frameworks), available from the Python Package Index.

• Introduction

  • Notes on availability

• Built-in Functions

• Built-in Constants

  • Constants added by the site module

• Built-in Types

  • Truth Value Testing

  • Boolean Operations — and, or, not

  • Comparisons

  • Numeric Types — int, float, complex

  • Iterator Types

  • Sequence Types — list, tuple, range

  • Text Sequence Type — str

  • Binary Sequence Types — bytes, bytearray, memoryview

  • Set Types — set, frozenset

  • Mapping Types — dict

  • Context Manager Types

  • Type Annotation Types — Generic Alias, Union

  • Other Built-in Types

  • Special Attributes

• Built-in Exceptions

  • Base classes

  • Concrete exceptions

  • Warnings

  • Exception hierarchy

• Text Processing Services

  • string — Common string operations

  • re — Regular expression operations

  • difflib — Helpers for computing deltas

  • textwrap — Text wrapping and filling

  • unicodedata — Unicode Database

  • stringprep — Internet String Preparation

  • readline — GNU readline interface

  • rlcompleter — Completion function for GNU readline

• Binary Data Services

  • struct — Interpret bytes as packed binary data

  • codecs — Codec registry and base classes

• Data Types

  • datetime — Basic date and time types

  • zoneinfo — IANA time zone support

  • calendar — General calendar-related functions

  • collections — Container datatypes

  • collections.abc — Abstract Base Classes for Containers

  • heapq — Heap queue algorithm

  • bisect — Array bisection algorithm

  • array — Efficient arrays of numeric values

  • weakref — Weak references

  • types — Dynamic type creation and names for built-in types

  • copy — Shallow and deep copy operations

  • pprint — Data pretty printer

  • reprlib — Alternate repr() implementation

  • enum — Support for enumerations

  • graphlib — Functionality to operate with graph-like structures

• Numeric and Mathematical Modules

  • numbers — Numeric abstract base classes

  • math — Mathematical functions

  • cmath — Mathematical functions for complex numbers

  • decimal — Decimal fixed point and floating point arithmetic

  • fractions — Rational numbers

  • random — Generate pseudo-random numbers

  • statistics — Mathematical statistics functions

• Functional Programming Modules

  • itertools — Functions creating iterators for efficient looping

  • functools — Higher-order functions and operations on callable objects

  • operator — Standard operators as functions

• File and Directory Access

  • pathlib — Object-oriented filesystem paths

  • os.path — Common pathname manipulations

  • fileinput — Iterate over lines from multiple input streams

  • stat — Interpreting stat() results

  • filecmp — File and Directory Comparisons

  • tempfile — Generate temporary files and directories

  • glob — Unix style pathname pattern expansion

  • fnmatch — Unix filename pattern matching

  • linecache — Random access to text lines

  • shutil — High-level file operations

• Data Persistence

  • pickle — Python object serialization

  • copyreg — Register pickle support functions

  • shelve — Python object persistence

  • marshal — Internal Python object serialization

  • dbm — Interfaces to Unix “databases”

  • sqlite3 — DB-API 2.0 interface for SQLite databases

• Data Compression and Archiving

  • zlib — Compression compatible with gzip

  • gzip — Support for gzip files

  • bz2 — Support for bzip2 compression

  • lzma — Compression using the LZMA algorithm

  • zipfile — Work with ZIP archives

  • tarfile — Read and write tar archive files

• File Formats

  • csv — CSV File Reading and Writing

  • configparser — Configuration file parser

  • netrc — netrc file processing

  • xdrlib — Encode and decode XDR data

  • plistlib — Generate and parse Apple .plist files

• Cryptographic Services

  • hashlib — Secure hashes and message digests

  • hmac — Keyed-Hashing for Message Authentication

  • secrets — Generate secure random numbers for managing secrets

• Generic Operating System Services

  • os — Miscellaneous operating system interfaces

  • io — Core tools for working with streams

  • time — Time access and conversions

  • argparse — Parser for command-line options, arguments and sub-commands

  • getopt — C-style parser for command line options

  • logging — Logging facility for Python

  • logging.config — Logging configuration

  • logging.handlers — Logging handlers

  • getpass — Portable password input

  • curses — Terminal handling for character-cell displays

  • curses.textpad — Text input widget for curses programs

  • curses.ascii — Utilities for ASCII characters

  • curses.panel — A panel stack extension for curses

  • platform — Access to underlying platform’s identifying data

  • errno — Standard errno system symbols

  • ctypes — A foreign function library for Python

• Concurrent Execution

  • threading — Thread-based parallelism

  • multiprocessing — Process-based parallelism

  • multiprocessing.shared_memory — Provides shared memory for direct access across processes

  • The concurrent package

  • concurrent.futures — Launching parallel tasks

  • subprocess — Subprocess management

  • sched — Event scheduler

  • queue — A synchronized queue class

  • contextvars — Context Variables

  • _thread — Low-level threading API

• Networking and Interprocess Communication

  • asyncio — Asynchronous I/O

  • socket — Low-level networking interface

  • ssl — TLS/SSL wrapper for socket objects

  • select — Waiting for I/O completion

  • selectors — High-level I/O multiplexing

  • asyncore — Asynchronous socket handler

  • asynchat — Asynchronous socket command/response handler

  • signal — Set handlers for asynchronous events

  • mmap — Memory-mapped file support

• Internet Data Handling

  • email — An email and MIME handling package

  • json — JSON encoder and decoder

  • mailcap — Mailcap file handling

  • mailbox — Manipulate mailboxes in various formats

  • mimetypes — Map filenames to MIME types

  • base64 — Base16, Base32, Base64, Base85 Data Encodings

  • binhex — Encode and decode binhex4 files

  • binascii — Convert between binary and ASCII

  • quopri — Encode and decode MIME quoted-printable data

  • uu — Encode and decode uuencode files

• Structured Markup Processing Tools

  • html — HyperText Markup Language support

  • html.parser — Simple HTML and XHTML parser

  • html.entities — Definitions of HTML general entities

  • XML Processing Modules

  • xml.etree.ElementTree — The ElementTree XML API

  • xml.dom — The Document Object Model API

  • xml.dom.minidom — Minimal DOM implementation

  • xml.dom.pulldom — Support for building partial DOM trees

  • xml.sax — Support for SAX2 parsers

  • xml.sax.handler — Base classes for SAX handlers

  • xml.sax.saxutils — SAX Utilities

  • xml.sax.xmlreader — Interface for XML parsers

  • xml.parsers.expat — Fast XML parsing using Expat

• Internet Protocols and Support

  • webbrowser — Convenient web-browser controller

  • cgi — Common Gateway Interface support

  • cgitb — Traceback manager for CGI scripts

  • wsgiref — WSGI Utilities and Reference Implementation

  • urllib — URL handling modules

  • urllib.request — Extensible library for opening URLs

  • urllib.response — Response classes used by urllib

  • urllib.parse — Parse URLs into components

  • urllib.error — Exception classes raised by urllib.request

  • urllib.robotparser — Parser for robots.txt

  • http — HTTP modules

  • http.client — HTTP protocol client

  • ftplib — FTP protocol client

  • poplib — POP3 protocol client

  • imaplib — IMAP4 protocol client

  • nntplib — NNTP protocol client

  • smtplib — SMTP protocol client

  • smtpd — SMTP Server

  • telnetlib — Telnet client

  • uuid — UUID objects according to RFC 4122

  • socketserver — A framework for network servers

  • http.server — HTTP servers

  • http.cookies — HTTP state management

  • http.cookiejar — Cookie handling for HTTP clients

  • xmlrpc — XMLRPC server and client modules

  • xmlrpc.client — XML-RPC client access

  • xmlrpc.server — Basic XML-RPC servers

  • ipaddress — IPv4/IPv6 manipulation library

• Multimedia Services

  • audioop — Manipulate raw audio data

  • aifc — Read and write AIFF and AIFC files

  • sunau — Read and write Sun AU files

  • wave — Read and write WAV files

  • chunk — Read IFF chunked data

  • colorsys — Conversions between color systems

  • imghdr — Determine the type of an image

  • sndhdr — Determine type of sound file

  • ossaudiodev — Access to OSS-compatible audio devices

• Internationalization

  • gettext — Multilingual internationalization services

  • locale — Internationalization services

• Program Frameworks

  • turtle — Turtle graphics

  • cmd — Support for line-oriented command interpreters

  • shlex — Simple lexical analysis

• Graphical User Interfaces with Tk

  • tkinter — Python interface to Tcl/Tk

  • tkinter.colorchooser — Color choosing dialog

  • tkinter.font — Tkinter font wrapper

  • Tkinter Dialogs

  • tkinter.messagebox — Tkinter message prompts

  • tkinter.scrolledtext — Scrolled Text Widget

  • tkinter.dnd — Drag and drop support

  • tkinter.ttk — Tk themed widgets

  • tkinter.tix — Extension widgets for Tk

  • IDLE

• Development Tools

  • typing — Support for type hints

  • pydoc — Documentation generator and online help system

  • Python Development Mode

  • Effects of the Python Development Mode

  • ResourceWarning Example

  • Bad file descriptor error example

  • doctest — Test interactive Python examples

  • unittest — Unit testing framework

  • unittest.mock — mock object library

  • unittest.mock — getting started

  • 2to3 - Automated Python 2 to 3 code translation

  • test — Regression tests package for Python

  • test.support — Utilities for the Python test suite

  • test.support.socket_helper — Utilities for socket tests

  • test.support.script_helper — Utilities for the Python execution tests

  • test.support.bytecode_helper — Support tools for testing correct bytecode generation

  • test.support.threading_helper — Utilities for threading tests

  • test.support.os_helper — Utilities for os tests

  • test.support.import_helper — Utilities for import tests

  • test.support.warnings_helper — Utilities for warnings tests

• Debugging and Profiling

  • Audit events table

  • bdb — Debugger framework

  • faulthandler — Dump the Python traceback

  • pdb — The Python Debugger

  • The Python Profilers

  • timeit — Measure execution time of small code snippets

  • trace — Trace or track Python statement execution

  • tracemalloc — Trace memory allocations

• Software Packaging and Distribution

  • distutils — Building and installing Python modules

  • ensurepip — Bootstrapping the pip installer

  • venv — Creation of virtual environments

  • zipapp — Manage executable Python zip archives

• Python Runtime Services

  • sys — System-specific parameters and functions

  • sysconfig — Provide access to Python’s configuration information

  • builtins — Built-in objects

  • __main__ — Top-level code environment

  • warnings — Warning control

  • dataclasses — Data Classes

  • contextlib — Utilities for with-statement contexts

  • abc — Abstract Base Classes

  • atexit — Exit handlers

  • traceback — Print or retrieve a stack traceback

  • __future__ — Future statement definitions

  • gc — Garbage Collector interface

  • inspect — Inspect live objects

  • site — Site-specific configuration hook

• Custom Python Interpreters

  • code — Interpreter base classes

  • codeop — Compile Python code

• Importing Modules

  • zipimport — Import modules from Zip archives

  • pkgutil — Package extension utility

  • modulefinder — Find modules used by a script

  • runpy — Locating and executing Python modules

  • importlib — The implementation of import

  • Using importlib.metadata

• Python Language Services

  • ast — Abstract Syntax Trees

  • symtable — Access to the compiler’s symbol tables

  • token — Constants used with Python parse trees

  • keyword — Testing for Python keywords

  • tokenize — Tokenizer for Python source

  • tabnanny — Detection of ambiguous indentation

  • pyclbr — Python module browser support

  • py_compile — Compile Python source files

  • compileall — Byte-compile Python libraries

  • dis — Disassembler for Python bytecode

  • pickletools — Tools for pickle developers

• MS Windows Specific Services

  • msilib — Read and write Microsoft Installer files

  • msvcrt — Useful routines from the MS VC++ runtime

  • winreg — Windows registry access

  • winsound — Sound-playing interface for Windows

• Unix Specific Services

  • posix — The most common POSIX system calls

  • pwd — The password database

  • spwd — The shadow password database

  • grp — The group database

  • crypt — Function to check Unix passwords

  • termios — POSIX style tty control

  • tty — Terminal control functions

  • pty — Pseudo-terminal utilities

  • fcntl — The fcntl and ioctl system calls

  • pipes — Interface to shell pipelines

  • resource — Resource usage information

  • nis — Interface to Sun’s NIS (Yellow Pages)

  • syslog — Unix syslog library routines

• Superseded Modules

  • optparse — Parser for command line options

  • imp — Access the import internals

• Undocumented Modules

  • Platform specific modules

• Security Considerations