The Python 3 Standard Library by Example

· 8 min read
book cover

Distributed with every copy of Python, the Standard Library contains hundreds of modules that provide tools for interacting with the operating system, interpreter, and Internet—all of them tested and ready to be used to jump-start the development of your applications. This book presents selected examples demonstrating how to use the most commonly used features of the modules that give Python its “batteries included” slogan, taken from the popular Python Module of the Week (PyMOTW) blog series.

Preview Online

Safari Books Online

The full text of the book is available through Safari Books Online. This digital edition matches the print and other digital versions completely.

Order Now!

Order your copy today directly from the publisher’s web site or from one of these online booksellers:

This Book’s Target Audience

The audience for this book is an intermediate Python 3 programmer, so although all of the source code is presented with discussion, only a few cases include line-by-line explanations. Every section focuses on the features of the modules, illustrated by the source code and output from fully independent example programs. Each feature is presented as concisely as possible, so the reader can focus on the module or function being demonstrated without being distracted by the supporting code.

An experienced programmer familiar with other languages may be able to learn Python from this book, but it is not intended to be an introduction to the language. Some prior experience writing Python programs will be useful when studying the examples.

Several sections, such as the description of network programming with sockets or hmac encryption, require domain-specific knowledge. The basic information needed to explain the examples is included here, but the range of topics covered by the modules in the standard library makes it impossible to cover every topic comprehensively in a single volume. The discussion of each module is followed by a list of suggested sources for more information and further reading, including online resources, RFC standards documents, and related books.

Python 3 versus 2

The Python community has largely completed the transition from Python version 2 to Python version 3. As the major version number change implies, there are many incompatibilities between Python 2 and 3, and not just in the language. Quite a few of the standard library modules have been renamed or otherwise reorganized in the new version.

The Python development community recognized that those incompatibilities would require an extended transition period, while the ecosystem of Python libraries and tools was updated to work with Python 3. Although many projects still rely on Python 2, it is only receiving security updates and was completely deprecated by 2020. All new feature work is happening in the Python 3 releases.

It can be challenging, though not impossible, to write programs that work with both versions. Doing so frequently requires examining the version of Python under which a program is running and using different module names for imports or different arguments to classes or functions. There are tools, outside of the standard library, to make the process simpler. To keep the examples in this book as concise as possible, while still relying only on the standard library, they are focused on Python 3. All of the examples have been tested under Python 3.5, the current release of the 3.x series at the time they were written, and may not work with Python 2 without modification. For examples designed to work with Python 2, refer to the Python 2 edition of the book, called The Python Standard Library By Example.

Again, in an effort to maintain clear and concise descriptions for each example, the differences between Python 2 and 3 are not highlighted in each chapter. The Porting Notes appendix summarizes some of the biggest differences, and is organized to be useful as an aid when porting from Python 2 to 3.

Downloading the Example Code

All of the source code for the examples in the book can be downloaded and used under the BSD license.

The original versions of the articles are available on the PyMOTW site.

Errata

Be the first to report an issue via the bug tracker on the issue tracker. As updates are made based on errata, they will be posted to pymotw.com.

Table of Contents

Introduction
Chapter 1. Text
1.1. string — Text Constants and Templates
1.2. textwrap — Formatting Text Paragraphs
1.3. re — Regular Expressions
1.4. difflib — Compare Sequences

Chapter 2. Data Structures
2.1. enum – Enumeration Type
2.2. collections — Container Data Types
2.3. array — Sequence of Fixed-type Data
2.4. heapq – Heap Sort Algorithm
2.5. bisect — Maintain Lists in Sorted Order
2.6. queue — Thread-safe FIFO Implementation
2.7. struct — Binary Data Structures
2.8. weakref — Impermanent References to Objects
2.9. copy — Duplicate Objects
2.10. pprint — Pretty-print Data Structures

Chapter 3. Algorithms
3.1. functools — Tools for Manipulating Functions
3.2. itertools — Iterator Functions
3.3. operator — Functional Interface to Built-in Operators
3.4. contextlib — Context Manager Utilities

Chapter 4. Dates and Times
4.1. time — Clock Time
4.2. datetime — Date and Time Value Manipulation
4.3. calendar — Work with Dates

Chapter 5. Mathematics
5.1. decimal — Fixed and Floating Point Math
5.2. fractions — Rational Numbers
5.3. random — Pseudorandom Number Generators
5.4. math — Mathematical Functions
5.5. statistics — Statistical Calculations

Chapter 6. The File System
6.1. os.path — Platform-independent Manipulation of Filenames
6.2. pathlib — Filesystem Paths as Objects
6.3. glob — Filename Pattern Matching
6.4. fnmatch — Unix-style Glob Pattern Matching
6.5. linecache — Read Text Files Efficiently
6.6. tempfile — Temporary File System Objects
6.7. shutil — High-level File Operations
6.8. filecmp — Compare Files
6.9. mmap — Memory-map Files
6.10. codecs — String Encoding and Decoding
6.11. io — Text, Binary, and Raw Stream I/O Tools

Chapter 7. Data Persistence and Exchange
7.1. pickle — Object Serialization
7.2. shelve — Persistent Storage of Objects
7.3. dbm — Unix Key-Value Databases
7.4. sqlite3 — Embedded Relational Database
7.5. xml.etree.ElementTree — XML Manipulation API
7.6. csv — Comma-separated Value Files

Chapter 8. Data Compression and Archiving
8.1. zlib — GNU zlib Compression
8.2. gzip — Read and Write GNU zip Files
8.3. bz2 — bzip2 Compression
8.4. tarfile — Tar Archive Access
8.4. tarfile — Tar Archive Access 489
8.5. zipfile — ZIP Archive Access

Chapter 9. Cryptography
9.1. hashlib — Cryptographic Hashing
9.2. hmac — Cryptographic Message Signing and Verification

Chapter 10. Concurrency with Processes, Threads, and Coroutines
10.1. subprocess — Spawning Additional Processes
10.2. signal — Asynchronous System Events
10.3. threading — Manage Concurrent Operations Within a Process
10.4. multiprocessing — Manage Processes Like Threads
10.5. asyncio — Asynchronous I/O, event loop, and concurrency tools
10.6. concurrent.futures — Manage Pools of Concurrent Tasks

Chapter 11. Networking
11.1. ipaddress — Internet Addresses
11.2. socket — Network Communication
11.3. selectors — I/O Multiplexing Abstractions
11.4. select — Wait for I/O Efficiently
11.5. socketserver — Creating Network Servers

Chapter 12. The Internet
12.1. urllib.parse — Split URLs into Components
12.2. urllib.request — Network Resource Access
12.3. urllib.robotparser — Internet Spider Access Control
12.4. base64 — Encode Binary Data with ASCII
12.5. http.server — Base Classes for Implementing Web Servers
12.6. http.cookies — HTTP Cookies
12.7. webbrowser — Displays web pages
12.8. uuid — Universally Unique Identifiers
12.9. json — JavaScript Object Notation
12.10. xmlrpc.client — Client Library for XML-RPC
12.11. xmlrpc.server — An XML-RPC server

Chapter 13. Email
13.1. smtplib — Simple Mail Transfer Protocol Client
13.2. smtpd — Sample Mail Servers
13.3. mailbox — Manipulate Email Archives
13.4. imaplib — IMAP4 Client Library

Chapter 14. Application Building Blocks
14.1. argparse — Command-Line Option and Argument Parsing
14.2. getopt — Command Line Option Parsing
14.3. readline — The GNU readline Library
14.4. getpass — Secure Password Prompt
14.5. cmd — Line-oriented Command Processors
14.6. shlex — Parse Shell-style Syntaxes
14.7. configparser — Work with Configuration Files
14.8. logging — Report Status, Error, and Informational Messages
14.9. fileinput — Command-Line Filter Framework
14.10. atexit — Program Shutdown Callbacks
14.11. sched — Timed Event Scheduler

Chapter 15. Internationalization and Localization
15.1. gettext — Message Catalogs
15.2. locale — Cultural Localization API

Chapter 16. Developer Tools
16.1. pydoc — Online Help for Modules
16.2. doctest — Testing Through Documentation
16.3. unittest — Automated Testing Framework
16.4. trace — Follow Program Flow
16.5. traceback — Exceptions and Stack Traces
16.6. cgitb — Detailed Traceback Reports
16.7. pdb — Interactive Debugger
16.8. profile and pstats — Performance Analysis
16.9. timeit — Time the execution of small bits of Python code.
16.10. tabnanny — Indentation validator
16.11. compileall — Byte-compile Source Files
16.12. pyclbr — Class Browser
16.13. venv — Create Virtual Environments
16.14. ensurepip — Install the Python Package Installer

Chapter 17. Runtime Features
17.1. site — Site-wide Configuration
17.2. sys — System-specific Configuration
17.3. os — Portable access to operating system specific features
17.4. platform — System Version Information
17.5. resource — System Resource Management
17.6. gc — Garbage Collector
17.7. sysconfig — Interpreter Compile-time Configuration

Chapter 18. Language Tools
18.1. warnings — Non-fatal Alerts
18.2. abc — Abstract Base Classes
18.3. dis — Python Bytecode Disassembler
18.4. inspect — Inspect Live Objects

Chapter 19. Modules and Packages
19.1. importlib — Python’s Import Mechanism
19.2. pkgutil — Package Utilities
19.3. zipimport — Load Python Code from ZIP Archives

Appendix A Porting Notes
A.1. References
A.2. New Modules
A.3. Renamed Modules
A.4. Removed Modules
A.5. Deprecated Modules
A.6. Summary of Changes to Modules

Appendix B Outside of the Standard Library
B.1. Text
B.2. Algorithms
B.3. Dates and Times
B.4. Mathematics
B.5. Data Persistence and Exchange
B.6. Cryptography
B.7. Concurrency with Processes, Threads, and Coroutines
B.8. The Internet
B.9. Email
B.10. Application Building Blocks
B.11. Developer Tools