The Python 2 Standard Library by Example
Distributed with every copy of Python, the Standard Library contains hundreds of modules that provide tools for interacting with the operating system, interpreter, and Internet—all of them tested and ready to be used to jump-start the development of your applications. This book presents selected examples demonstrating how to use the most commonly used features of the modules that give Python its “batteries included” slogan, taken from the popular Python Module of the Week (PyMOTW) blog series.
Preview Online
Sample Chapters
A free PDF containing the table of contents, foreword, introduction, index, and chapter 2 (“Data Structures”) from the final version of the manuscript is available on publisher’s web site.
A free e-book containing the table of contents, foreword, introduction, index, and chapter 1 (“Text”) is available through the iTunes book store.
Safari Books Online
The full text of the book is available through Safari Books Online. This digital edition matches the print and other digital versions completely.
Read What Reviewers Have To Say
Slashdot
…I like this class of documentation that is between a cookbook and a reference manual. I find it useful that the examples are not so terse nor overly verbose. I also appreciate the quality of the code and the references for further readings. I think that this book fills a void that will make many Python programmers more proficient.
—Ahmed Al-Saadi
Python Zone
This book is and will be a key resource for many serious Python programmers and provides a resource for understanding and using many of the Standard Library capabilities. It will be in a handy space on my bookshelf and is highly recommended.
—Tom Brander
Musings of an Anonymous Geek
… the book is written by an authoritative, knowledgeable, experienced, and prolific Python developer. While he’s a creative thinker, his work is balanced by a healthy dose of pragmatism and grounded in best practices.
—Brian Jones
Triumph Over Madness
My copy of The Python Standard Library by Example arrived this morning and it’s totally kick-ass.
—Reed Wade
Order Now!
Order your copy today by going directly from the publisher’s web site or from one of these online booksellers:
This Book’s Target Audience
The audience for this book is an intermediate Python 2 programmer, so although all of the source code is presented with discussion, only a few cases include line-by-line explanations. Every section focuses on the features of the modules, illustrated by the source code and output from fully independent example programs. Each feature is presented as concisely as possible, so the reader can focus on the module or function being demonstrated without being distracted by the supporting code.
An experienced programmer familiar with other languages may be able to learn Python from this book, but it is not intended to be an introduction to the language. Some prior experience writing Python programs will be useful when studying the examples.
Several sections, such as the description of network programming with sockets or hmac encryption, require domain-specific knowledge. The basic information needed to explain the examples is included here, but the range of topics covered by the modules in the standard library makes it impossible to cover every topic comprehensively in a single volume. The discussion of each module is followed by a list of suggested sources for more information and further reading, including online resources, RFC standards documents, and related books.
Although the current transition to Python 3 is well under way, Python 2 is still likely to be the primary version of Python used in production environments for years to come because of the large amount of legacy Python 2 source code available and the slow transition rate to Python 3. All of the source code for the examples has been updated from the original online versions and tested with Python 2.7, the final release of the 2.x series.
Python 3 versus 2
The Python community has largely completed the transition from Python version 2 to Python version 3. As the major version number change implies, there are many incompatibilities between Python 2 and 3, and not just in the language. Quite a few of the standard library modules have been renamed or otherwise reorganized in the new version.
The Python development community recognized that those incompatibilities would require an extended transition period, while the ecosystem of Python libraries and tools was updated to work with Python 3. Although many projects still rely on Python 2, it is only receiving security updates and was completely deprecated by 2020. All new feature work is happening in the Python 3 releases.
A new version of this book, The Python 3 Standard Library By Example, is available for developers transitioning to or already working in Python 3.
Downloading the Example Code
All of the source code for the examples in the book can be downloaded and used under the BSD license.
The original versions of the articles are available on the PyMOTW site.
Errata
Be the first to report an issue via the bug tracker on the issue tracker. As updates are made based on errata, they will be posted to pymotw.com.
Table of Contents
Tables xxxi
Foreword xxxiii
Acknowledgments xxxvii
About the Author xxxix
Introduction 1
Chapter 1: Text 3
1.1 string—Text Constants and Templates 4
1.2 textwrap—Formatting Text Paragraphs 9
1.3 re—Regular Expressions 13
1.4 difflib—Compare Sequences 61
Chapter 2: Data Structures 69
2.1 collections—Container Data Types 70
2.2 array—Sequence of Fixed-Type Data 84
2.3 heapq—Heap Sort Algorithm 87
2.4 bisect—Maintain Lists in Sorted Order 93
2.5 Queue—Thread-Safe FIFO Implementation 96
2.6 struct—Binary Data Structures 102
2.7 weakref—Impermanent References to Objects 106
2.8 copy—Duplicate Objects 117
2.9 pprint—Pretty-Print Data Structures 123
Chapter 3: Algorithms 129
3.1 functools—Tools for Manipulating Functions 129
3.2 itertools—Iterator Functions 141
3.3 operator—Functional Interface to Built-in Operators 153
3.4 contextlib—Context Manager Utilities 163
Chapter 4: Dates and Times 173
4.1 time—Clock Time 173
4.2 datetime—Date and Time Value Manipulation 180
4.3 calendar—Work with Dates 191
Chapter 5: Mathematics 197
5.1 decimal—Fixed and Floating-Point Math 197
5.2 fractions—Rational Numbers 207
5.3 random—Pseudorandom Number Generators 211
5.4 math—Mathematical Functions 223
Chapter 6: The File System 247
6.1 os.path—Platform-Independent Manipulation of Filenames 248
6.2 glob—Filename Pattern Matching 257
6.3 linecache—Read Text Files Efficiently 261
6.4 tempfile—Temporary File System Objects 265
6.5 shutil—High-Level File Operations 271
6.6 mmap—Memory-Map Files 279
6.7 codecs—String Encoding and Decoding 284
6.8 StringIO—Text Buffers with a File-like API 314
6.9 fnmatch—UNIX-Style Glob Pattern Matching 315
6.10 dircache—Cache Directory Listings 319
6.11 filecmp—Compare Files 322
Chapter 7: Data Persistence and Exchange 333
7.1 pickle—Object Serialization 334
7.2 shelve—Persistent Storage of Objects 343
7.3 anydbm—DBM-Style Databases 347
7.4 whichdb—Identify DBM-Style Database Formats 350
7.5 sqlite3—Embedded Relational Database 351
7.6 xml.etree.ElementTree—XML Manipulation API 387
7.7 csv—Comma-Separated Value Files 411
Chapter 8: Data Compression and Archiving 421
8.1 zlib—GNU zlib Compression 421
8.2 gzip—Read and Write GNU Zip Files 430
8.3 bz2—bzip2 Compression 436
8.4 tarfile—Tar Archive Access 448
8.5 zipfile—ZIP Archive Access 457
Chapter 9: Cryptography 469
9.1 hashlib—Cryptographic Hashing 469
9.2 hmac—Cryptographic Message Signing and Verification 473
Chapter 10: Processes and Threads 481
10.1 subprocess—Spawning Additional Processes 481
10.2 signal—Asynchronous System Events 497
10.3 threading—Manage Concurrent Operations 505
10.4 multiprocessing—Manage Processes like Threads 529
Chapter 11: Networking 561
11.1 socket—Network Communication 561
11.2 select—Wait for I/O Efficiently 594
11.3 SocketServer—Creating Network Servers 609
11.4 asyncore—Asynchronous I/O 619
11.5 asynchat—Asynchronous Protocol Handler 629
Chapter 12: The Internet 637
12.1 urlparse—Split URLs into Components 638
12.2 BaseHTTPServer—Base Classes for Implementing Web Servers 644
12.3 urllib—Network Resource Access 651
12.4 urllib2—Network Resource Access 657
12.5 base64—Encode Binary Data with ASCII 670
12.6 robotparser—Internet Spider Access Control 674
12.7 Cookie—HTTP Cookies 677
12.8 uuid—Universally Unique Identifiers 684
12.9 json—JavaScript Object Notation 690
12.10 xmlrpclib—Client Library for XML-RPC 702
12.11 SimpleXMLRPCServer—An XML-RPC Server 714
Chapter 13: Email 727
13.1 smtplib—Simple Mail Transfer Protocol Client 727
13.2 smtpd—Sample Mail Servers 734
13.3 imaplib—IMAP4 Client Library 738
13.4 mailbox—Manipulate Email Archives 758
Chapter 14: Application Building Blocks 769
14.1 getopt—Command-Line Option Parsing 770
14.2 optparse—Command-Line Option Parser 777
14.3 argparse—Command-Line Option and Argument Parsing 795
14.4 readline—The GNU Readline Library 823
14.5 getpass—Secure Password Prompt 836
14.6 cmd—Line-Oriented Command Processors 839
14.7 shlex—Parse Shell-Style Syntaxes 852
14.8 ConfigParser—Work with Configuration Files 861
14.9 logging—Report Status, Error, and Informational Messages 878
14.10 fileinput—Command-Line Filter Framework 883
14.11 atexit—Program Shutdown Callbacks 890
14.12 sched—Timed Event Scheduler 894
Chapter 15: Internationalization and Localization 899
15.1 gettext—Message Catalogs 899
15.2 locale—Cultural Localization API 909
Chapter 16: Developer Tools 919
16.1 pydoc—Online Help for Modules 920
16.2 doctest—Testing through Documentation 921
16.3 unittest—Automated Testing Framework 949
16.4 traceback—Exceptions and Stack Traces 958
16.5 cgitb—Detailed Traceback Reports 965
16.6 pdb—Interactive Debugger 975
16.7 trace—Follow Program Flow 1012
16.8 profile and pstats—Performance Analysis 1022
16.9 timeit—Time the Execution of Small Bits of Python Code 1031
16.10 compileall—Byte-Compile Source Files 1037
16.11 pyclbr—Class Browser 1039
Chapter 17: Runtime Features 1045
17.1 site—Site-Wide Configuration 1046
17.2 sys—System-Specific Configuration 1055
17.3 os—Portable Access to Operating System Specific Features 1108
17.4 platform—System Version Information 1129
17.5 resource—System Resource Management 1134
17.6 gc—Garbage Collector 1138
17.7 sysconfig—Interpreter Compile-Time Configuration 1160
Chapter 18: Language Tools 1169
18.1 warnings—Nonfatal Alerts 1170
18.2 abc—Abstract Base Classes 1178
18.3 dis—Python Bytecode Disassembler 1186
18.4 inspect—Inspect Live Objects 1200
18.5 exceptions—Built-in Exception Classes 1216
Chapter 19: Modules and Packages 1235
19.1 imp—Python’s Import Mechanism 1235
19.2 zipimport—Load Python Code from ZIP Archives 1240
19.3 pkgutil—Package Utilities 1247
Index of Python Modules 1259
Index 1261