The pyproject.toml
file allows package creators to define the build system as
a dependency as well as a projects metadata.
Also, other kinds of meta-data and the install requirements can be
defined in it.
If you are interested in a sample project, try pypa/sampleproject or try the packaging tutorial.
Example
[project]
name = "infer_pyproject"
version = "0.1.0"
description = "Create a pyproject.toml file for an existing project."
authors = [
{name = "Martin Thoma", email="[email protected]"},
{email = "[email protected]"}
]
license = {file = "LICENSE.txt"}
readme = "README.md"
requires-python = ">=3.6"
keywords = ["packaging", "dependency", "infer", "pyproject.toml"]
classifiers = [
"Topic :: Software Development"
]
# Requirements: This is done differently by poetry!
dependencies = [
"Click>=7.0"
]
[project.optional-dependencies]
dev = [
"black>=18.3-alpha.0",
]
[project.urls]
homepage = "https://github.com/MartinThoma/infer_pyproject"
documentation = "https://github.com/MartinThoma/infer_pyproject"
repository = "https://github.com/MartinThoma/infer_pyproject"
[project.scripts]
poetry = "infer_pyproject.cli:main"
[build-system]
requires = [
"setuptools >= 35.0.2",
"setuptools_scm >= 2.0.0, <3"
]
build-backend = "setuptools.build_meta"
[tool.black]
line-length = 88
target_version = ['py36']
include = '\.pyi?$'
exclude = '''
(
/(
\.eggs # exclude a few common directories in the
| \.git # root of the project
| \.hg
| \.mypy_cache
| \.tox
| \.venv
| _build
| buck-out
| build
| dist
)/
| foo.py # also separately exclude a file named foo.py in
# the root of the project
)
'''
History
- 1990-08-09: First commit of cpython
- 1998: distutils was drafted in IPC7 (source)
- 1999-03-22: First distutil commit to cpython
- 2000-11-16: PEP 229 -- Using Distutils to Build Python
- 2003-07-06: First capture of pypi.org found on waybackmachine
- 2004-03-19: First commit of setuptools
- 2006-05-13: First version of setuptools was released
- 2007-09-14: First commit of virtualenv
- 2008-10-16: First commit of pip
- 2013-08-13: First commit of twine
- 2015-01-24: First commit of PyPI warehouse
- 2015-03-16: First version of flit was released
- 2015-09-30: PEP 517 -- A build-system independent format for source trees (provisional)
- 2015-10-26: PEP 516 -- Build system abstraction for pip/conda etc (rejected)
- 2015-11-11: PEP 508 -- Dependency specification for Python Software Packages (active)
- 2016-05-10: PEP 518 -- Specifying Minimum Build System Requirements for Python Projects (provisional)
- 2016-11-18: First commit of pipfile
- 2017-01-20: First commit of pipenv
- 2018-02-28: First version of Poetry was released
- 2020-06-22: PEP 621 - still not supported by poetry in August 2022 (#3332)
Tools
- Python Package
- A bundle of software. This includes code and meta-data, such as requirements, a short and a long description, the license. It also contains instructions how to build the software. Formerly this was done with distutils.
- distutils
- Lets you create source distributions (
python setup.py sdist
). Sometime building takes a long time, so you might want to share already built distributions. You can do that withpython setup.py bdist
. - setuptools
- Like distutils, but a 3rd party library. It is de-facto standard, but does not come with Python.
setup.py
dependency declaration- Python file which specifies a package. As it can be arbitrary code, there is no way to know the dependencies of a package for sure without executing
setup.py
. - PyPI (Python Packaging Index) software repository
- PyPI is the official third-party software repository for Python. Here people can share their code in form of Python packages.
easy_install
- Easy_install is a package manager which is replaced by pip, because it could not uninstall and did not know what was installed. Other reasons as well.
- egg distribution distribution format
- "Egg" is a single-file importable distribution format for Python-related projects. Eggs are to Pythons as Jars are to Java, but eggs are richer than jars; they hold interesting metadata such as licensing details, release dependencies, ... (source). It is a zip file.
pip
- pip is a de facto standard package manager for Python. It allows to install packages and installs required packages. pip introduced
requirements.txt
requirements.txt
dependency declaration- It allows pinning versions of a dependency. The
setup.py
includes abstract requirements, therequirements.txt
includes concrete ones. Abstract requirements are more flexible, concrete ones are stable. - wheel distribution distribution format
- The wheel binary package format is specified in PEP 427. It is similar to egg distributions. It is a zip file.
- twine
- Allowed to securely upload a package to PyPI.
- conda
- conda is a Python-agnostic packaging tool and installer. If you need more than Python / if you don't have Python installed. It supports C, Fortran, R, Perl, Java, ...
- pipfile and pipfile.lock dependency declaration
- A pipfile should be easier to maintain than multiple
requirements.txt
files, e.g. for testing / environments (dev, stage, prod). The pipfile.lock is automatically generated and tracks interdependencies of the required packages; it is similar togemfile.lock
in Ruby andyarn.lock
in JavaScript. pipenv is the tool to use with those files. SO-Questions: 28 for pipfile, 485 for pipenv pyproject.toml
dependency declaration- A build-system independent way to specify project dependencies.
setup.py
is bad as it is 3rd party and requires code execution. It is a way to step away from distutils / setuptools. The specified build system can be setuptools as well. The pyproject.toml is similar to the Cargo.toml of Rust - pyenv
- Install different Python versions. It is similar to rbenv from Ruby.
- pipenv
- Wrapper for virtualenv. Has 485 questions on SO. See pipfile for more info.
- virtualenvwrapper
- Another virtuelenv wrapper. Has 570 questions on SO.
- Poetry
- Meant to be a successor of pipenv, but seems not production-ready yet (source, 13 SO questions).
- DepHell
- A Python project management tool. Can convert between setup.py <-> pyproject.toml.
- flit
- Flit is a way to put Python packages and modules on PyPI. It is a 3rd party replacement for setuptools, but has no SO tag.
See also
- Jeff Triplett: Why Python devs should use Pipenv, 2018-02-28.
- Sarahan, McCormick, Fillion-Robin: The Sheer Joy of Packaging, 2018.
- Poetry: The pyproject.toml file
- Frost Ming: A deeper look into Pipenv and Poetry, 2019-01-04.
- Chad Smith: Five Myths About Pipenv, 2018-11-30.