In this guide, I'm going to show you how can easily publish a package to the Python Package Index so that others can install and use your work. Once the package is available on PyPI, anyone will be able to run pip install to install the package on their machine. This allows people to import your code into their own modules as a library, but you can also release command-line tools via PyPI; I will show you how to do both.

note: these instructions are Mac and Linux centric. I don't think they would be much different, but I don't do much development work on Windows so I don't know the nuances of that platform.

The Code

This first thing you need is some code to publish! I have created a minimal project that you can use as a starting point. You can find the code here. Let's take a look.

First let's walk through the folder structure:

├── README.md
├── package_boilerplate
│   ├── __init__.py
│   └── importable.py
├── requirements.txt
├── scripts
│   └── boilerplate-cli
├── setup.py
└── version.py

These are all of the files inside of the python package. First, we have a directory named package_boilerplate. This directory contains all of the code for our module. All of the files outside of this directory are metadata for the package itself and are used in the package publishing process, for installing dependencies, and stuff like that. We'll go through the other files shortly.

README.md is markdown formatted text containing details about the package. This content is usually rendered as HTML and made visible in places like GitHub and PyPI.

If we look inside of __init__.py, we see the following:

from colored import fg, bg, attr

def main():
    print(f"{fg('dark_orange')}{attr('bold')}You called me!{attr('reset')}")

We're doing a couple of things here:  first, we import a library called colored. Colored is a tool to colourize your program's terminal output. It's fun!

So basically, this file will print some coloured text to the terminal if main is called. If you continue reading, you will see how we can publish this as a command-line tool through pip!

Next, let's look inside of importable.py:

from colored import fg, bg, attr

print(f"{fg('orchid')}{attr('bold')}You imported me!{attr('reset')}")

Again, we're just print some coloured text to the terminal but this time we can import this code in an external module. The text will be printed as soon as the file is imported since the code is not contained in a function.

Ok, so that is the logic of our program. Let's continue down the folder structure to see what else is needed to make this publishable.

The next file is requirements.txt. If you've ever written a python module that depends on other external python packages, then you may be familiar with this file. The file is used to tell pip which versions of which modules it should install all at once if you run pip install -r requirements.txt. Python packages are no different; if they rely on dependencies, pip needs to know what to install. We'll reference this file in setup.py.

Next, we have the scripts directory. The folder name doesn't matter and isn't required, but I prefer to keep things separate and organized. Inside of this folder we have a file called boilerplate-cli. Notice how we don't give the file an extension like .py and we used the naming convention boilerplate-cli instead of boilerplate_cli. This is because this is the file that will be installed as a command-line program in the users PATH. We want the the user to be able to invoke the program by running boilerplate-cli and not boilerplate_cli.py because that is the expected convention for command-line programs.

Inside of this file we have this:

#!/usr/bin/env python

from package_boilerplate import main

main()

The first line is important because we have to tell the user's operating system what kind of code this it because it doesn't have a file extension. This is called a shebang. Next we import the entrypoint into the program from __init__.py that we named main and we run the function.

Calling this function kicks off our program! Right now it only prints text to the terminal, but it could be the entrypoint into a more complicated program. We will reference this file in the setup.py section to make it available as a command-line program for the user.

The most important part of this process is the setup.py file. This tells pip how to bundle the package and what all the settings and dependencies are.

"""Module setup."""

import runpy
from setuptools import setup, find_packages

PACKAGE_NAME = "package-boilerplate"
version_meta = runpy.run_path("./version.py")
VERSION = version_meta["__version__"]


with open("README.md", "r") as fh:
    long_description = fh.read()


def parse_requirements(filename):
    """Load requirements from a pip requirements file."""
    lineiter = (line.strip() for line in open(filename))
    return [line for line in lineiter if line and not line.startswith("#")]


if __name__ == "__main__":
    setup(
        name=PACKAGE_NAME,
        version=VERSION,
        packages=find_packages(),
        install_requires=parse_requirements("requirements.txt"),
        python_requires=">=3.6.3",
        scripts=["scripts/boilerplate-cli"],
        description="This is a description.",
        long_description=long_description,
        long_description_content_type="text/markdown",
    )

First we import a few utilities called runpy and setuptools. runpy is built into python, but setuptools needs to be installed which we will go over in the next section.

Next, we get the python metadata out of the file version.py and obtain the current version number VERSION. This file just contains the current version number of the package you are publishing. We'll go over why it's split into its own file in the next section.

Next, long_description is pulled out of the content from README.md. Using the content from the readme means that this description only needs to live in one place.

Then, I've added a helper function called parse_requirements. The python setup.py format expects that install_requires property contains a list of dependencies that this package relies on. Since it's burdensome to maintain a requirements.txt file with your dependencies as well as the dependencies in a list here, this parse_requirements function simple imports the dependencies from requirements.txt so that you only need to maintain them in one place.

Finally, we call setup with some values to define the package. The section that specifies packages=find_packages() is pointing to the directory named package_boilerplate and is discovered automatically by this function. Name and version are used in the PyPI directory listing and also are what people reference when they want to install the package. install_requires pulls in the list of requirements from the requirement.txt file. python_requires=">=3.6.3" is the version of python that the user must have installed. And finally, scripts is where you specify the command line program that should be installed for the user.

There are other values that you should set as well such as description and author. You can read more about the setup values available here.

The last file, version.py is quite small. This file is used for two things: first, it is imported in the setup.py module above to use as the package version; and second, the version value is stored in the __version__ attribute. This is recommend so that the package version can be discovered programatically.

A tip: as you are working, you can install your package locally before you even publish it. This way, you will know that it works before submitting it to PyPI. To install the package locally, you can run pip install -e . in the package directory.

Publishing Your Package

So you've written your package and you want to publish it! We're almost there. There are just a few more steps to publish your package.

The first thing will want to do is install a tool called twine by running pip install twine. Twine is a tool for publishing Python packages on PyPI. Then, we also need to install setuptools since we are using it in our setup script: pip install setuptools.

Now, we need to create a distribution package that we will publish. To do this, run:  python setup.py sdist bdist_wheel which will create build and dist directories.

Once the build has completed, we can check for any errors or warnings using twine. Run twine check dist/*  to check the distribution package for errors. If all goes well, you should see this:

Checking distribution dist/package_boilerplate-1.0.0-py3-none-any.whl: Passed
Checking distribution dist/package-boilerplate-1.0.0.tar.gz: Passed

Once this is done, we want to publish our package to the PyPI test site. We can publish to test.pypi.org before we publish to the real index to make sure everything looks good! To do this, we can instruct twine to use a different repository like so: twine upload --repository-url https://test.pypi.org/legacy/ dist/*

If we run this now, we will see the following:

Enter your username:

Looks like we need to create an account! So head over to the registration page for the PyPI test site and create an account. If you run the command again and enter your username and password hopefully you should see:

Uploading distributions to https://test.pypi.org/legacy/
Uploading package_boilerplate-1.0.0-py3-none-any.whl
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4.95k/4.95k [00:00<00:00, 37.2kB/s]
Uploading package-boilerplate-1.0.0.tar.gz
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4.11k/4.11k [00:01<00:00, 3.76kB/s]

If you do, congratulations! You have successfully published your package!

Note: make sure your package name is unique!

All that remains is to publish your package to the official pypi.org. To do this, you just need to create another account there and publish your package without specifying a different repository like so: twine upload dist/*.

Installing Your Package

Now that your package is installed, I bet you want to install it to see how it works! Let's install our package from the PyPI test index: pip install --index-url https://test.pypi.org/simple/ package-boilerplate. In the future, if we don't provide the --index-url parameter, then we would be requesting a package from the official index. If all goes well, you should see this:

Looking in indexes: https://test.pypi.org/simple/
Collecting package-boilerplate
  Downloading https://test-files.pythonhosted.org/packages/10/63/208bbd4ea4427f5eb0e4e6248973b387f1dec4ed60333c4daf416c310903/package_boilerplate-1.0.0-py3-none-any.whl
Requirement already satisfied: colored==1.3.93 in /usr/local/lib/python3.7/site-packages (from package-boilerplate) (1.3.93)
Installing collected packages: package-boilerplate
Successfully installed package-boilerplate-1.0.0

If so, then the package is installed! So let's use it. Remember that command-line program we created? Well now we should be able to run it from anywhere by calling invoking the name of the script:

$ boilerplate-cli

And the text is orange like we wanted! What if we wanted to import the module in a different project? Let's try:

$ python
Python 3.7.3 (default, Mar 27 2019, 09:23:15) 
[Clang 10.0.1 (clang-1001.0.46.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

It works! And we even get the coloured text.

Once you publish your package to the official PyPI index, you will be able to install the package by running pip install package-boilerplate. Of course, you would want to choose a different name for the package throughout the process and use that here.