In this guide, I'm going to show you how can easily publish a package to the Python Package Index so that others can install and use your work. Once the package is available on PyPI, anyone will be able to run
pip install to install the package on their machine. This allows people to import your code into their own modules as a library, but you can also release command-line tools via PyPI; I will show you how to do both.
note: these instructions are Mac and Linux centric. I don't think they would be much different, but I don't do much development work on Windows so I don't know the nuances of that platform.
This first thing you need is some code to publish! I have created a minimal project that you can use as a starting point. You can find the code here. Let's take a look.
First let's walk through the folder structure:
1 2 3 4 5 6 7 8 9
These are all of the files inside of the python package. First, we have a directory named
package_boilerplate. This directory contains all of the code for our module. All of the files outside of this directory are metadata for the package itself and are used in the package publishing process, for installing dependencies, and stuff like that. We'll go through the other files shortly.
README.md is markdown formatted text containing details about the package. This content is usually rendered as HTML and made visible in places like GitHub and PyPI.
If we look inside of
__init__.py, we see the following:
1 2 3 4
We're doing a couple of things here: first, we import a library called colored. Colored is a tool to colourize your program's terminal output. It's fun!
So basically, this file will print some coloured text to the terminal if
main is called. If you continue reading, you will see how we can publish this as a command-line tool through
Next, let's look inside of
1 2 3
Again, we're just print some coloured text to the terminal but this time we can import this code in an external module. The text will be printed as soon as the file is imported since the code is not contained in a function.
Ok, so that is the logic of our program. Let's continue down the folder structure to see what else is needed to make this publishable.
The next file is requirements.txt. If you've ever written a python module that depends on other external python packages, then you may be familiar with this file. The file is used to tell
pip which versions of which modules it should install all at once if you run
pip install -r requirements.txt. Python packages are no different; if they rely on dependencies,
pip needs to know what to install. We'll reference this file in
Next, we have the
scripts directory. The folder name doesn't matter and isn't required, but I prefer to keep things separate and organized. Inside of this folder we have a file called
boilerplate-cli. Notice how we don't give the file an extension like
.py and we used the naming convention
boilerplate-cli instead of
boilerplate_cli. This is because this is the file that will be installed as a command-line program in the users
PATH. We want the the user to be able to invoke the program by running
boilerplate-cli and not
boilerplate_cli.py because that is the expected convention for command-line programs.
Inside of this file we have this:
1 2 3 4 5
The first line is important because we have to tell the user's operating system what kind of code this it because it doesn't have a file extension. This is called a shebang. Next we import the entrypoint into the program from
__init__.py that we named
main and we run the function.
Calling this function kicks off our program! Right now it only prints text to the terminal, but it could be the entrypoint into a more complicated program. We will reference this file in the
setup.py section to make it available as a command-line program for the user.
The most important part of this process is the
[setup.py](https://docs.python.org/3.7/distutils/setupscript.html) file. This tells pip how to bundle the package and what all the settings and dependencies are.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
First we import a few utilities called
runpy is built into python, but
setuptools needs to be installed which we will go over in the next section.
Next, we get the python metadata out of the file
version.py and obtain the current version number
VERSION. This file just contains the current version number of the package you are publishing. We'll go over why it's split into its own file in the next section.
long_description is pulled out of the content from
README.md. Using the content from the readme means that this description only needs to live in one place.
Then, I've added a helper function called
parse_requirements. The python
setup.py format expects that
install_requires property contains a list of dependencies that this package relies on. Since it's burdensome to maintain a
requirements.txt file with your dependencies as well as the dependencies in a list here, this
parse_requirements function simple imports the dependencies from
requirements.txt so that you only need to maintain them in one place.
Finally, we call
setup with some values to define the package. The section that specifies
packages=find_packages() is pointing to the directory named
package_boilerplate and is discovered automatically by this function. Name and version are used in the PyPI directory listing and also are what people reference when they want to install the package.
install_requires pulls in the list of requirements from the
python_requires=">=3.6.3" is the version of python that the user must have installed. And finally,
scripts is where you specify the command line program that should be installed for the user.
There are other values that you should set as well such as
author. You can read more about the setup values available here.
The last file,
version.py is quite small. This file is used for two things: first, it is imported in the
setup.py module above to use as the package version; and second, the version value is stored in the
__version__ attribute. This is recommend so that the package version can be discovered programatically.
A tip: as you are working, you can install your package locally before you even publish it. This way, you will know that it works before submitting it to PyPI. To install the package locally, you can run
pip install -e . in the package directory.
Publishing Your Package
So you've written your package and you want to publish it! We're almost there. There are just a few more steps to publish your package.
The first thing will want to do is install a tool called twine by running
pip install twine. Twine is a tool for publishing Python packages on PyPI. Then, we also need to install setuptools since we are using it in our setup script:
pip install setuptools.
Now, we need to create a distribution package that we will publish. To do this, run:
python setup.py sdist bdist_wheel which will create
Once the build has completed, we can check for any errors or warnings using
twine check dist/* to check the distribution package for errors. If all goes well, you should see this:
Once this is done, we want to publish our package to the PyPI test site. We can publish to test.pypi.org before we publish to the real index to make sure everything looks good! To do this, we can instruct
twine to use a different repository like so:
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
If we run this now, we will see the following:
Looks like we need to create an account! So head over to the registration page for the PyPI test site and create an account. If you run the command again and enter your username and password hopefully you should see:
1 2 3 4 5
If you do, congratulations! You have successfully published your package!
Note: make sure your package name is unique!
All that remains is to publish your package to the official pypi.org. To do this, you just need to create another account there and publish your package without specifying a different repository like so:
twine upload dist/*.
Installing Your Package
Now that your package is installed, I bet you want to install it to see how it works! Let's install our package from the PyPI test index:
pip install --index-url https://test.pypi.org/simple/ package-boilerplate. In the future, if we don't provide the
--index-url parameter, then we would be requesting a package from the official index. If all goes well, you should see this:
1 2 3 4 5 6
If so, then the package is installed! So let's use it. Remember that command-line program we created? Well now we should be able to run it from anywhere by calling invoking the name of the script:
1 2 3 4 5
Once you publish your package to the official PyPI index, you will be able to install the package by running
pip install package-boilerplate. Of course, you would want to choose a different name for the package throughout the process and use that here.
I hope this guide is helpful. If something doesn't make sense, please let me know.