Credit: Dreamstime
Powerful and versatile as it is, Python lacks a few key capabilities out of the box. For one, there is no native mechanism for compiling a Python program into a standalone executable package.
To be fair, the original use case for Python never called for standalone redistributables. Python programs have, by and large, been run in place on systems where a copy of the Python interpreter lived. But as the language's popularity grew, there was also a growing demand for running Python applications on systems with no installed Python runtime.
Several third parties have engineered solutions for deploying standalone Python applications. The most popular solution, and the most mature, is PyInstaller. PyInstaller doesn’t make the process of packaging a Python application totally painless, but it goes a long way.
In this article, we’ll explore the basics of using PyInstaller, including how PyInstaller works, how to use PyInstaller to create a standalone Python executable, how to fine-tune the Python executables you create, and how to avoid some of the common pitfalls that go with using it.
Creating a PyInstaller package
PyInstaller is a Python package, installed with pip (pip install pyinstaller). You can install PyInstaller in your default Python installation, but it’s best to create a virtual environment for the project you want to package and install PyInstaller there.
PyInstaller works by reading your Python program, analysing all its imports, and bundling copies of those imports with your program and a copy of the Python runtime.
PyInstaller reads in your program from its entry point. For instance, if your program’s entry point is myapp.py, you would run pyinstaller myapp.py to perform the analysis. PyInstaller can detect and automatically package many common Python packages, like NumPy, but you might need to provide hints in some cases. (More on this later.)
After analysing your code and discovering all the libraries and modules it uses, PyInstaller then generates a “spec file.” A Python script with the extension .spec, this file includes details about how your Python application needs to be packaged. The first time you run PyInstaller on your application, it will generate a spec file from scratch and populate the file with sane defaults. Don’t discard this file; it’s the key to refining a PyInstaller deployment!
Finally, PyInstaller attempts to produce an executable from the application, bundled with all its dependencies. When it’s finished, a subfolder named dist (by default; you are free to specify a different name) will appear in the project directory. This, in turn, contains a directory that is your bundled application—it has an executable file to run, along with all the libraries and other supplemental files required.
All you need to do to distribute your program, then, is package up this directory as a .zip file or with some other bundle. The bundle will typically need to be extracted in a directory where the user has write permissions in order to run.
Testing a PyInstaller package
There’s a fair chance your first attempt at using PyInstaller to package an application won’t be completely successful.
To check whether your PyInstaller package works, navigate to the directory containing the bundled executable and run the executable file from the command line, not by double-clicking it in a visual file explorer. If it fails to run, the errors you’ll see printed to the command line should provide a hint as to what’s wrong.
The most common reason a PyInstaller package fails is that PyInstaller did not bundle a required file. Such missing files fall into a few categories:
- Hidden or missing imports: Sometimes PyInstaller can’t detect the import of a package or library, typically because it is imported dynamically. In this case, you need to manually specify the package or library.
- Missing standalone files: If the program depends on external data files that need to be bundled with the program, PyInstaller has no way of knowing it. You’ll need to manually indicate the files to include.
- Missing binaries: Here again, if your program depends on an external binary like a .DLL that PyInstaller can’t detect, you’ll need to manually include it.
The good news is that PyInstaller provides an easy way to deal with the above problems. The .spec file created by PyInstaller includes fields we can fill in to provide the details that PyInstaller missed.
Editing the spec file
To bundle the additional files you need, open the .spec file in a text editor and look for the definition of the Analysis object. Several parameters passed to Analysis are blank lists, but you can edit them to specify missing details:
hiddenimportsfor hidden or missing imports: Add to this list one or more strings with the names of libraries you want included with your application. If you wanted to addpandasandbokeh, for instance, you would specify that as['pandas','bokeh']. Note that the libraries in question must be installed in the same instance of Python where you’re running PyInstaller—another good argument for using a virtual environment.datasfor missing standalone files: Add here one or more specifications for files in your project tree that you want to include with your project. Each file must be passed as a tuple indicating the relative path to the file in your project directory and the relative path within the distribution directory where you want to place the file. For instance, if you had a file,./models/mainmodel.dat, that you wanted to include with your application, and you wanted to place it in a matching subdirectory in your distribution directory, you would use('./models/mainmodel.dat','./models')as one entry in thehiddenimportslist. Note that you can useglob-style wildcards to specify more than one file.binariesfor missing standalone binaries: As withdatas, you can usebinariesto pass a list of tuples that specify the locations of binaries in the project tree and their destinations in the distribution directory. Again, you can useglob-style wildcards.
Keep in mind that any of the lists passed to Analysis can be programmatically generated earlier in the .spec file. After all, the .spec file is just a Python script by another name.
After you make changes to the .spec file, rerun PyInstaller to rebuild the package. However, from now on, be sure to pass the modified .spec file as the parameter (e.g., pyinstaller myapp.spec). Test the executable as before. If something is still broken, you can re-edit the .spec file and repeat the process until everything works.
Finally, when you’re satisfied everything works as intended, you might want to edit the .spec file to prevent your packaged application from presenting a command-line window when launched. In the EXE object settings in the .spec file, set console=False. Suppressing the console is useful if your application has a GUI and you don’t want a spurious command-line window leading users astray. Of course, don’t change this setting if your application requires a command line.
Refining a PyInstaller package
Once you have your application packaged with PyInstaller and running properly, the next thing you’ll likely want to do is slim it down a little. PyInstaller packages are not known for being svelte.
Because Python is a dynamic language, it’s difficult to predict just what will be needed at runtime by a given program. For that reason, when PyInstaller detects a package import, it includes everything in that package, whether or not it’s actually used at runtime by your program.
Here’s the good news, though: PyInstaller includes a mechanism for selectively excluding entire packages, or individual namespaces within packages. For instance, let’s say your program imports package foo, which includes foo.bar and foo.bip. If you know for a fact that your program only uses logic in foo.bar, you can safely exclude foo.bip and save some space.
To do this, you use the excludes parameter passed to the Analysis object in the .spec file. You can pass a list of names—top-level modules or dotted namespaces—to exclude from your package. For example, to exclude foo.bip, you would simply specify ['foo.bip'].
IDG
PyInstaller creates the deliverable directory. Because of the large number of files in the directory, a third-party installer project like NSIS may be used to pack up the directory into a self-extracting archive, and automatically create a shortcut to the executable.
One common exclusion is test suites. If a package your program imports has a test suite, the test suite could end up being included in your PyInstaller package. Unless you actually run the test suite in your deployed program, you can safely exclude it.
Another common exclusion is tkinter, the Python library for creating simple cross-platform graphical user interfaces. By default tkinter should not be included (older versions of PyInstaller used to include it by default), but if it is, you can exclude it by adding 'tkinter' to the excludes list. Omitting tkinter will reduce the size of the package by around 7MB.
Bear in mind that packages created using exclusions should be thoroughly tested. If you end up excluding functionality that is used in some future scenario you didn’t anticipate, your application will break.
PyInstaller tips
Let's close with best practices and mistakes to avoid when using PyInstaller.
- Build your PyInstaller package on the deployment operating system: PyInstaller does not support cross-platform builds. If you need to deploy your standalone Python application on MacOS, Linux, and Windows systems, then you will need to install PyInstaller and build separate versions of the application on each of these operating systems.
- Build your PyInstaller package as you develop your app: As soon as you know you will be deploying your project with PyInstaller, build your
.specfile and start refining the PyInstaller package in parallel with the development of your application. This way, you can add exclusions or inclusions as you go, and test the way new features are deployed with the application as you write them. - For apps with a module entry point, make a stub: Some applications use a module as an entry point, for instance by having a
__main__.pyfile that's invoked when the module is imported. PyInstaller doesn't have a mechanism for using such a module as an entry point. To make that work, create a "stub" file—a .py file in the top levet of your project that takes the same steps to run your application as the contents of the__main__.pyfile. Then feed PyInstaller the stub to perform the analysis. - Don’t use PyInstaller’s --onefile mode: PyInstaller includes a command line switch,
--onefile, that packs your entire application into a single self-extracting executable. This sounds like a great idea—you only have to deliver one file!—but it has pitfalls. On WIndows, whenever you run the application, it must first unpack all of the files within the executable to a temporary directory. If the application is big (200MB, for instance), unpacking can mean a delay of several seconds. To get around this, use the default single-directory mode instead, and just pack everything up as a.zipfile. - Create an installer for your PyInstaller app: If you want a way to deploy your application other than a .zip file or the single-file distribution, consider using an installer utility like the open source Nullsoft Scriptable Install System. It adds very little overhead to the size of the deliverable and lets you configure many aspects of the installation process, like creating shortcuts to your executable.
- Use code signing on WIndows to mark the generated executables: If you have a code-signing certificate and want to use it to sign a PyInstaller-generated project to keep it from being flagged as malware, there's a recipe for doing this. To save time, you can integrate the signing process into the project's .spec file.
- Don’t expect speedups: PyInstaller is a packaging system, not a compiler or an optimiser. Code packaged with PyInstaller does not run any faster than it would when run on the original system. If you want to speed up Python code, use a C-accelerated library suited to the task, or a project like Cython.
