A couple of years back, I was developing an application for Ubuntu. For unspecified reasons, I wanted to develop the GUI in Qt/C++, but implement all logic in Python. The default Python that came with Ubuntu at the time could not run my scripts, so I had to ship a compatible Python version along with my application. Basically, I had to embed the Python interpreter into my program, by linking to it, and then load it at runtime to run my Python scripts. As it turned out, my app was a flop, but at least I got to make a tutorial out of it.

In this tutorial, I'll show you how to embed the Python interpreter in your own Qt/C++ applications. We'll be creating an application that minifies CSS stylesheet files. Actually, only the GUI will be in Qt, the CSS minification will be handled by Python using rCSSmin—a CSS minifier written in Python. We'll statically link the Python interpreter to our application, and distribute the Python standard library in an archive, which will be loaded at runtime. That's the only external file we'll be shipping with our application, since the rCSSmin module will be stored in our program's executable using Qt's resource system. Pretty cool stuff, right?

This tutorial was written for Ubuntu 15.04, nevertheless, much of the information is still useful for other Ubuntu releases, Linux distributions, Mac, or Windows.

Prepare the Development Environment

Install the development tools (typically come pre-installed):

$ sudo apt-get install build-essential

Install Qt Creator:

$ sudo apt-get install qtcreator

Install the Python static library and other development files:

$ sudo apt-get install python3-dev

Python Script

rCSSmin is a stand-alone application, however, even the simplest Python modules depend on Python's standard library to run. Since Python's standard library is very extensive, it's not very efficient to pack and distribute the whole library with our Qt application. For that reason, we'll create a script that determines rCSSmin dependencies, compiles them, and conveniently packs them in a zip archive. To further save space, we'll compile rCSSmin to bytecode and store it in a binary file.

Create a file named compile.py:

#!/usr/bin/env python3

import os
import sys
from modulefinder import ModuleFinder
import zipfile
import py_compile
import marshal
from random import randint

RUN_DIR = os.path.dirname(os.path.realpath(__file__))
STD_LIB = os.path.join(sys.exec_prefix, 'lib', 'python%d.%d' % (sys.version_info.major, sys.version_info.minor))
STD_LIB_OUT = os.path.join(RUN_DIR, 'libpy%d%d.zip' % (sys.version_info.major, sys.version_info.minor))
DEP_OUT = os.path.join(RUN_DIR, 'dep.txt')
TMP_FILE = os.path.join(RUN_DIR, '~tmp%d' % randint(0,1000000))

def create_pyc(src_file):
    if not os.path.isfile(src_file):
        raise Exception('File "%s" does not exist.' % src_file)

    if os.path.exists(TMP_FILE):
        raise Exception('Path "%s" exists.' % TMP_FILE)

    py_compile.compile(src_file, cfile=TMP_FILE, doraise=True)

    with open(TMP_FILE, 'rb') as f:
        pyc = f.read()

    if os.path.isfile(TMP_FILE):
        os.remove(TMP_FILE)

    return pyc

def create_code_obj(src_module, dst_file):
    """Compiles the source module to a Python code object and serializes
    using the marshal format"""

    if not os.path.isfile(src_module):
        raise Exception('File "%s" does not exist.' % src_module)

    with open(src_module, 'r') as f:
        code_str = f.read()

    print('Writing code object to "%s"...' % dst_file)

    with open(dst_file, 'wb') as f:
        marshal.dump(compile(code_str, '<string>', 'exec', optimize=2), f)

def zip_std_lib(src_module, dst_file):
    """Compiles the Python standard library modules used by the source module
    and outputs to zip file."""

    finder = ModuleFinder()
    finder.run_script(src_module)

    modules = set()

    print('Writing dependencies to "%s"...' % DEP_OUT)

    with open(DEP_OUT, 'w') as f:
        for name, mod in finder.modules.items():
            print('%s: ' % name, end='', file=f)
            print(mod.__file__, file=f)

            if mod.__file__ is None:
                continue

            path = os.path.realpath(mod.__file__)

            if not path.startswith(os.path.normpath(STD_LIB)):
                continue

            while(os.path.dirname(path) != os.path.normpath(STD_LIB)):
                path = os.path.dirname(path)

            if os.path.isfile(path):
                modules.add(path)
            elif os.path.isdir(path):
                for root, dirs, files in os.walk(path):
                    for i in files:
                        modules.add(os.path.join(root, i))

        print('-' * 50, file=f)
        print('### Modules NOT imported ###', file=f)
        print('\n'.join(finder.badmodules.keys()), file=f)

    modules = sorted([i for i in modules if i.endswith(('.py','.pyc')) and not os.path.dirname(i).endswith('__pycache__')])

    print('Writing standard library to "%s"...' % dst_file)

    with zipfile.ZipFile(dst_file, 'w', compression=zipfile.ZIP_DEFLATED) as z:
        for i in modules:
            root, ext = os.path.splitext(i)

            if ext == '.py':
                arcname = os.path.relpath(root, STD_LIB) + '.pyc'
                pyc = create_pyc(i)
            else:
                arcname = os.path.relpath(i, STD_LIB)
                with open(i, 'rb') as f:
                    pyc = f.read()

            z.writestr(arcname, pyc)

if __name__ == "__main__":
    if len(sys.argv) > 1:
        src_module = os.path.join(RUN_DIR, sys.argv[1])
    else:
        raise Exception('Source module not specified.')

    if not os.path.isfile(src_module):
        raise Exception('File "%s" does not exist.' % src_module)

    zip_std_lib(src_module, STD_LIB_OUT)
    create_code_obj(src_module, os.path.join(RUN_DIR, os.path.basename(src_module) + '.codeobj'))

If you haven't done so already, download rCSSmin, and uncompress it. To use the above script, execute:

$ ./compile.py PATH/TO/rcssmin.py

The script will output 3 files:

1. libpy34.zip: A subset of the Python standard library in a zip archive (The suffix number could be different, depending on your Ubuntu release).

2. rcssmin.py.codeobj: The rCSSmin code object, serialized using Marshal.

3. dep.txt: A list of rCSSmin dependencies, as determined by the script.

Qt Application

Run Qt Creator and create a new Qt Widget Application. Name the project embedPython.

Create a new directory named res in your project's root directory and copy rcssmin.py.codeobj. In your project, add a new Qt Resource File named embedPython.qrc. Open the resource file to add rcssmin.py.codeobj with / prefix.

Open embedPython.pro and add the following lines at the end:

INCLUDEPATH += /usr/include/python3.4m
LIBS += -L/usr/lib/x86_64-linux-gnu
LIBS += -Wl,-Bstatic -lpython3.4m -Wl,-Bdynamic
LIBS += -lz -lexpat -ldl -lutil

Python 3.4 is the version of Python shipped with Ubuntu 15.04. If you are running a different release, you should find out which Python version you have installed, and then modify embedPython.pro accordingly:

$ find /usr/include/ -maxdepth 1 -name python*

/usr/include/python3.4
/usr/include/python3.4m
/usr/include/python2.7

Open embedPython.ui in the form editor. Add 2 plainTextEdit widgets and a pushButton, as shown in the Figure below:

In your project, create a new class named PyRun. We will use PyRun to initialize the Python interpreter, load the rCSSmin module using Qt's resource system, and execute the cssmin function inside the rCSSmin module.

Open pyrun.h:

#ifndef PYRUN_H
#define PYRUN_H

#include <Python.h>
#include <marshal.h>

#include <QString>

class PyRun
{
public:
    PyRun(QString);
    ~PyRun();
    QString cssmin(QString);

private:
    std::wstring execFile;
    std::wstring pythonPath;
    bool hasError();
    PyObject* importModule(const QByteArray&, const QString&);
    PyObject* callFunction(PyObject*, QString, PyObject*);
    QString ObjectToString(PyObject*);
};

#endif // PYRUN_H

Open pyrun.cpp:

#include "pyrun.h"

#include <QString>
#include <QStringList>
#include <QDir>
#include <QFileInfo>
#include <QDebug>

PyRun::PyRun(QString execFile)
{
    this->execFile = execFile.toStdWString();

    QStringList pythonPath;
    pythonPath << QDir::toNativeSeparators(QFileInfo(QFileInfo(execFile).absoluteDir(), "libpy34.zip").canonicalFilePath());

    this->pythonPath = pythonPath.join(":").toStdWString();

    // Path of our executable
    Py_SetProgramName((wchar_t*) this->execFile.c_str());

    // Set module search path
    Py_SetPath(this->pythonPath.c_str());

    Py_NoSiteFlag = 1;

    // Initialize the Python interpreter
    Py_InitializeEx(0);

    qDebug() << "Python interpreter version:" << QString(Py_GetVersion());
    qDebug() << "Python standard library path:" << QString::fromWCharArray(Py_GetPath());

    QFile f("://res/rcssmin.py.codeobj");

    if(f.open(QIODevice::ReadOnly))
    {
        QByteArray codeObj = f.readAll();
        f.close();
        importModule(codeObj, "rcssmin");
    }
}

PyRun::~PyRun()
{
    Py_Finalize();
}

PyObject* PyRun::importModule(const QByteArray& codeObj, const QString& moduleName)
{
    PyObject *poModule = NULL;

    // Get reference to main module
    PyObject *mainModule = PyImport_AddModule("__main__");

    // De-serialize Python code object
    PyObject *poCodeObj = PyMarshal_ReadObjectFromString((char*)codeObj.data(), codeObj.size());

    if(!hasError())
    {
        // Load module from code object
        poModule = PyImport_ExecCodeModule(moduleName.toUtf8().data(), poCodeObj);

        if(!hasError())
        {
            // Add module to main module as moduleName
            PyModule_AddObject(mainModule, moduleName.toUtf8().data(), poModule);
        }

        // Release object reference (Python cannot track references automatically in C++!)
        Py_XDECREF(poCodeObj);
    }

    return poModule;
}

PyObject *PyRun::callFunction(PyObject *poModule, QString funcName, PyObject *poArgs)
{
    PyObject *poRet = NULL;

    // Get reference to function funcName in module poModule
    PyObject *poFunc = PyObject_GetAttrString(poModule, funcName.toUtf8().data());

    if(!hasError())
    {
        // Call function with arguments poArgs
        poRet = PyObject_CallObject(poFunc, poArgs);

        if(hasError())
        {
            poRet = NULL;
        }

        // Release reference to function
        Py_XDECREF(poFunc);
    }

    // Release reference to arguments
    Py_XDECREF(poArgs);

    return poRet;
}

QString PyRun::cssmin(QString css)
{
    QString res;

    // Get reference to rcssmin module
    PyObject *poModule = PyImport_AddModule("rcssmin");

    if(!hasError())
    {
        PyObject *poRet = callFunction(poModule, "cssmin", Py_BuildValue("(s)", css.toUtf8().data()));
        res = ObjectToString(poRet);
    }

    return res;
}

QString PyRun::ObjectToString(PyObject *poVal)
{
    QString val;

    if(poVal != NULL)
    {
        if(PyUnicode_Check(poVal))
        {
            // Convert Python Unicode object to UTF8 and return pointer to buffer
            char *str = PyUnicode_AsUTF8(poVal);

            if(!hasError())
            {
                val = QString(str);
            }
        }

        // Release reference to object
        Py_XDECREF(poVal);
    }

    return val;
}

bool PyRun::hasError()
{
    bool error = false;

    if(PyErr_Occurred())
    {
        // Output error to stderr and clear error indicator
        PyErr_Print();
        error = true;
    }

    return error;
}

In our MainWindow class, we handle the pushButton click action, run cssmin, and display the result in the plainTextEdit widget.

Open mainwindow.h:

#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include "pyrun.h"

#include <QMainWindow>

namespace Ui {
class MainWindow;
}

class MainWindow : public QMainWindow
{
    Q_OBJECT

public:
    explicit MainWindow(PyRun *pyRun, QWidget *parent = 0);
    ~MainWindow();

private slots:
    void on_pushButton_clicked();

private:
    Ui::MainWindow *ui;
    PyRun *pyRun;
};

#endif // MAINWINDOW_H

Open mainwindow.cpp:

#include "mainwindow.h"
#include "ui_mainwindow.h"

MainWindow::MainWindow(PyRun *pyRun, QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);
    this->pyRun = pyRun;
}

MainWindow::~MainWindow()
{
    delete ui;
}

void MainWindow::on_pushButton_clicked()
{
    ui->plainTextEdit_2->setPlainText(pyRun->cssmin(ui->plainTextEdit->toPlainText()));
}

Finally, in main we create an instance of PyRun, and then pass it to our mainwindow. Open main.cpp:

#include "pyrun.h"
#include "mainwindow.h"

#include <QApplication>

int main(int argc, char *argv[])
{
    QApplication a(argc, argv);
    PyRun *pyRun = new PyRun(a.applicationFilePath());
    MainWindow w(pyRun);
    w.show();

    return a.exec();
}

Testing the Application

Before running your Qt application for the first time, make sure that you copy libpy34.zip in the same directory as the embedPython executable. Then, download a large CSS file, such as the CSS stylesheet of the jQuery Mobile framework, and copy-paste the CSS contents in the Input plainTextEdit widget. After clicking on Minify, you should see the minified text in the Output plainTextEdit, as shown below:

As mentioned earlier, the Python interpreter is statically linked to our application. To verify, you can use the ldd command, which prints a list of all shared libraries required by a program. If the Python interpreter is indeed statically linked to our application, libpython3.4m.so.1.0 should not be in the ldd list:

$ ldd embedPython

linux-vdso.so.1 =>  (0x00007fff34546000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fe1324f0000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fe1322c7000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe1320c3000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fe131ec0000)
libQt5Widgets.so.5 => /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5 (0x00007fe131819000)
libQt5Gui.so.5 => /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5 (0x00007fe1312cc000)
libQt5Core.so.5 => /usr/lib/x86_64-linux-gnu/libQt5Core.so.5 (0x00007fe130d89000)
...

You can find all source code in this tutorial in my GitHub repository. If you spot any errors, or have comments, don't hesitate to contact me.