A couple of years back, I was developing an application for Ubuntu. For unspecified reasons, I wanted to develop the GUI in Qt/C++, but implement all logic in Python. The default Python that came with Ubuntu at the time could not run my scripts, so I had to ship a compatible Python version along with my application. Basically, I had to embed the Python interpreter into my program, by linking to it, and then load it at runtime to run my Python scripts. As it turned out, my app was a flop, but at least I got to make a tutorial out of it.
In this tutorial, I'll show you how to embed the Python interpreter in your own Qt/C++ applications. We'll be creating an application that minifies CSS stylesheet files. Actually, only the GUI will be in Qt, the CSS minification will be handled by Python using rCSSmin—a CSS minifier written in Python. We'll statically link the Python interpreter to our application, and distribute the Python standard library in an archive, which will be loaded at runtime. That's the only external file we'll be shipping with our application, since the rCSSmin module will be stored in our program's executable using Qt's resource system. Pretty cool stuff, right?
This tutorial was written for Ubuntu 15.04, nevertheless, much of the information is still useful for other Ubuntu releases, Linux distributions, Mac, or Windows.
Prepare the Development Environment
Install the development tools (typically come pre-installed):
$ sudo apt-get install build-essential
Install Qt Creator:
$ sudo apt-get install qtcreator
Install the Python static library and other development files:
$ sudo apt-get install python3-dev
Python Script
rCSSmin is a stand-alone application, however, even the simplest Python modules depend on Python's standard library to run. Since Python's standard library is very extensive, it's not very efficient to pack and distribute the whole library with our Qt application. For that reason, we'll create a script that determines rCSSmin dependencies, compiles them, and conveniently packs them in a zip archive. To further save space, we'll compile rCSSmin to bytecode and store it in a binary file.
Create a file named compile.py
:
#!/usr/bin/env python3
import os
import sys
from modulefinder import ModuleFinder
import zipfile
import py_compile
import marshal
from random import randint
RUN_DIR = os.path.dirname(os.path.realpath(__file__))
STD_LIB = os.path.join(sys.exec_prefix, 'lib', 'python%d.%d' % (sys.version_info.major, sys.version_info.minor))
STD_LIB_OUT = os.path.join(RUN_DIR, 'libpy%d%d.zip' % (sys.version_info.major, sys.version_info.minor))
DEP_OUT = os.path.join(RUN_DIR, 'dep.txt')
TMP_FILE = os.path.join(RUN_DIR, '~tmp%d' % randint(0,1000000))
def create_pyc(src_file):
if not os.path.isfile(src_file):
raise Exception('File "%s" does not exist.' % src_file)
if os.path.exists(TMP_FILE):
raise Exception('Path "%s" exists.' % TMP_FILE)
py_compile.compile(src_file, cfile=TMP_FILE, doraise=True)
with open(TMP_FILE, 'rb') as f:
pyc = f.read()
if os.path.isfile(TMP_FILE):
os.remove(TMP_FILE)
return pyc
def create_code_obj(src_module, dst_file):
"""Compiles the source module to a Python code object and serializes
using the marshal format"""
if not os.path.isfile(src_module):
raise Exception('File "%s" does not exist.' % src_module)
with open(src_module, 'r') as f:
code_str = f.read()
print('Writing code object to "%s"...' % dst_file)
with open(dst_file, 'wb') as f:
marshal.dump(compile(code_str, '<string>', 'exec', optimize=2), f)
def zip_std_lib(src_module, dst_file):
"""Compiles the Python standard library modules used by the source module
and outputs to zip file."""
finder = ModuleFinder()
finder.run_script(src_module)
modules = set()
print('Writing dependencies to "%s"...' % DEP_OUT)
with open(DEP_OUT, 'w') as f:
for name, mod in finder.modules.items():
print('%s: ' % name, end='', file=f)
print(mod.__file__, file=f)
if mod.__file__ is None:
continue
path = os.path.realpath(mod.__file__)
if not path.startswith(os.path.normpath(STD_LIB)):
continue
while(os.path.dirname(path) != os.path.normpath(STD_LIB)):
path = os.path.dirname(path)
if os.path.isfile(path):
modules.add(path)
elif os.path.isdir(path):
for root, dirs, files in os.walk(path):
for i in files:
modules.add(os.path.join(root, i))
print('-' * 50, file=f)
print('### Modules NOT imported ###', file=f)
print('\n'.join(finder.badmodules.keys()), file=f)
modules = sorted([i for i in modules if i.endswith(('.py','.pyc')) and not os.path.dirname(i).endswith('__pycache__')])
print('Writing standard library to "%s"...' % dst_file)
with zipfile.ZipFile(dst_file, 'w', compression=zipfile.ZIP_DEFLATED) as z:
for i in modules:
root, ext = os.path.splitext(i)
if ext == '.py':
arcname = os.path.relpath(root, STD_LIB) + '.pyc'
pyc = create_pyc(i)
else:
arcname = os.path.relpath(i, STD_LIB)
with open(i, 'rb') as f:
pyc = f.read()
z.writestr(arcname, pyc)
if __name__ == "__main__":
if len(sys.argv) > 1:
src_module = os.path.join(RUN_DIR, sys.argv[1])
else:
raise Exception('Source module not specified.')
if not os.path.isfile(src_module):
raise Exception('File "%s" does not exist.' % src_module)
zip_std_lib(src_module, STD_LIB_OUT)
create_code_obj(src_module, os.path.join(RUN_DIR, os.path.basename(src_module) + '.codeobj'))
If you haven't done so already, download rCSSmin, and uncompress it. To use the above script, execute:
$ ./compile.py PATH/TO/rcssmin.py
The script will output 3 files:
-
libpy34.zip
: A subset of the Python standard library in a zip archive (The suffix number could be different, depending on your Ubuntu release). -
rcssmin.py.codeobj
: The rCSSmin code object, serialized using Marshal. -
dep.txt
: A list of rCSSmin dependencies, as determined by the script.
Qt Application
Run Qt Creator and create a new Qt Widget Application. Name the project embedPython
.
Create a new directory named res
in your project's root directory and copy rcssmin.py.codeobj
. In your project, add a new Qt Resource File named embedPython.qrc
. Open the resource file to add rcssmin.py.codeobj
with /
prefix.
Open embedPython.pro
and add the following lines at the end:
INCLUDEPATH += /usr/include/python3.4m
LIBS += -L/usr/lib/x86_64-linux-gnu
LIBS += -Wl,-Bstatic -lpython3.4m -Wl,-Bdynamic
LIBS += -lz -lexpat -ldl -lutil
Python 3.4 is the version of Python shipped with Ubuntu 15.04. If you are running a different release, you should find out which Python version you have installed, and then modify embedPython.pro
accordingly:
$ find /usr/include/ -maxdepth 1 -name python*
/usr/include/python3.4
/usr/include/python3.4m
/usr/include/python2.7
Open embedPython.ui
in the form editor. Add 2 plainTextEdit widgets and a pushButton, as shown in the Figure below:
In your project, create a new class named PyRun
. We will use PyRun
to initialize the Python interpreter, load the rCSSmin module using Qt's resource system, and execute the cssmin
function inside the rCSSmin module.
Open pyrun.h
:
#ifndef PYRUN_H
#define PYRUN_H
#include <Python.h>
#include <marshal.h>
#include <QString>
class PyRun
{
public:
PyRun(QString);
~PyRun();
QString cssmin(QString);
private:
std::wstring execFile;
std::wstring pythonPath;
bool hasError();
PyObject* importModule(const QByteArray&, const QString&);
PyObject* callFunction(PyObject*, QString, PyObject*);
QString ObjectToString(PyObject*);
};
#endif // PYRUN_H
Open pyrun.cpp
:
#include "pyrun.h"
#include <QString>
#include <QStringList>
#include <QDir>
#include <QFileInfo>
#include <QDebug>
PyRun::PyRun(QString execFile)
{
this->execFile = execFile.toStdWString();
QStringList pythonPath;
pythonPath << QDir::toNativeSeparators(QFileInfo(QFileInfo(execFile).absoluteDir(), "libpy34.zip").canonicalFilePath());
this->pythonPath = pythonPath.join(":").toStdWString();
// Path of our executable
Py_SetProgramName((wchar_t*) this->execFile.c_str());
// Set module search path
Py_SetPath(this->pythonPath.c_str());
Py_NoSiteFlag = 1;
// Initialize the Python interpreter
Py_InitializeEx(0);
qDebug() << "Python interpreter version:" << QString(Py_GetVersion());
qDebug() << "Python standard library path:" << QString::fromWCharArray(Py_GetPath());
QFile f("://res/rcssmin.py.codeobj");
if(f.open(QIODevice::ReadOnly))
{
QByteArray codeObj = f.readAll();
f.close();
importModule(codeObj, "rcssmin");
}
}
PyRun::~PyRun()
{
Py_Finalize();
}
PyObject* PyRun::importModule(const QByteArray& codeObj, const QString& moduleName)
{
PyObject *poModule = NULL;
// Get reference to main module
PyObject *mainModule = PyImport_AddModule("__main__");
// De-serialize Python code object
PyObject *poCodeObj = PyMarshal_ReadObjectFromString((char*)codeObj.data(), codeObj.size());
if(!hasError())
{
// Load module from code object
poModule = PyImport_ExecCodeModule(moduleName.toUtf8().data(), poCodeObj);
if(!hasError())
{
// Add module to main module as moduleName
PyModule_AddObject(mainModule, moduleName.toUtf8().data(), poModule);
}
// Release object reference (Python cannot track references automatically in C++!)
Py_XDECREF(poCodeObj);
}
return poModule;
}
PyObject *PyRun::callFunction(PyObject *poModule, QString funcName, PyObject *poArgs)
{
PyObject *poRet = NULL;
// Get reference to function funcName in module poModule
PyObject *poFunc = PyObject_GetAttrString(poModule, funcName.toUtf8().data());
if(!hasError())
{
// Call function with arguments poArgs
poRet = PyObject_CallObject(poFunc, poArgs);
if(hasError())
{
poRet = NULL;
}
// Release reference to function
Py_XDECREF(poFunc);
}
// Release reference to arguments
Py_XDECREF(poArgs);
return poRet;
}
QString PyRun::cssmin(QString css)
{
QString res;
// Get reference to rcssmin module
PyObject *poModule = PyImport_AddModule("rcssmin");
if(!hasError())
{
PyObject *poRet = callFunction(poModule, "cssmin", Py_BuildValue("(s)", css.toUtf8().data()));
res = ObjectToString(poRet);
}
return res;
}
QString PyRun::ObjectToString(PyObject *poVal)
{
QString val;
if(poVal != NULL)
{
if(PyUnicode_Check(poVal))
{
// Convert Python Unicode object to UTF8 and return pointer to buffer
char *str = PyUnicode_AsUTF8(poVal);
if(!hasError())
{
val = QString(str);
}
}
// Release reference to object
Py_XDECREF(poVal);
}
return val;
}
bool PyRun::hasError()
{
bool error = false;
if(PyErr_Occurred())
{
// Output error to stderr and clear error indicator
PyErr_Print();
error = true;
}
return error;
}
In our MainWindow class, we handle the pushButton click action, run cssmin
, and display the result in the plainTextEdit widget.
Open mainwindow.h
:
#ifndef MAINWINDOW_H
#define MAINWINDOW_H
#include "pyrun.h"
#include <QMainWindow>
namespace Ui {
class MainWindow;
}
class MainWindow : public QMainWindow
{
Q_OBJECT
public:
explicit MainWindow(PyRun *pyRun, QWidget *parent = 0);
~MainWindow();
private slots:
void on_pushButton_clicked();
private:
Ui::MainWindow *ui;
PyRun *pyRun;
};
#endif // MAINWINDOW_H
Open mainwindow.cpp
:
#include "mainwindow.h"
#include "ui_mainwindow.h"
MainWindow::MainWindow(PyRun *pyRun, QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow)
{
ui->setupUi(this);
this->pyRun = pyRun;
}
MainWindow::~MainWindow()
{
delete ui;
}
void MainWindow::on_pushButton_clicked()
{
ui->plainTextEdit_2->setPlainText(pyRun->cssmin(ui->plainTextEdit->toPlainText()));
}
Finally, in main we create an instance of PyRun
, and then pass it to our mainwindow. Open main.cpp
:
#include "pyrun.h"
#include "mainwindow.h"
#include <QApplication>
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
PyRun *pyRun = new PyRun(a.applicationFilePath());
MainWindow w(pyRun);
w.show();
return a.exec();
}
Testing the Application
Before running your Qt application for the first time, make sure that you copy libpy34.zip
in the same directory as the embedPython
executable. Then, download a large CSS file, such as the CSS stylesheet of the jQuery Mobile framework, and copy-paste the CSS contents in the Input plainTextEdit widget. After clicking on Minify, you should see the minified text in the Output plainTextEdit, as shown below:
As mentioned earlier, the Python interpreter is statically linked to our application. To verify, you can use the ldd
command, which prints a list of all shared libraries required by a program. If the Python interpreter is indeed statically linked to our application, libpython3.4m.so.1.0
should not be in the ldd
list:
$ ldd embedPython
linux-vdso.so.1 => (0x00007fff34546000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fe1324f0000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fe1322c7000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe1320c3000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fe131ec0000)
libQt5Widgets.so.5 => /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5 (0x00007fe131819000)
libQt5Gui.so.5 => /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5 (0x00007fe1312cc000)
libQt5Core.so.5 => /usr/lib/x86_64-linux-gnu/libQt5Core.so.5 (0x00007fe130d89000)
...
You can find all source code in this tutorial in my GitHub repository. If you spot any errors, or have comments, don't hesitate to contact me.