1 .. highlight:: c 2 3 4 .. _embedding: 5 6 *************************************** 7 Embedding Python in Another Application 8 *************************************** 9 10 The previous chapters discussed how to extend Python, that is, how to extend the 11 functionality of Python by attaching a library of C functions to it. It is also 12 possible to do it the other way around: enrich your C/C++ application by 13 embedding Python in it. Embedding provides your application with the ability to 14 implement some of the functionality of your application in Python rather than C 15 or C++. This can be used for many purposes; one example would be to allow users 16 to tailor the application to their needs by writing some scripts in Python. You 17 can also use it yourself if some of the functionality can be written in Python 18 more easily. 19 20 Embedding Python is similar to extending it, but not quite. The difference is 21 that when you extend Python, the main program of the application is still the 22 Python interpreter, while if you embed Python, the main program may have nothing 23 to do with Python --- instead, some parts of the application occasionally call 24 the Python interpreter to run some Python code. 25 26 So if you are embedding Python, you are providing your own main program. One of 27 the things this main program has to do is initialize the Python interpreter. At 28 the very least, you have to call the function :c:func:`Py_Initialize`. There are 29 optional calls to pass command line arguments to Python. Then later you can 30 call the interpreter from any part of the application. 31 32 There are several different ways to call the interpreter: you can pass a string 33 containing Python statements to :c:func:`PyRun_SimpleString`, or you can pass a 34 stdio file pointer and a file name (for identification in error messages only) 35 to :c:func:`PyRun_SimpleFile`. You can also call the lower-level operations 36 described in the previous chapters to construct and use Python objects. 37 38 39 .. seealso:: 40 41 :ref:`c-api-index` 42 The details of Python's C interface are given in this manual. A great deal of 43 necessary information can be found here. 44 45 46 .. _high-level-embedding: 47 48 Very High Level Embedding 49 ========================= 50 51 The simplest form of embedding Python is the use of the very high level 52 interface. This interface is intended to execute a Python script without needing 53 to interact with the application directly. This can for example be used to 54 perform some operation on a file. :: 55 56 #define PY_SSIZE_T_CLEAN 57 #include <Python.h> 58 59 int 60 main(int argc, char *argv[]) 61 { 62 wchar_t *program = Py_DecodeLocale(argv[0], NULL); 63 if (program == NULL) { 64 fprintf(stderr, "Fatal error: cannot decode argv[0]\n"); 65 exit(1); 66 } 67 Py_SetProgramName(program); /* optional but recommended */ 68 Py_Initialize(); 69 PyRun_SimpleString("from time import time,ctime\n" 70 "print('Today is', ctime(time()))\n"); 71 if (Py_FinalizeEx() < 0) { 72 exit(120); 73 } 74 PyMem_RawFree(program); 75 return 0; 76 } 77 78 The :c:func:`Py_SetProgramName` function should be called before 79 :c:func:`Py_Initialize` to inform the interpreter about paths to Python run-time 80 libraries. Next, the Python interpreter is initialized with 81 :c:func:`Py_Initialize`, followed by the execution of a hard-coded Python script 82 that prints the date and time. Afterwards, the :c:func:`Py_FinalizeEx` call shuts 83 the interpreter down, followed by the end of the program. In a real program, 84 you may want to get the Python script from another source, perhaps a text-editor 85 routine, a file, or a database. Getting the Python code from a file can better 86 be done by using the :c:func:`PyRun_SimpleFile` function, which saves you the 87 trouble of allocating memory space and loading the file contents. 88 89 90 .. _lower-level-embedding: 91 92 Beyond Very High Level Embedding: An overview 93 ============================================= 94 95 The high level interface gives you the ability to execute arbitrary pieces of 96 Python code from your application, but exchanging data values is quite 97 cumbersome to say the least. If you want that, you should use lower level calls. 98 At the cost of having to write more C code, you can achieve almost anything. 99 100 It should be noted that extending Python and embedding Python is quite the same 101 activity, despite the different intent. Most topics discussed in the previous 102 chapters are still valid. To show this, consider what the extension code from 103 Python to C really does: 104 105 #. Convert data values from Python to C, 106 107 #. Perform a function call to a C routine using the converted values, and 108 109 #. Convert the data values from the call from C to Python. 110 111 When embedding Python, the interface code does: 112 113 #. Convert data values from C to Python, 114 115 #. Perform a function call to a Python interface routine using the converted 116 values, and 117 118 #. Convert the data values from the call from Python to C. 119 120 As you can see, the data conversion steps are simply swapped to accommodate the 121 different direction of the cross-language transfer. The only difference is the 122 routine that you call between both data conversions. When extending, you call a 123 C routine, when embedding, you call a Python routine. 124 125 This chapter will not discuss how to convert data from Python to C and vice 126 versa. Also, proper use of references and dealing with errors is assumed to be 127 understood. Since these aspects do not differ from extending the interpreter, 128 you can refer to earlier chapters for the required information. 129 130 131 .. _pure-embedding: 132 133 Pure Embedding 134 ============== 135 136 The first program aims to execute a function in a Python script. Like in the 137 section about the very high level interface, the Python interpreter does not 138 directly interact with the application (but that will change in the next 139 section). 140 141 The code to run a function defined in a Python script is: 142 143 .. literalinclude:: ../includes/run-func.c 144 145 146 This code loads a Python script using ``argv[1]``, and calls the function named 147 in ``argv[2]``. Its integer arguments are the other values of the ``argv`` 148 array. If you :ref:`compile and link <compiling>` this program (let's call 149 the finished executable :program:`call`), and use it to execute a Python 150 script, such as: 151 152 .. code-block:: python 153 154 def multiply(a,b): 155 print("Will compute", a, "times", b) 156 c = 0 157 for i in range(0, a): 158 c = c + b 159 return c 160 161 then the result should be: 162 163 .. code-block:: shell-session 164 165 $ call multiply multiply 3 2 166 Will compute 3 times 2 167 Result of call: 6 168 169 Although the program is quite large for its functionality, most of the code is 170 for data conversion between Python and C, and for error reporting. The 171 interesting part with respect to embedding Python starts with :: 172 173 Py_Initialize(); 174 pName = PyUnicode_DecodeFSDefault(argv[1]); 175 /* Error checking of pName left out */ 176 pModule = PyImport_Import(pName); 177 178 After initializing the interpreter, the script is loaded using 179 :c:func:`PyImport_Import`. This routine needs a Python string as its argument, 180 which is constructed using the :c:func:`PyUnicode_FromString` data conversion 181 routine. :: 182 183 pFunc = PyObject_GetAttrString(pModule, argv[2]); 184 /* pFunc is a new reference */ 185 186 if (pFunc && PyCallable_Check(pFunc)) { 187 ... 188 } 189 Py_XDECREF(pFunc); 190 191 Once the script is loaded, the name we're looking for is retrieved using 192 :c:func:`PyObject_GetAttrString`. If the name exists, and the object returned is 193 callable, you can safely assume that it is a function. The program then 194 proceeds by constructing a tuple of arguments as normal. The call to the Python 195 function is then made with:: 196 197 pValue = PyObject_CallObject(pFunc, pArgs); 198 199 Upon return of the function, ``pValue`` is either ``NULL`` or it contains a 200 reference to the return value of the function. Be sure to release the reference 201 after examining the value. 202 203 204 .. _extending-with-embedding: 205 206 Extending Embedded Python 207 ========================= 208 209 Until now, the embedded Python interpreter had no access to functionality from 210 the application itself. The Python API allows this by extending the embedded 211 interpreter. That is, the embedded interpreter gets extended with routines 212 provided by the application. While it sounds complex, it is not so bad. Simply 213 forget for a while that the application starts the Python interpreter. Instead, 214 consider the application to be a set of subroutines, and write some glue code 215 that gives Python access to those routines, just like you would write a normal 216 Python extension. For example:: 217 218 static int numargs=0; 219 220 /* Return the number of arguments of the application command line */ 221 static PyObject* 222 emb_numargs(PyObject *self, PyObject *args) 223 { 224 if(!PyArg_ParseTuple(args, ":numargs")) 225 return NULL; 226 return PyLong_FromLong(numargs); 227 } 228 229 static PyMethodDef EmbMethods[] = { 230 {"numargs", emb_numargs, METH_VARARGS, 231 "Return the number of arguments received by the process."}, 232 {NULL, NULL, 0, NULL} 233 }; 234 235 static PyModuleDef EmbModule = { 236 PyModuleDef_HEAD_INIT, "emb", NULL, -1, EmbMethods, 237 NULL, NULL, NULL, NULL 238 }; 239 240 static PyObject* 241 PyInit_emb(void) 242 { 243 return PyModule_Create(&EmbModule); 244 } 245 246 Insert the above code just above the :c:func:`main` function. Also, insert the 247 following two statements before the call to :c:func:`Py_Initialize`:: 248 249 numargs = argc; 250 PyImport_AppendInittab("emb", &PyInit_emb); 251 252 These two lines initialize the ``numargs`` variable, and make the 253 :func:`emb.numargs` function accessible to the embedded Python interpreter. 254 With these extensions, the Python script can do things like 255 256 .. code-block:: python 257 258 import emb 259 print("Number of arguments", emb.numargs()) 260 261 In a real application, the methods will expose an API of the application to 262 Python. 263 264 .. TODO: threads, code examples do not really behave well if errors happen 265 (what to watch out for) 266 267 268 .. _embeddingincplusplus: 269 270 Embedding Python in C++ 271 ======================= 272 273 It is also possible to embed Python in a C++ program; precisely how this is done 274 will depend on the details of the C++ system used; in general you will need to 275 write the main program in C++, and use the C++ compiler to compile and link your 276 program. There is no need to recompile Python itself using C++. 277 278 279 .. _compiling: 280 281 Compiling and Linking under Unix-like systems 282 ============================================= 283 284 It is not necessarily trivial to find the right flags to pass to your 285 compiler (and linker) in order to embed the Python interpreter into your 286 application, particularly because Python needs to load library modules 287 implemented as C dynamic extensions (:file:`.so` files) linked against 288 it. 289 290 To find out the required compiler and linker flags, you can execute the 291 :file:`python{X.Y}-config` script which is generated as part of the 292 installation process (a :file:`python3-config` script may also be 293 available). This script has several options, of which the following will 294 be directly useful to you: 295 296 * ``pythonX.Y-config --cflags`` will give you the recommended flags when 297 compiling: 298 299 .. code-block:: shell-session 300 301 $ /opt/bin/python3.11-config --cflags 302 -I/opt/include/python3.11 -I/opt/include/python3.11 -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall 303 304 * ``pythonX.Y-config --ldflags --embed`` will give you the recommended flags 305 when linking: 306 307 .. code-block:: shell-session 308 309 $ /opt/bin/python3.11-config --ldflags --embed 310 -L/opt/lib/python3.11/config-3.11-x86_64-linux-gnu -L/opt/lib -lpython3.11 -lpthread -ldl -lutil -lm 311 312 .. note:: 313 To avoid confusion between several Python installations (and especially 314 between the system Python and your own compiled Python), it is recommended 315 that you use the absolute path to :file:`python{X.Y}-config`, as in the above 316 example. 317 318 If this procedure doesn't work for you (it is not guaranteed to work for 319 all Unix-like platforms; however, we welcome :ref:`bug reports <reporting-bugs>`) 320 you will have to read your system's documentation about dynamic linking and/or 321 examine Python's :file:`Makefile` (use :func:`sysconfig.get_makefile_filename` 322 to find its location) and compilation 323 options. In this case, the :mod:`sysconfig` module is a useful tool to 324 programmatically extract the configuration values that you will want to 325 combine together. For example: 326 327 .. code-block:: pycon 328 329 >>> import sysconfig 330 >>> sysconfig.get_config_var('LIBS') 331 '-lpthread -ldl -lutil' 332 >>> sysconfig.get_config_var('LINKFORSHARED') 333 '-Xlinker -export-dynamic' 334 335 336 .. XXX similar documentation for Windows missing 337