2010/12/23 Jeff Johnson <n3...@mac.com>:
>
> On Dec 23, 2010, at 4:24 PM, Per Øyvind Karlsen wrote:
>
>> To make it possible to consider enabling embedding of interpreters,
>> I'd say it's crucial to not tie them in as dependencies to break rpm
>> completely
>> if missing (ie. perl changing path to libperl.so for each micro
>> release) or pulling
>> in large dependencies or anything...
>>
>> Here's a patch that adds dlopen() support for python embedding, autofoo stuff
>> already broken for both perlembed & pythonembed already, trying to fix it and
>> avoiding linking when doing dlopen() gave me headaches and for to
>> figure out later..
>>
>
> No patch, but I can guess ...
though, here goes.. ;)
I think the behaviour when no python can be loaded isn't the entirely
appropriate, also 'rpmlib(BuiltinPythonScripts)' should probably
rather be determined at run time as well...
>
> Let's deal with perl for an example, largely because
> an embedded perl interpreter can be instantiated
> multiple times, with multiple identical loads,
> which is (perhaps) more general than other widdle
> interpreters like python and ruby.
>
> When rpmperlNew is called, this script is fed to the interpreter:
>
> static const char * rpmperlInitStringIO = "\
> use strict;\n\
> use IO::String;\n\
> our $io = IO::String->new;\n\
> select $io;\n\
> ";
>
> This -- of course -- breaks almost always becuse
> the IO::String module isn't Artistic enuf any more.
>
> When the IO::String module isn't present, then
> the necessary mapping of stdout to a buffer doesn't happen.
>
> Without buffer content, all %{perl OPTS ARGS: BODY} macro expansions
> break.
>
> Whether IO:String a "hard" or "soft" or "I'm an Artist!" dependency
> I'l leave to the bikeshed. But there's certainly ways within perl
> to detect a failed module load, and print out errors, and try fallbacks
> without writing dlopen() directly.
>
> Module loading using dlopen(3) is built into perl, I see no reason to try to
> out-guess the Artists in C code in RPM.
Well, currently to get it building I have to pass include path to
CPPFLAGS & library path + rpath to LDFLAGS.
With libperl.so for me being located in
/usr/lib/perl5/5.12.2/x86_64-linux-thread-multi/CORE/libperl.so, this
obviously blows...
>
> But the startup script needs to be moved into a template, very not hard
> mechanically, but how that perl interpreter initialization WILL need
> some serious thought from some perl Artist willing to think through
> perl embedding, and to suggest something reasonable.
>
> All I've done is the bare minimum necessary to get perl embedded,
> sketching in structural elements that WILL be needed for "real world"
> usage. E.g. I could have quite easily hard-wired a stdout file descriptor
> into a buffer using C code; I chose IO::String instead.
>
> Make sense?
Not sure if I get things properly, haven't looked into rpmio/rpmperl.c
much at all, but it does at least link against libperl.so, intended or
not..?
>
> Meanwhile I'm really really hoping to be able to piggy back dlopen(3)
> using already implemented functionality in one of these widdle interpreters,
> rather than have to go through the tedium of Yet Another dlopen(3) Wrapping.
>
> But without some clear interest in SOME embedding (JavaScript was the chosen
> embedding, lua is the historical embedding, the other embeddings are there
> largely
> to pre-empt vacuous discussions about the relative merits of the widdle
> interpreters:
>
> You like <yadda_yadda_interpreter>? Here it is, implemented in RPM, go
> figger and use!
I'm all ready to enable these, just trying to avoid pulling in a lot
of dependencies and features without anyone really using them..
We'll get there eventually... :)
--
Regards,
Per Øyvind
--- rpmio/rpmpython.c 2010-11-21 06:09:34.728809000 +0100
+++ /home/peroyvind/RPM/rpm5/BUILD/rpm-5.3.7/rpmio/rpmpython.c 2010-12-23 22:12:50.106785798 +0100
@@ -10,6 +10,66 @@
#if defined(WITH_PYTHONEMBED)
#include <Python.h>
#include <cStringIO.h>
+
+#if defined(HAVE_DLFCN_H)
+#include <dlfcn.h>
+#include <rpmlog.h>
+
+extern PyAPI_FUNC(void) Py_SetProgramName(char *);
+static PyAPI_FUNC(void) (*Py_SetProgramName_p) (char *);
+extern PyAPI_FUNC(void) Py_Initialize(void);
+static PyAPI_FUNC(void) (*Py_Initialize_p) (void);
+extern PyAPI_FUNC(void) Py_Finalize(void);
+static PyAPI_FUNC(void) (*Py_Finalize_p) (void);
+extern PyAPI_FUNC(int) Py_IsInitialized(void);
+static PyAPI_FUNC(int) (*Py_IsInitialized_p) (void);
+
+extern PyAPI_FUNC(PyObject *) PyRun_StringFlags(const char *, int, PyObject *,
+ PyObject *, PyCompilerFlags *);
+static PyAPI_FUNC(PyObject *) (*PyRun_StringFlags_p) (const char *, int, PyObject *,
+ PyObject *, PyCompilerFlags *);
+extern PyAPI_FUNC(int) PyRun_AnyFileExFlags(FILE *, const char *, int, PyCompilerFlags *);
+static PyAPI_FUNC(int) (*PyRun_AnyFileExFlags_p) (FILE *, const char *, int, PyCompilerFlags *);
+
+extern PyAPI_FUNC(void) PySys_SetArgv(int, char **);
+static PyAPI_FUNC(void) (*PySys_SetArgv_p) (int, char **);
+extern PyAPI_FUNC(void *) PyCObject_Import(char *module_name, char *cobject_name);
+static PyAPI_FUNC(void *) (*PyCObject_Import_p) (char *module_name, char *cobject_name);
+
+extern PyAPI_FUNC(PyObject *) PyImport_AddModule(const char *name);
+static PyAPI_FUNC(PyObject *) (*PyImport_AddModule_p) (const char *name);
+extern PyAPI_FUNC(PyObject *) PyModule_GetDict(PyObject *);
+static PyAPI_FUNC(PyObject *) (*PyModule_GetDict_p) (PyObject *);
+extern PyAPI_FUNC(PyObject *) PySys_GetObject(char *);
+static PyAPI_FUNC(PyObject *) (*PySys_GetObject_p) (char *);
+
+extern PyAPI_FUNC(char *) PyString_AsString(PyObject *);
+static PyAPI_FUNC(char *) (*PyString_AsString_p) (PyObject *);
+
+extern PyAPI_FUNC(void) PyErr_Print(void);
+static PyAPI_FUNC(void) (*PyErr_Print_p) (void);
+extern PyAPI_FUNC(void) PyErr_Clear(void);
+static PyAPI_FUNC(void) (*PyErr_Clear_p) (void);
+extern PyAPI_FUNC(int) Py_FlushLine(void);
+static PyAPI_FUNC(int) (*Py_FlushLine_p) (void);
+#else
+#define Py_SetProgramName_p Py_SetProgramName
+#define Py_Initialize_p Py_Initialize
+#define Py_Finalize_p Py_Finalize
+#define Py_IsInitialized_p Py_IsInitialized
+#define PyRun_StringFlags_p PyRun_StringFlags
+#define PyRun_AnyFileExFlags_p PyRun_AnyFileExFlags
+#define PySys_SetArgv_p PySys_SetArgv
+#define PyCObject_Import_p PyCObject_Import
+#define PyImport_AddModule_p PyImport_AddModule
+#define PyModule_GetDict_p PyModule_GetDict
+#define PySys_GetObject_p PySys_GetObject
+#define PyString_AsString_p PyString_AsString
+#define PyErr_Print_p PyErr_Print
+#define PyErr_Clear_p PyErr_Clear
+#define Py_FlushLine_p Py_FlushLine
+#endif
+
#endif
#include "debug.h"
@@ -27,7 +87,7 @@ static void rpmpythonFini(void * _python
rpmpython python = _python;
#if defined(WITH_PYTHONEMBED)
- Py_Finalize();
+ Py_Finalize_p();
#endif
python->I = NULL;
}
@@ -63,8 +123,46 @@ static rpmpython rpmpythonI(void)
/*...@globals _rpmpythonI @*/
/*...@modifies _rpmpythonI @*/
{
- if (_rpmpythonI == NULL)
+ if (_rpmpythonI == NULL) {
+#if defined(HAVE_DLFCN_H)
+ void *h;
+ char libpython[64];
+
+ snprintf(libpython, sizeof(libpython), "libpython%d.%d.so", PY_MAJOR_VERSION, PY_MINOR_VERSION);
+
+ h = dlopen (libpython, RTLD_LAZY|RTLD_GLOBAL);
+ if (!h)
+ {
+ rpmlog(RPMLOG_WARNING, D_("Unable to open \"%s\" (%s), "
+ "embedded python will not be available\n"),
+ libpython, dlerror());
+ }
+ if(!((Py_SetProgramName_p = dlsym(h, "Py_SetProgramName"))
+ && (Py_Initialize_p = dlsym(h, "Py_Initialize"))
+ && (Py_Finalize_p = dlsym(h, "Py_Finalize"))
+ && (Py_IsInitialized_p = dlsym(h, "Py_IsInitialized"))
+ && (PyRun_StringFlags_p = dlsym(h, "PyRun_StringFlags"))
+ && (PyRun_AnyFileExFlags_p = dlsym(h, "PyRun_AnyFileExFlags"))
+ && (PySys_SetArgv_p = dlsym(h, "PySys_SetArgv"))
+ && (PyCObject_Import_p = dlsym(h, "PyCObject_Import"))
+ && (PyImport_AddModule_p = dlsym(h, "PyImport_AddModule"))
+ && (PyModule_GetDict_p = dlsym(h, "PyModule_GetDict"))
+ && (PySys_GetObject_p = dlsym(h, "PySys_GetObject"))
+ && (PyString_AsString_p = dlsym(h, "PyString_AsString"))
+ && (Py_FlushLine_p = dlsym(h, "Py_FlushLine"))
+ && (PyErr_Print_p = dlsym(h, "PyErr_Print"))
+ && (PyErr_Clear_p = dlsym(h, "PyErr_Clear"))
+ )) {
+ rpmlog(RPMLOG_WARNING, D_("Opened library \"%s\" is incompatible (%s), "
+ "embedded python will not be available\n"),
+ libpython, dlerror());
+ if (dlclose (h))
+ rpmlog(RPMLOG_WARNING, "Error closing library \"%s\": %s", libpython,
+ dlerror());
+ } else
+#endif
_rpmpythonI = rpmpythonNew(NULL, 0);
+ }
return _rpmpythonI;
}
@@ -83,17 +181,19 @@ fprintf(stderr, "==> %s(%p, %d) python %
if (av == NULL) av = _av;
#if defined(WITH_PYTHONEMBED)
- if (!Py_IsInitialized()) {
- Py_SetProgramName((char *)_av[0]);
- Py_Initialize();
- }
- if (PycStringIO == NULL)
- PycStringIO = PyCObject_Import("cStringIO", "cStringIO_CAPI");
+ if(_rpmpythonI != NULL) {
+ if (!Py_IsInitialized_p()) {
+ Py_SetProgramName_p((char *)_av[0]);
+ Py_Initialize_p();
+ }
+ if (PycStringIO == NULL)
+ PycStringIO = PyCObject_Import_p("cStringIO", "cStringIO_CAPI");
- if (initialize) {
- int ac = argvCount((ARGV_t)av);
- (void) PySys_SetArgv(ac, (char **)av);
- (void) rpmpythonRun(python, rpmpythonInitStringIO, NULL);
+ if (initialize) {
+ int ac = argvCount((ARGV_t)av);
+ (void) PySys_SetArgv_p(ac, (char **)av);
+ (void) rpmpythonRun(python, rpmpythonInitStringIO, NULL);
+ }
}
#endif
@@ -110,7 +210,7 @@ fprintf(stderr, "==> %s(%p,%s)\n", __FUN
if (python == NULL) python = rpmpythonI();
- if (fn != NULL) {
+ if (python != NULL && fn != NULL) {
#if defined(WITH_PYTHONEMBED)
const char * pyfn = ((fn == NULL || !strcmp(fn, "-")) ? "<stdin>" : fn);
FILE * pyfp = (!strcmp(pyfn, "<stdin>") ? stdin : fopen(fn, "rb"));
@@ -118,7 +218,7 @@ fprintf(stderr, "==> %s(%p,%s)\n", __FUN
PyCompilerFlags cf = { .cf_flags = 0 };
if (pyfp != NULL) {
- PyRun_AnyFileExFlags(pyfp, pyfn, closeit, &cf);
+ PyRun_AnyFileExFlags_p(pyfp, pyfn, closeit, &cf);
rc = RPMRC_OK;
}
#endif
@@ -159,28 +259,28 @@ fprintf(stderr, "==> %s(%p,%s,%p)\n", __
if (python == NULL) python = rpmpythonI();
- if (str != NULL) {
+ if (python != NULL && str != NULL) {
const char * val = rpmpythonSlurp(str);
#if defined(WITH_PYTHONEMBED)
PyCompilerFlags cf = { .cf_flags = 0 };
- PyObject * m = PyImport_AddModule("__main__");
- PyObject * d = (m ? PyModule_GetDict(m) : NULL);
- PyObject * v = (m ? PyRun_StringFlags(val, Py_file_input, d, d, &cf) : NULL);
+ PyObject * m = PyImport_AddModule_p("__main__");
+ PyObject * d = (m ? PyModule_GetDict_p(m) : NULL);
+ PyObject * v = (m ? PyRun_StringFlags_p(val, Py_file_input, d, d, &cf) : NULL);
if (v == NULL) {
- PyErr_Print();
+ PyErr_Print_p();
} else {
if (resultp != NULL) {
- PyObject * sys_stdout = PySys_GetObject("stdout");
+ PyObject * sys_stdout = PySys_GetObject_p("stdout");
if (sys_stdout != NULL && PycStringIO_OutputCheck(sys_stdout)) {
PyObject * o = (*PycStringIO->cgetvalue)(sys_stdout);
- *resultp = (PyString_Check(o) ? PyString_AsString(o) : "");
+ *resultp = (PyString_Check(o) ? PyString_AsString_p(o) : "");
} else
*resultp = "";
}
Py_DECREF(v);
- if (Py_FlushLine())
- PyErr_Clear();
+ if (Py_FlushLine_p())
+ PyErr_Clear_p();
rc = RPMRC_OK;
}
#endif
