Hi Richard,
I can give my points on the topic.
You're very much right on the current PySide issues with Boost.Python.
The size of the bindings is quite horrible and although we did quite a
bit of optimizations to get them down, we've hit a plateau there with no
major improvement in sight along that route. Hindsight is always easy,
and you're right that we should've drawn the conclusions much earlier,
but at the time the route we took made perfect sense. However, we've
learned much from the experience and produced a lot of code which is
perfectly useful and of good quality (the API extraction and the
typesystem parts) even if we change the generator implementation.
You're also right that we should've published the project at a much
earlier stage. However, we didn't deem PySide quite ready much earlier,
and in any case publishing open source projects take often surprisingly
much time in a corporate environment. The damage for Arno Rehn's
duplicate efforts has been done, and that's something I personally
regret. On the other hand, I don't think we should've delayed the
publication due to our Boost.Python generator size issues, as we much
prefer being able to work in the open and the bindings are already quite
usable on the desktop for Qt itself, despite of their size.
The Shiboken project was started as a private experiment by some of the
OpenBossa team members and once its potential was understood, it was
developed further to see if it could solve our size problems, and once
the initial figures were acquired, we decided to move full steam ahead
with it. The skeleton bindings generated by it are currently a bit
larger than PyQt, but it's about one-third of the size Boost.Python
PySide was at the respective point, so I believe we'll be able to match
PyQt there. (Although we're not really here to compete, even if it at
times would seem like it.) And with the existing binding generator unit
tests and stuff available, we really hope to make quite rapid progress
on getting all of Qt covered.
Having followed the discussion after the project publication, I don't
have a slightest doubt that we couldn't achieve at least the same size,
and probably smaller bindings using Smoke. PySide's runtime performance
is already with Boost.Python very good, and there we don't have
comparative data from Smoke. However, my own intuition tells me that
wouldn't be much of an issue either. Obviously, this begs the question
why haven't we already switched to Smoke, and here the unfortunate
answer is that the reality has bitten us. We (Nokia) need to have
something "good enough" to develop on ready in time, and Shiboken will
fit the task even better than sufficiently. Changing the implementation
once again would incur an unacceptable delay for us. Hence, we're
pushing the Nokia-funded PySide development efforts in that direction
for the time being.
However, this being said, in a slightly longer term (maybe after half a
year or so - too early to make any commitments), we definitely see Smoke
a valid future path for PySide, as stubborn duplication of effort for no
reason is just plain dumb, after all. Having tuned the generator
architecture modular to facilitate the current transition, it should
only get easier to change engines mid-flight once again in the future.
:-) So, if the technical merits support it, the long term transition
plan could be: Boost.Python -> CPython (via Shiboken) -> Smoke.
We really, genuinely are interested in collaborating with you guys, but
obviously the real-world obligations are limiting our latitude in the
short term. Maybe we should first begin investigating merging of Arno
Rehn's and our API extractor efforts, and then later on experiment with
Smoke bindings generation?
I'd also like to note that the text above was written more or less with
my Nokia goggles on. We're really serious on making PySide governance
model totally open, so that once the project really takes off, the
project direction won't be decided according to Nokia's whims but on
technical merits only. We're not quite there yet (as the project hasn't
really taken off yet, either), but we definitely won't want to close any
doors, especially at this early stage. If you're interested in
experimenting with PySide Smoke generation already, we'd be more than
happy to create a pyside-smoke git repo and any other required
facilities for that instead of you having to fork the project or worse
yet, start a yet another one. :-) Such work would be of great help for
PySide in the near future.
Whew, that was one long mail...
Cheers,
ma.
ext Richard Dale wrote:
Thanks to the release of PySide last month I have put a lot of thought into
Python bindings, started a QtScript bindings project, and have even become
something of a Python fan! I thought I'd write a brain dump of my thoughts on
using the Smoke libraries for a Python binding.
Smoke was originally designed by Ashley Winters and the PerlQt team in 2002.
Since then it has been used for QtRuby, Qyoto C#, PHP, Common Lisp and PerlQt4
bindings. The idea is very simple and I would call it a 'moc on steriods' as
it works just like slots and signals are implemented in Qt, but for the entire
library rather than only some methods, and has features like virtual method
override callback handling, and caters for multiple inheritance, which the moc
lacks.
I am currently working on a project to implement a Smoke based binding for
QtScript with Ian Monroe of the Amarok team. There is a an existing QtScript
bindings project, but the Amarok guys had found the libs were too large, and
start up time was too slow. The Smoke library to wrap for QtCore, QtGui,
QtNetworking, QtSql, QtSvg, QtXml and QtOpenGL is only 4.3 Mb for the 613
classes it wraps. The existing QtScript bindings initialize all the classes,
and all their methods at startup which is slow, and it is about 16.3Mb for the
Qt libraries.
Since the release of PySide I have studied the Python C api and looked at the
code of several Python bindings projects, in order to get an idea of what
would be involved.
I started looking at the Boost::Python code generated for the current version
of PySide, and it certainly is very human readable, and I especially how
operator methods are defined. But I think it was designed for relatively small
projects where you might have 20-30 C++ classes, and you write the
Boost::Python code at the same time as you are writing the C++ classes, using
Python for prototyping. However, I don't think it was intended to be machine
generated, and used on the scale 600+ classes libraries like Qt. The total
size of PySide for wrapping just the Qt libraries, let alone any extra KDE
classes or whatever, is 30Mb, and that makes it unusable for small devices
like Maemo based ones. It is also very much all or nothing - it looks quite
hard to customize it to use less memory, or add more runtime dynamism. I don't
think the PySide team should have done a first release based on Boost::Python
and I have no idea why they have persisted for so long using something which
is so obviously not suitable.
Next I looked at how the Python C api works by playing with the 'Noddy'
example in the docs, and reading up on how the descriptor protocol is used
with '__getattribute__', and also how metaclasses work. Here is a summary in
Python of how Smoke would be used:
class SmokeMeta(type):
def __new__(cls, ...):
# Construct new Qt C++ instances here
def __getattribute__(self, name):
# Intercept class method calls here,
# return a callable to handle calls to
# static C++ methods in the Smoke library
class QWidget(object):
__metaclass__ = SmokeMeta
def __getattribute__(self, name):
# Intercept instance method calls here,
# return a callable to handle calls to
# C++ methods in the Smoke library
So an actual implementation would be the same as the code above, but written
in C. I think the Python C api would be a good fit to use with Smoke.
After understanding the C api better, I studied the Gnome pygobject project,
which is what I would call a 'dynamic binding' like Smoke, which looks up
method calls and classes at runtime, instead of them being hard wired into the
bindings library at code generation time. It uses both GObject itself, and
gobject-introspection libraries at runtime. I think it is very impressive and
it uses custom versions of the tp_getattro() C function on the Python class
structs to intercept calls to __getattribute__ just like I thought could be
done with Smoke. The code is LGPL'd and so it could either be used directly,
or at least you could get ideas from it for a dynamic Python binding.
I had read about an experimental branch in PySide called 'Shiboken' that uses
pretty much the vanilla Python C api, and so I checked it out of gitorious and
had a look. In gitorious it didn't have any Qt classes wrapped, and didn't
have any Qt marshalling either, and so it wasn't possible to tell how large it
might be. I spoke with 'hugpol' on IRC and he told me that they had a version
that wrapped QtCore on an internal git server, and it was about 2.2 Mb. In
PyQt, QtCore is about 2Mb and in the Boost::Python version of PySide is it
4.4Mb. I think the Smoke version would be less than 1Mb. So I think with
enough work, it might be possible to produce a Shiboken version of PySide that
was about the same size or slightly bigger than PyQt.
How much work is 'enough work' though to match PyQt? I studied the PyQt code
last and it really is very impressive indeed. It looks exactly as though a
Python expert has worked fulltime for over 10 years on it with help from the
community that uses it. It starts up fast, as it loads methods lazily only
when they are needed. In fact it uses about the same mill to start up as they
Smoke based QtScript bindings do, and about half the mill of QtRuby does (I
think that is because Ruby is slow, rather than Smoke being slow). I need to
study it more to see what it does, but there really doesn't seem to be much to
fault at all.
I think the bindings generators based on the QtJambi one that PySide, Smoke
and QtScript all use are really good, and they do match the SIP code
generation approach parsing '.sip' files, instead of parsing the C++ headers
directly and adding XML metadata from config files. However, because we didn't
know about the PySide project Arno Rehn developed a bindings generator for
Smoke in a Google Summer of Code project this year, which is based on the
QtScript one just like the PySide team did. Maybe their code bases can be
merged.
The most important advantage of dynamic language bindings is that they are
language independent and not Python only, and are also smaller than
conventional approaches. I think a dynamic Python binding like the gobject-
introspection based pygobject, or a smoke based one is sufficiently interesting
technically and different enough from PyQt to be worthwhile. On the other hand,
although I think Shiboken can be made to work, at best it would be much the
same as PyQt and even that would take a pretty heroic effort as far as I can
see.
To me the only justification I can see for implementing another Python binding
(apart from the GPL vs LGPL license issue which I personally don't care about
much), would be to implement a Maemo based development environment that
combined Python, Ruby and QtScript using common bindings libs with a lower
memory footprint than other approaches. Instead of just doing a "Let's kill
PyQt on all possible platforms", it could be "Let's develop a great multi-
language RAD environment for Maemo". For instance, I can't see the KDE project
switching from PyQt/PyKDE to PySide anytime soon, no matter what approach
PySide takes. And pretending that it would be easy in that area or anywhere
PyQt is already entrenched isn't being realistic IMHO.
-- Richard
_______________________________________________
PySide mailing list
[email protected]
http://lists.openbossa.org/listinfo/pyside
_______________________________________________
PySide mailing list
[email protected]
http://lists.openbossa.org/listinfo/pyside
