On Jan 13, 2007, at 1:05 PM, Pablo Martin wrote: > I think many people is aware that current swig bindings generation is > creating way too big modules, that take loads of ram to build and in > some systems arent even linkable due to too many symbols. The > problem is > already very visible from python and a lot of people is complaining > (either that they cant build the module or it takes too long). > Im thiking in a solution that would involve generating several modules > instead of only one, so instead of cspace module, we could have > csgeom, > csutil, cstool, engine... I'm still not sure what modules would be > necessary.
Some notes: - It may not necessarily be true that breaking the bindings apart into smaller modules would improve build time (in fact, it might lengthen it). A couple years ago, when considering the same issue, I ran CS/scripts/perl5/cswigpl5.inc through the C++ preprocessor to see how much actual code was being fed to the compiler. At the time, the result was >12MB. So, that's 12MB of source code the compiler has to handle. It's a lot of code. 12MB is 12MB whether contained in one file or twenty. The Swig-generated files have grown a good deal since my earlier testing. - Breaking the bindings apart into smaller modules, although not necessarily improving overall build time, might improve the "perceived build time". Watching twelve files build at twenty seconds each is perhaps perceptually more acceptable than waiting four minutes for a single file to build. - Breaking the bindings apart into smaller modules will almost certainly resolve the linking problems on various platforms. For this reason alone, such a change should be considered. - As a rough heuristic, I might suggest creating one binding module per top-level CS header directory. For instance, csutil, csgeom, iutil, iengine, ivaria, etc. This won't be a perfectly balanced break- down, however, it is important to factor the "ifoo" stuff into the equation since the SCF interfaces account for a significant portion of the bulk of the Swig-generated code. > I'll talk about how it would work from python so old ways would > still be > supported, id need to know how similar could be done for java and perl > at least. Ideally some talk could be done with maintainers of other > scripting interfaces, regrettably i dont know who you are :) > Thing is python allows to have a module in a folder, so the following > structure could be used: > cspace/__init__.py > cspace/csgeom.py > cspace/_csgeom.so or .pyd > cspace/csutil.py > cspace/_csutil.so or .pyd > Then with the following code in __init__.py all would still work the > same as now: > from csgeom import * > from csutil import * > This way people could still do "from cspace import *". It is important to do this in a way which does not penalize end- users. The breaking-apart of the bindings should be an "implementation detail" invisible (or at least not obvious) to clients. The above approach should work admirably for Python. Perl also can handle this sort of thing easily with a little hand-written (or perhaps auto-generated) Perl glue code which brings all the modules together. Lua, too, I believe can be made to manage this in much the same way with a small amount of glue code. Java might require some extra effort, possibly in the form of 'importing' the various sub-packages into the top-level 'org.crystalspace3d' package. > So, this said, the other problem is build system would have to be > changed, as having the modules in cs main folder would no longer help, > they really would have to end in the commented cspace folder. I'm > totally not sure about what the best way to do this would be, but i > thought the best would be to build this in some out/bindings/language/ > folder (instead of main folder) so they can be frozen inside their > language dirs under scripts/language/cspace/ Agreed. In fact, the csjava/Jamfile already does something very similar. It creates a "bindings/java" directory underneath its "out/ blah/foo/plugins/csjava" folder into which it deposits all of the "final" build targets (the ones which will be installed). It would be reasonable to generalize this so that final build targets for all scripting languages are instead deposited into "out/bindings/language/". > Then also there is a detail that could be attacked at the same time, > right now cspython links in the _cspace swig generated library, also > cel's blcelc is linking in _cspace and _blcelc libraries. This is > convenient for distribution but i'm not sure about whether its best > approach, in any case i'd like to know if other languages are doing > similar or they use the swig generated dlls. (in python the dlls are > only used from pure python apps, if using embedded code the versions > linked in are used instead). This is an approach which needs to be applied to the Perl bindings and to the Lua bindings (whenever Lua is added ultimately to the project). Presently, the Perl Jamfile plays around with the low-level object file list when building the Perl CS plugin and stand-alone Perl module. This is ugly. The "library" approach is much cleaner and superior in my opinion. -- ES ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Crystal-main mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/crystal-main Unsubscribe: mailto:[EMAIL PROTECTED]
