Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Hi, too long a silence around this important topic for my taste - so let me tease you folks with this here new kid on the block: http://aruiz.synaptia.net/siliconisland/2010/03/buildj-build-configuration-for-the-mases.html ;) Cheers, -- Thorsten pgp1VOCzhBbNV.pgp Description: PGP signature
Re: [tools-dev] New Build System Requirements
Mathias Bauer wrote: > I just wanted to point out that you seem to underestimate the value > of a less diverse build system. Most probably because you don't > have to maintain it. ;-) > Quite the contrary. I have to personally setup all the win32 build prerequisites on each and every box or vm I want to build OOo with - and it's a royal PITA. That's why I maintain that cygwin is not the problem. But as you noted elsewhere, we'll likely need to keep it no matter what (for the while), so I guess this point is settled. ;) Cheers, -- Thorsten pgpWOOBB9eg2h.pgp Description: PGP signature
Re: [tools-dev] New Build System Requirements
Hi Mathias, you wrote: > This doesn't rule out CMake as a serious competitor, but it shows that > its main benefit (being able to build with VS on Windows) comes at a price. > and > Lately I gained some experience with dependencies in OOo and reducing > them. IMHO it's easier to understand how the different parts of OOo work > together the more the complexity of the dependencies is reduced. This > helps new developers to understand. And even the more experienced > developers benefit from the better maintainability. > > So I think that aiming for leaner dependencies (not only) in the long > run is a worthwile goal and I wouldn't consider any change in the build > system that wouldn't go for it. Whether it's the number one or the > number two priority doesn't matter for me: IMHO it's a must have > priority. > I find those arguments a bit contrived - the first one is a truism, and the second one seems to mix tight implementation dependencies with well-defined, sometimes posix-standardized, black-boxed external tool dependencies. This is nothing to spend two seconds' thoughts on, for all the platforms out there with a decent package and dependency management system, simply because you trade an automagically-resolved dependency for reuse of existing functionality - something I value quite high in a project where developer resources are amongst the scarcest. The story on win32 (and other, not-so-standard platforms I believe you alluded to) is totally different, of course, but you won't fix that by prohibiting sed & awk - fixing the root cause here means performing cross-compilation for those platforms. (you need to build/debug natively on those platforms? sure, but binning e.g. cygwin from the impressive list of ~14 build prerequisites http://wiki.services.openoffice.org/wiki/Development/OpenOffice.org_Building_Guide/Building_on_Windows#software_requirements means getting rid of the _least problematic_ one - trivial to install, easy to update, free, built-in package management etc. - shouldn't we rather eliminate the real deal-breakers there first?) So again, the build system dependencies for me are one of the least important criteria, and when it comes to python, we have that dependency *anyway*, on *any* developer machine, due to mercurial. Just my 2c. :) -- Thorsten pgp8q5cZttuwv.pgp Description: PGP signature
Re: [tools-dev] New Build System Requirements
bjoern michaelsen - Sun Microsystems - Hamburg Germany wrote: > I started a new wikipage here: > http://wiki.services.openoffice.org/wiki/Build_Environment_Effort/New_Build_System_Requirements > > collecting the major requirements for a new build system and what needs > to be done to implement these requirements with GNU make and CMake > (those two currently seem to be the only serious contestants). If you > find additional requirements, feel free to add them to the page. > Nice page, thanks Björn! Though I think the section about debuggability appears a bit, err, skewed - I would think none of the contestants are "easily debuggable" to the common man, when it comes to the internals. ;) Having lean dependencies is also nice, but more of a runner-up criteria for me - it would prolly only make a little dent in the dep list OOo has anyway... Cheers, -- Thorsten pgpHu4KL4w27J.pgp Description: PGP signature
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
bjoern michaelsen - Sun Microsystems - Hamburg Germany wrote: > > But the actual information contained in the above lines is actually > > this: > > > > rscdep source files: tools/bootstrp/* > > rscdep link libs: tl > Quoting a band from Hamburg: "Jein" (=Yes and No). > Of course, there is a lot of superfluous syntax in the current > description, but you are dropping some information too: > - Where is the information which naming convention the tl-lib is > following? > That information is implicitely already contained in the name 'tl'. As you notice further down, the gmakegen script has a static map for those lib names. > - What about the files in tools/bootstrp, that are not generating > object files that should be liked into rscdep (sspretty.cxx for > example)? > Yeah, badly chosen example. The absolute majority of OOo code _is_ organized in a fashion that allotting sources files to libs works on a per-directory basis. At any rate, the generator script can do both. Regarding your point of _where_ the globbing should take place: currently it's run from the generated makefile, so your thoughts on when to re-run the generator don't apply. ;) > > [makefile generator input] > > > > What do you think? > Looking a lot better that the GNU make input files, that are currently > used in the GNU make prototype for sure. However, I wonder how the > deeply nested lists and maps of python are looking to the > (python)-untrained eye. Subjecting an unsuspecting colleague to the > syntax will tell, I guess. > Thanks. :) So, unsuspecting colleagues with the python-untrained eyes: what would be your take on this? > Another thing to be considered with this is the introduction of python > as a dependency that early in the toolchain. AFAIK currently we have > these dependencies for the build system: > > [snip] > Generally a worthy goal to reduce dependencies; though of little practical relevance here IMHO. FWICT python has a broader supported base than java, and I think we're past this phase of having _that_ being an optional build requirement, do we? ;) Additionally, hg needs python anyway. > Given that the syntax of the "build task description language" should > be simple (because, if one needs it to be complex, one is likely Doing > It Wrong(tm)) I wonder, if something that can be processed by the > POSIX-shell or the C-Preprocessor would not be possible too(*). > Actually, I am rather confident that would be possible. > Likely. There are two reasons I decided against using those: * ugliness (I have yet to see the cpp macro program that does not hurt my eyes) * too much expressiveness (you'll get full turing completeness easily, when you process stuff with the shell. I explicitely wanted to keep things declarative, and prohibit those small local if-then-else workarounds) > Still, there remains the problem with makefile generation (when? what > are the dependencies? does the generator glob for files or does the GNU > make build system do this?), which I guess we should discuss first, > because "All problems in computer science can be solved by another > level of indirection except for the problem of too many layers of > indirection." > Sure. :) So, for my current draft, the gnu make system does it. For a future visual studio project generator, I guess the generator script would have to do that. Both approaches are valid in their respective area, I think. Whether the former might pose performance issues remains to be seen, but would have an easy fix (just fix the generator. having gnu makefiles as the authoritative input would make that much harder). Cheers, -- Thorsten pgpWb5TN0x6CQ.pgp Description: PGP signature
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Jussi Pakkanen wrote: > > I think it's cleaner, and there's definitely not much (should I say > > any?) redundancy left. Additionally, one can enhance the script to > > generate makefiles for pretty much every make tool of this world, > > including eclipse/netbeans/visual studio project files. > > I would like to point out that what you are doing is generating your > own language and a build tool/generator based on that. > Yes. I'm piggy-backing on Björn's own rewrite, adding the (IMHO) crucial features that a new build system *should* have, once we're going for this endeavour. > There's nothing wrong with it as such, but this is reinventing the > wheel again (just like Google's GYP). Instead of custom dmake/ > build.pl you would have custom gnumakegen/gnumake_or_something. > What is the benefit you get from this instead of using something > like CMake that already has a mature implementation of this > functionality? Even if CMake eventually turns out to be too slow, > would it not make more sense to write your own custom CMake back > end rather than the configuration/generation front end? > I guess it's now my turn to ask for sample code here. ;) I've no strong opinion on cmake, except for the fact that it sucks at cross-building; the input syntax is ~ok, though not really good on enforcing structure; no idea how much effort there is writing a custom backend vs. having a dedicated approach in the first place (i.e. how much code could we share, e.g. from the eclipse/visual studio output targets in cmake, vs. having it all custom anyways?) > For further information here is a Google Tech Talk about CMake and all > related things (testing, code coverity, packaging, etc, etc) by one of > the creators. If the build tool decision is not yet final, it is worth > watching. > > http://www.youtube.com/watch?v=8Ut9o4OdSC0 > Mostly marketing. I'm not too interested who else is using it, whether it has reached a tipping point etc. etc., but rather how an actual cmake solution to the requirements mentioned before may look like (scaffolding, drafting, etc. totally acceptable - just need the gist of it). :) Cheers, -- Thorsten pgpBJfu60iKAW.pgp Description: PGP signature
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Stephan Bergmann wrote: > >So I hacked up some python to parse a sort of declarative input > >file, and generate gnu makefiles with Björn's eval framework out of > >that. > > At what stage of the development/build process would that script be called? > I'd personally use it to dynamically generate the makefiles (i.e. from configure, or triggered from a smallish global makefile). -- Thorsten pgprzdC1BzTxq.pgp Description: PGP signature
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Mathias Bauer wrote:
> > Understood. I pretty much agree with the goals, assume you did your
> > due diligence on verifying that gnu make does not trip over on the
> > full input set (that was my point of cautioning you, with the bjam
> > story), and am now trying to explore ideas on how to make the actual
> > makefiles appealing - the current state is not convincing, just
> > plainly because they're not substantially different from the ideal
> > dmake makefile in the existing system - and with their redundancies,
> > will suffer the same bitrot as the old system.
> Sounds good. :-)
> Any improvement would be welcome.
>
Ok, took longer than expected to find enough time for this, but here
we go:
After my initial performance worries had been sorted out, there are
basically two points left I'd require from an all-new build system:
- declarative syntax, as little duplication as possible
- ability to do cross-platform builds
For the former, look at this gnu makefile snippet (taken from
gnu_make cws):
[...]
$(eval $(call gb_Executable_Executable,rscdep))
$(eval $(call gb_Executable_set_linked_libs,rscdep,\
$(call gb_OOoLibrary_get_linknames,\
tl \
) \
))
$(eval $(call gb_Executable_add_exception_objects,rscdep,\
tools/bootstrp/appdef \
tools/bootstrp/command \
[etc.]
Call me purist, but there's a lot of noise in these lines, that
convey little, if any information. Plus, there's redundancy, in the
form of name prefixes, gnu make plumbing to call, etc., and also the
need to exhaustively list all object files (by listing paths names
of cxx files, stripped from the .cxx extension).
But the actual information contained in the above lines is actually
this:
rscdep source files: tools/bootstrp/*
rscdep link libs: tl
So I hacked up some python to parse a sort of declarative input
file, and generate gnu makefiles with Björn's eval framework out of
that. Corresponding to the above lines, this would be the input
snippet:
targets: [
{
executable: {
name: 'rscdep'
arch: host
}
sources: {
glob: '*.cxx'
paths: ['tools/bootstrp']
}
linking: ['tl']
} ]
I think it's cleaner, and there's definitely not much (should I say
any?) redundancy left. Additionally, one can enhance the script to
generate makefiles for pretty much every make tool of this world,
including eclipse/netbeans/visual studio project files.
The "arch: host" line is not yet fleshed out in the script, nor in
the gnu_make cws, but is indented for doing cross-builds: just
annotate every target with one out of "arch: [host:target:both]" to
later advise the build system which output architecture to compile
for.
Prototypical script, input files & patch against CWS gnu_make
attached (to handle input file globs). For testing, call it like
this: ./gmakegen.py tools_rscdep.in
What do you think?
-- Thorsten
gnumakegen.tgz
Description: application/compressed-tar
pgpJpcsblWlGK.pgp
Description: PGP signature
Re: [tools-dev] requirements for a Windows build bot and cws box
Martin Hollmichel wrote: > >>Pavel, Thorsten, you may want to comment or approve these > >>spendings for the developer budget ? > > > >+1 for the Windows box/server. We desperately need communi Windows > >machine. I;d go with terminal services directly from the > >beginning. > > > >-1 for the regular yearly payment for the hosting services. Can we > >do it better/cheeper? Can anyone offer/sponsor this service for > >us? > please keep in mind that this yearly payment includes 100MBit and > unlimitted traffic, that also will allow the download of builded > cws, I think it's worth the price, but I'm open for better or > cheaper offers, how long we want to wait for those offers ? > +1 for the full package; if someone finds a cheaper offer that's of course fine by me. Win32 builds are a pain point for me, and historically the dev budget has not been fully spent, so no point in haggling too much here. What about reviewing the hosting options again next year? Cheers, -- Thorsten pgpIBr6yMwHjZ.pgp Description: PGP signature
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Mathias Bauer wrote: > So if you could explain how bjam (or any other make system that > someone wants to suggest here) solves our problems or why the > problems that require bjam to be resolved are even bigger than > those we try to fix, we might be able to get somewhere. > I did that, if you could check my initial posting. Bjoern said he profiled gnu make with large enough synthetic rule sets, point is settled for the while, I'd say. > You can find most of our findings, discussions etc. in the wiki (I have > posted the link several times) or in Björn's and my blogs. > Nothing we're discussing here except Bjoern's initial blog from Friday, sorry Mathias. This is not to criticize anybody, just pointing out that, when such things are announced after-the-fact, many of the stupid ideas & questions that have probably been resolved internally before, will crop up in the public discussion again. > Björn also will post some more data about GNU make investigations > soon. They never were planned to be "internal findings", but > posting them without prior explanation wouldn't have made sense > and even now they don't add anything to the discussion we wanted > to start. The topic is not "can GNU make do that", but "do we have > the right goals" and "GNU make can reach the goals, which other > tools can do it also or perhaps even better". > Understood. I pretty much agree with the goals, assume you did your due diligence on verifying that gnu make does not trip over on the full input set (that was my point of cautioning you, with the bjam story), and am now trying to explore ideas on how to make the actual makefiles appealing - the current state is not convincing, just plainly because they're not substantially different from the ideal dmake makefile in the existing system - and with their redundancies, will suffer the same bitrot as the old system. Cheers, -- Thorsten pgpzK3a9qWqvu.pgp Description: PGP signature
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
bjoern michaelsen - Sun Microsystems - Hamburg Germany wrote: > > # loads cxxflags, weird sw include setup etc. > > include sw_defaults > Here you will either need to use dirty tricks or you will clash in a > global namespace, because you would like these defaults to be applied > to msword and others in sw, but not the others. > That's exactly why I wonder if we should have a level of abstraction above naked gnu makefiles - i.e. a DSL that gets translated into makefiles. > > And in fact inescapable, ~every implementation will leak under its > > abstractions. So I guess you're rather arguing for having only _one_ > > implementation - which is not obtainable in a cross-platform, > > opensource project. ;) > No, I am all for (according to the Zen of Python): > "There should be one-- and preferably only one --obvious way to do it." > Zen is nice, Python even more so, but let's face reality - the world out here is not homogeneous (and not a single version for every used tool) at all. Cheers, -- Thorsten pgppjb4Is4gry.pgp Description: PGP signature
[tools-dev] Re: Building OpenOffice.org with GNU make
Hi Mathias, most of the points you've raised I already replied to in my followup to Bjoern (including my ideal msword lib makefile) - Mathias Bauer wrote: > build.pl uses module dependencies, not target dependencies, so it has an > inherent susceptibility to bottlenecks. Basically all of our c++ sources > could be built in parallel (as they don't depend on each other), but > with build.pl we always have to wait until header files are "delivered" > or created. And because the dependency granularity level is the "module" > (not a real target like e.g. a library), we can't use as much > parallelization as possible. This becomes even more painful if you don't > build the whole office, but only some parts of it, e.g. in a split build > or if you rebuild several "modules". > Ah interesting - so we're then moving away from the solver concept? > No, really, there's nothing nailed until now. If you or anybody else > knew a better way and(!) offered help and cooperation, there's nothing > that would hold us back from doing it differently. > I find this "and(!)" slightly worrying - not that I would not lend a helping hand; but why are you refusing advise from people just because they may not have time helping you with coding? > > That's why he went for bjam ... > > Can you explain if bjam is able to fulfill the requirements Björn and I > have mentioned and what else it can do better than GNU make? Or can you > at least explain why you perhaps prefer it? > I didn't say I'd prefer it. I was just pointing to that project, and the experiences. Maybe it would help the discussion if more of the internal findings you mentioned would be public - come to think of it, maybe it would have been nice to have the whole discussion that led to this prototype public, in the first place ... ;) Cheers, -- Thorsten pgpKhF392crUR.pgp Description: PGP signature
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Björn Michaelsen wrote: > > That's not what I meant. The reason to also hand over a specific > > dmake option for multi-processor builds is that sometimes only one > > or two directories have their actual dependencies satisfied & are > > thus buildable. So it would be ~trivial to only hand "-P16" over to > > build.pl, and let it distribute that across the active dmake > > instances. > No, thats not trivial: the current bottlenecks cant really be noticed > by build.pl until its to late (unless build.pl kills runnning > dmake-processes for a directories and restarts them again, which will > hardly improve performance and might trigger additional subtile and hard > to debug bugs in the build system). > This is a circular argument. Correct dependencies and the underlying build/make system used are orthogonal. As an aside, I guess I'm only questioning the rationale here, not the goal - getting rid of the build.pl perl hell would be quite appreciated. ;) > > There's still loads of redundancy - > I cannot see a lot of redundancy there, unless one is willing to > reorganize quite a bit of the files in the source tree: > - clean up the weird include paths in sw > - for each shared library, keep all files in one directory > That would allow to collect the source files implicitly (still, > collecting source files dynamically might be a performance issue for > complete (re-)builds). > Granted, quite some redundancy results out of OOo's inconsistency on many levels; still, why not encoding that knowledge in one place, instead of spreading it over umpteen makefiles again? Actually the current libs.mk does a better job of encapsulating things, compared to the manual fetch of lib names via OOoLibrary_get*, RTLibrary_get* etc. - so what I'd ideally like to see is something along this lines (asking for a DSL again, I guess), instead of target_lib_msword.mk: # loads cxxflags, weird sw include setup etc. include sw_defaults libtarget=msword msword_SOURCES= \ $(listcxx sw/source/filter/rtf) \ $(listcxx sw/source/filter/ww8) msword_linklibs=$(extractdeps $(msword_SOURCES)) The last line may well be a pipe dream for the while, but at least in theory, one should be able to glean the needed libs from the #include statements ... > > Yes. And? It's still significantly lowering the barrier of entry. > > Loads of other projects do it (with the same limitation). > An digression beforehand: I guess we are talking about Windows here > mostly > Nope. I'm talking about IDE users in general, who are used to method/param completion, symbol database etc etc. > So a generated project file might introduce subtle different builds, > is dependent on the version of the generator and is read-only. Since you > can "generate a project from existing code" trivially in NetBeans, > Visual Studio and others and just as well kick of a makefile-based > build in all modern IDEs, what exactly is gained by generated project > files? > If that gives the same developer experience - sure. I'd then be interested in some pointers inside the wiki's build guide. ;) > > Additionally, and since you mentioned the desire to have only one > > make instance - last time someone tried to have gnu make hold all of > > OOo's dependency tree in one process, that guy (Kai Backman) ended > > up with absolutely pathetic performance & ridiculous mem usage. > I did more than a few scalability tests with GNU make. There is a magic > barrier on the number of rules it can handle (above ~1 > scalability grinds to a halt), but rarely one on targets. Unfortunately, > we will only really know in the end (In theory, theory and practice are > same. In practice -- not.). But the four modules in the prototype are > already a significant part of the source files and probably a major > part of the header targets (via #include). > Would then be interesting to know how implicit rules are treated inside gnu make (i.e. whether each instantiation counts) - we'd be in that ballpark easily, with some 11k cxx files & what else comes on top. > (1) http://www.joelonsoftware.com/articles/LeakyAbstractions.html > Prior art: http://www2.parc.com/csl/groups/sda/publications/papers/Kiczales-IMSA92/for-web.pdf And in fact inescapable, ~every implementation will leak under its abstractions. So I guess you're rather arguing for having only _one_ implementation - which is not obtainable in a cross-platform, opensource project. ;) Cheers, -- Thorsten pgp1ToyAHALCx.pgp Description: PGP signature
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Björn Michaelsen wrote: > > - regarding parallelization, that's surely fixable with much less > >effort in build.pl, no? > Currently we are starting one dmake-process per directory and that > dmake process does paralellization the directory. Implementing a > recursive jobserver that communicates between dmake and build.pl would > not only be ugly, it would also be a major effort. > That's not what I meant. The reason to also hand over a specific dmake option for multi-processor builds is that sometimes only one or two directories have their actual dependencies satisfied & are thus buildable. So it would be ~trivial to only hand "-P16" over to build.pl, and let it distribute that across the active dmake instances. > > - what kind of dependency tracking is missing in the current system? > Those that bite you on compatible builds. > Ah. That's what I thought. So nothing inherently missing in dmake/build.pl, but "just" bugs in the makefiles. > > - the question of correct dependencies is probably rather > >orthogonal to the question of which build system to use. much of > >the problems here are in modules like scp2, helpcontent, > >writerfilter etc. where tons of stuff is built via ad-hoc rules > >that simply don't get dependencies right. Having those handled in > >a declarative way would convince me a bit better, that a change > >in the build system actually addresses these issues - > >(see also Kay's > >http://blogs.sun.com/GullFOSS/entry/and_what_about_make) > We are using a declarative language -- also Kay knows what we are doing > and has contributed valuable input. I guess he would tell me, if I > tried something obviously wrong. ;-) > And yes, the build system to use is only "the color of the bikeshed", > but this is a chance to get rid of quite a bit of self-maintained > dependencies. As said elsewhere, the effort never aimed to simple > replace build.pl/dmake with GNU make for its own sake. > Sorry, but http://hg.services.openoffice.org/hg/cws/gnu_make/raw-file/2518db232510/sw/prj/target_lib_msword.mk then leaves a lot to be desired. There's still loads of redundancy - and actually, when compared to a dmake-style makefile without the carried-over cargo-cult contents - quite similar. > see my other replies here and on the blog. As a sidenote: project files > for common IDEs only give you more trouble, if they are one way (and > they currently all are): They are a just minor simplification for > newcomers for a simple build without changing anything. But I leave it > to you to explain to release engineers, why it is their job to translate > the changes made by a new dev in his IDE-project back to the cmake > source. They will rightfully refuse that. Thus the newcomer will have > to fiddle with the makefiles again manually, winning nothing in the end > (other than additional confusion and probably some needless > communication on mailing lists). > Yes. And? It's still significantly lowering the barrier of entry. Loads of other projects do it (with the same limitation). And the need to add a new file to the project is usually not the first, not the second, but the nth thing you do when starting to hack ... So stepping back a bit, I should probably mention (again) that I find the general idea really great - the make system is truly arcane & leaves a lot to desire. But despite your initial request for input, plans & thoughts, I cannot but have the impression you're already quite determined to follow the outlined route - sadly the build system has seen quite a few attempts for change; and survived all of them ~unmodified, thus some extra thought & consultation is surely advisable ... From my humble point of view, what has usually worked best in OOo land is some iterative approach to change; which in this case & my opinion would mean cleaning up makefiles one by one, either using a declarative DSL directly that could later be mapped to gnu make or whatever tool we see fit, or - using a meta-language like automake cmake, or something homegrown, *still* use dmake for the while, and then, after some critical mass has been attained, switch the make tool wholesale (and adapt the metalang-to-makefile generator). Additionally, and since you mentioned the desire to have only one make instance - last time someone tried to have gnu make hold all of OOo's dependency tree in one process, that guy (Kai Backman) ended up with absolutely pathetic performance & ridiculous mem usage. That's why he went for bjam ... Cheers, -- Thorsten pgp2YkH7PTPIv.pgp Description: PGP signature
[tools-dev] Re: Building OpenOffice.org with GNU make
Michael wrote: > [reasons against autotools] > Granted. Though re-working OOo's make system in and of itself will consume substantial resources, so I guess the question whether to code another make system that's only used inside OOo, or whether to use/extend something that's in wide-spread use out there deserves some thoughts ... > [cmake] > > but regarding IDEs, Bjoern just yesterday tried loading the sw module and > the inc directory from the solver into some IDEs... > iirc, visual studio parsed the headers for half an hour, and was then > unusably slow; netbeans (with some gigabytes of heap) was barely usable. > Reportedly eclipse works rather well with a limited set of modules; which will be more or less attained via the split build endeavour at some point in time. > does anybody really use an IDE for developing OOo? > from what i know people use text editors, or they use an IDE basically > like a text editor. > I know a few cases where people have tried; ultimately though, if one becomes a core dev, folks work on OOo the way you describe it (which may be related to the fact that maintaining those project files manually is a PITA). But may considerably lower the barrier of entry for new devs. Cheers, -- Thorsten pgpRFrO0S5w04.pgp Description: PGP signature
[tools-dev] Re: Building OpenOffice.org with GNU make
Hi Bjoern, *, so first off, really cool to see thoughts & effort spent on this: http://blogs.sun.com/GullFOSS/entry/building_openoffice_org_with_gnu I didn't really spent much time mulling over this, it's basically just my current random thoughts on it: - regarding parallelization, that's surely fixable with much less effort in build.pl, no? - what kind of dependency tracking is missing in the current system? - the question of correct dependencies is probably rather orthogonal to the question of which build system to use. much of the problems here are in modules like scp2, helpcontent, writerfilter etc. where tons of stuff is built via ad-hoc rules that simply don't get dependencies right. Having those handled in a declarative way would convince me a bit better, that a change in the build system actually addresses these issues - (see also Kay's http://blogs.sun.com/GullFOSS/entry/and_what_about_make) More generally, I'd guess the whole make system question, once posed, has a much (if not more) potential for religious wars than the dscm question. :) In line with that, why not using autotools? It's ugly, it has many drawbacks, but it's _the_ standard for FLOSS. Plus, it has excellent cross-build capabilities, something I consider increasingly important. Or something like cmake, which could give you project files for common IDEs (see http://lists.freedesktop.org/archives/ooo-build/2009-August/000181.html for some initial attempts)? Cheers, -- Thorsten pgpcxXUQkKnaI.pgp Description: PGP signature
Re: [tools-dev] Ant version upgrade to 1.7.1?
Rene Engelhard wrote: > > This is very unfortunate since the latest version of XCode available > > for PPC only compes with ant 1.6.5 > > [snip] > > oldstable is ignorable, but Debian stable contains 1.7.0 and will never get > 1.7.1 (well, only when squeeze is released) > Incidentally, what exactly is it that makes hsqldb require that specific ant version? Maybe this is something easily fixable _there_? Cheers, -- Thorsten signature.asc Description: Digital signature
Re: [tools-dev] Warn recent changes in files being deleted: does it make sense?
Jens-Heiner Rechtien wrote: > > Does it really "detect" these cases or does that only work when "hg > > rename" was being used. If I would have used only filesystem commands to > > rename files would hg recognize that? > > How should any SCM detect filesystem only renames? This is just asked to > much. You have to use "hg rename", of course. Anything else would have > to rely on heuristics which might fail, hardly something a SCM can do. > Hi Heiner, hm, why is there any heuristic needed? If, for a commit, a file vanishes on one place in the tree and re-appears in another, it's a move - and that's how git handles such cases; automagically. ;) Cheers, -- Thorsten signature.asc Description: Digital signature
[tools-dev] Re: [tools-tinderbox] Moving to bost 1.3?
[Cc to tinderbox removed] Frank Schönheit - Sun Microsystems Germany wrote: > As an additional note, it has been suggested to not commit the > boost*.tar.gz to boost/download, but make it a pre-requisite which needs > to be downloaded before building. This would (for 1.39) save >50M in the > repository for every version ever committed there. > Opinions on this plan are also welcome. > Hi Frank, hm - in the light of heavy-weight commits like the .sdf one, ain't this just a micro-optimisation? Unless such stuff gets downloaded automagically, it's a big nuisance in the already-full-of-nuisances ooo build experience. Or invent a nice solution that does auto-downloads, and switch a few other huge external libs to that (like icu). ;) Just my 2c, -- Thorsten signature.asc Description: Digital signature
[tools-dev] svnall mails, anybody?
Hi, I was wondering, is anybody getting svnall commit mails still? It started trickling for me a while ago, and has now stopped completely... Thanks, -- Thorsten - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
Re: [tools-dev] tcsh-support in the build still needed?
Philipp Lohmann wrote: > Why is it that everybody wants EVERYTHING configurable from with/without > mozilla and we need to cope with every buggy freetype version (as > example, insert your favorite "system" library here), but all of a > sudden it would be considered a good idea imposing a shell on people ? > Hi Philipp, to give tcsh the justice it deserves, the comparison should rather be between being able to choose freetype (every buggy version) or, lets say, AGG for font rasterization. I believe we want to support every other buggy version of bash... ;) > > I have a better idea: let's skip that configure crap and instead make > everything that is configurable now a prerequisite. > > For quite a few prerequisites, that's effectively the case already (java is a case in point I guess). :/ Cheers, -- Thorsten - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
Re: [tools-dev] Re: Experimenting with CMake
Christian Lohmaier wrote: > >> This is one of the reasons for including copies of external libraries > >> in OOo, this is one of the reasons for rejecting contributions not > >> covered by the SCA. > >> > > Nonsense. Every piece of code OOo uses can be modified under at > > least LGPL terms (or looser). There is no reason for SCA. ;) > > Misses the point completely. > Nope. Was replying to the last part of your sentence, and intended it to be somewhat of an off-hand remark. So, > But please don't drag this thread into politics please. > Didn't start, but surely don't want to wander off from the actual (and important) topic as well. :) Cheers, -- Thorsten - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
Re: [tools-dev] Re: Experimenting with CMake
Christian Lohmaier wrote: > [being able to fix bugs in a ooo-copied piece of software] > > This is one of the reasons for including copies of external libraries > in OOo, this is one of the reasons for rejecting contributions not > covered by the SCA. > Nonsense. Every piece of code OOo uses can be modified under at least LGPL terms (or looser). There is no reason for SCA. ;) Cheers, -- Thorsten - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
Re: [tools-dev] Re: Experimenting with CMake
bjoern michaelsen - Sun Microsystems - Hamburg Germany wrote: >> A recurring theme in OOo conference presentations and similar material >> seems to be that hacking on OOo is hard for newcomers partly because >> it is such a complex beast to build. Making it easier could bring in >> more contributors. > dmake is not what makes OOo complex. > I guess that slightly misses the point, as I believe Jussi also proposes getting rid of the build.pl/dmake marriage, plus much of the hard-coded, non-declarative stuff in the makefiles. I'm overall *much* in favor of change here, as the build system is one of the first contact points for new devs, and it *is* arcane, non-standard, and largely inefficient (I have a good recollection of Kai's jam measurements...). Whether cmake is the best possible choice I don't know, though (but then again, if someone volunteers to work on that, I'd definitely not discourage him). Cheers, -- Thorsten - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
[tools-dev] OpenGrok, svn & bonsai-replacement
Hi, just been told today that svn.services.openoffice.org/opengrok/ is operational, which totally rocks. Played a bit with it, and apparently it does lack the functionality bonsai delivered for cvs (it's a much nicer replacement for lxr, though). So, I guess what I'd like to have is ViewVC with the additional tweaks the gnome repo provides, e.g. when doing CWS development, I frequently review changes like this: http://svn.gnome.org/viewvc/ooo-build/branches/ooo-build-3-0/?view=query&dir=&file=&file_match=exact&who=&who_match=exact&comment=&comment_match=exact&querysort=date&hours=2&date=month&mindate=&maxdate=&limit_changes=100 Any chance of getting this set up (we could surely help with the details)? Cheers, -- Thorsten - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] SVN and email notifications (resend)
On Sun, Oct 19, 2008 at 12:11:11AM +0200, Frank Schönheit - Sun Microsystems Germany wrote: > > notifications to [EMAIL PROTECTED] should work now. > > [snip] > > I know that for SVN, the notion of project and module is meaningless, so > anything to set up here is certainly non-trivial. Nonetheless, it would > be useful to have some kind of subscription to a subset of allsvn. > Hi Frank, I have no strong feelings about it, but: ain't this easily achievable with a mail filter? I once started monitoring the [EMAIL PROTECTED] list, but quickly moved on to allcvs, simply because the partition into projects was so artificial... ...and won't commits to svx/source/form be of interest to you as well? ;) Cheers, -- Thorsten - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] SVN and email notifications (resend)
On Wed, Sep 24, 2008 at 05:21:11PM +0200, Jens-Heiner Rechtien wrote: > Email notifications can be implemented with the post-commit hook. We'll > implement that. > Hi Heiner, is this working yet? Haven't seen commits yet, though subscribed to [EMAIL PROTECTED] Cheers, -- Thorsten - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] OpenOffice.org subversion repository R/W enabled on Tuesday
On Mon, Sep 29, 2008 at 01:46:55PM +0200, Stefan Taxhet wrote: >> Again, what about EIS' feature to upload a key? (Which I did long ago). >> Is that working? > > There is no automated process that installs keys for svn+ssh write > access - intentionally. > > While key upload via EIS might look handsome it lacks the ability to > comment, transparency and notifications IssueZilla offers. That's why we > decided to use IZ for the migration as well as for ongoing maintenance. > Hi Stefan, then, why is it still there? I was also lured into thinking it will be enough to upload there (as, in fact, I consider Issuezilla less convenient ;)). Cheers, -- Thorsten - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] Problem using MS VC++ 2005 Express to build 2.4.0on Windows XP with --enable-debug
On Sun, Apr 06, 2008 at 03:51:37PM +1000, Graham Stoney wrote: > Do you mind if I add your patch as an attachment to issue 47225, so > others can also find it easily? > Not at all, please do! Cheers, -- Thorsten - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] Problem using MS VC++ 2005 Express to build 2.4.0 on Windows XP with --enable-debug
On Sun, Apr 06, 2008 at 08:47:49AM +1000, Graham Stoney wrote: > Is there a patch available to fix --disable-mozilla on Windows? Even a > makefile hack to get past xmlsecurity would do for now. > Hi Graham, attached patch should fix this. Cheers, -- Thorsten --- xmlsecurity/source/xmlsec/makefile.mk 7 Nov 2007 10:06:47 - 1.10 +++ xmlsecurity/source/xmlsec/makefile.mk 7 Mar 2008 14:37:05 - @@ -45,6 +45,11 @@ ENABLE_EXCEPTIONS = TRUE .INCLUDE : settings.mk .INCLUDE : $(PRJ)$/util$/target.pmk +.IF "$(WITH_MOZILLA)" == "NO" [EMAIL PROTECTED]: + @echo "No mozilla -> no nss -> no libxmlsec -> no xmlsecurity..." +.ENDIF + .IF "$(SYSTEM_LIBXML)" == "YES" CFLAGS+=-DSYSTEM_LIBXML $(LIBXML_CFLAGS) .ENDIF @@ -53,10 +58,6 @@ CFLAGS+=-DSYSTEM_LIBXML $(LIBXML_CFLAGS) CDEFS += -DXMLSEC_CRYPTO_MSCRYPTO -DXMLSEC_NO_XSLT .ELSE CDEFS += -DXMLSEC_CRYPTO_NSS -DXMLSEC_NO_XSLT -.IF "$(WITH_MOZILLA)" == "NO" [EMAIL PROTECTED]: - @echo "No mozilla -> no nss -> no libxmlsec -> no xmlsecurity.." -.ENDIF .ENDIF # --- Files - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] Problem with solenv/bin/linkoo in SRC680_m235
Stephan Bergmann <[EMAIL PROTECTED]> writes: > No. What I meant is if you tell a newcomer she need only call linkoo > once and never copy newly build libs around, that will fail for her if > she works on one of the libs from the blacklist. > True. Luckily, those libs are complex & low-level enough to be almost tamper-proof from newcomers. ;-) To get back to where we started: any ideas/concepts on how to properly fix this problem? Cheers, -- Thorsten - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] Problem with solenv/bin/linkoo in SRC680_m235
Stephan Bergmann <[EMAIL PROTECTED]> writes: > If at all, hacks like symlinking are something for experienced > developers who know exactly the implications of doing this. You > should *never* encourage a newcomer to do such hacks (cf. the > blacklist already present in linkoo). > Hi Stephan, well, linkoo & this blacklist have served me and a bunch of others quite well over more than a year - I'd guess that's enough empirical evidence to say that it does work reliably. And for the newcomer, this 'hack' is nicely wrapped into a tool she can readily use. > If we want features like linkoo, first make the code ready for > them. (Maybe I was in a rush when I introduced osl_loadModuleRelative, > maybe some vnd.sun.star.expand:$WHATEVER-PREFIX scheme would be > better. I am very open for discussion about *that*.) > Fine with me. I'm not insisting on the current solution, only pointing out that triple copying around of files (local output -> solver -> installation tree) is a major PITA and plainly non-obvious for new developers. So, constructive ideas, anybody? -- Thorsten - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] Problem with solenv/bin/linkoo in SRC680_m235
Michael Meeks <[EMAIL PROTECTED]> writes:
> > I don't think you can easily make that work again with some env variable:
> > osl_loadModuleRelative("foo") called from /resolved/path/to/bar.so
> > causes a dlopen("/resolved/path/to/foo.so").
>
> Sure - the env. var. can clobber / affect what loadModuleRelative does
> (and un-relativeize it - or at least make it fall-back to what it used
> to do on failure ? ;-)
>
Yes, please let's have such an (optional) mode. I've always loathed
this silly copying, and it's just another stupid thing to explain to
new developers...
> > (And I am sure there already are other problems with linkoo,
> > anyway, like the RPATH=$ORIGIN thing.)
>
> Has been working for the last year or more rather well - albeit with
> some black-lists for the previous offending libraries that insisted on
> asking the OS where they were installed & looking only there for other
> libs ;-)
>
Confirmed.
Cheers,
-- Thorsten
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] Re: [releases] git: Version control system for OOo?
Jörg Jahnke <[EMAIL PROTECTED]> writes: > Jan Holesovsky schrieb: > > Of course we could do this - but you know how slow the resyncs are. > > I had a CWS where resync took 36 hours - no kidding. Here in the > > office, we have a build machine that is about that powerful as what > > CN can offer for the entire SVN with all its users. How long do you > > think the resync will take there locally with a tool like git that > > is _designed_ to do resyncs and merges quickly... One minute? > > Five? At most. Not mentioning that git solves conflicts much > > better - because it has the entire history of both branches. > > > > What would be an acceptable time for a cwsresync for you? 36 hours > certainly seems much too high, at least when the resync is to be done > regularly. But would 1 hour perhaps be acceptable? And if with > Subversion a resync were possible within 1 hour, would the scenario > which Bernd described be imaginable for your work? > Hi Joerg, well, given that git is able to perform the task in about no time, why would anything that takes an order of magnitude longer be acceptable? Besides that, my trouble with that centralized approach are e.g. that this mandates exactly one policy - for commit rights, licensing, process, you get the idea. Having a staging tree of DSCM repos allows for relaxed rules on some of them, while still maintaining our standards on the 'vanilla' repo. I mean, a dev can simply clone that repo, and commit right away on his disk, without any administrative overhead whatsoever. Cheers, -- Thorsten - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] Re: [releases] git: Version control system for OOo?
Martin Hollmichel <[EMAIL PROTECTED]> writes: > I would not like to declare subversion as a interim solution, this > sounds like being not good enough. > Well - yes. I do think it's not good enough, in the sense that a DSCM is a better alternative. > I think we can agree on: "We are continuously for the best solutions > for developing OpenOffice.org". > Of course we can. :-) Cheers, -- Thorsten If you're not failing some of the time, you're not trying hard enough. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] Re: [releases] git: Version control system for OOo?
Martin Hollmichel <[EMAIL PROTECTED]> writes:
> A switch to DSCM is the bigger and a promising step: enhancements of
> our development model and style look possible...
>
Right - generally, a distributed approach fits the multi-tier,
decentralised world of OOo much better. Besides that, having the full
repository on my local disk just rocks, in terms of speed, latency,
and what I can do with simple scripts.
Taking the tree graph from kendy's presentation (the one with the
bubbles labelled OOo master, StarOffice, OOBuild etc.), this model
makes line-by-line differentiation from the master repository
('vanilla' OOo) straight forward - at least for StarOffice, we've not
been able to do that in the past, the granularity for differentiation
was on module level.
> I guess some of you may think that with a transition from CVS to
> subversion this discussion might be dead, but I definitely don't
> think so. The opportunities we have with a modern SCM are to much to
> get ignored but we need some time for such thought which I don't
> want to let get passed by by sitting on our old CVS.
>
Indeed, I'm a bit concerned that moving to subversion will make us
suffer from the 'good enough' syndrome. But I can clearly live with
that, if we agree that it's an interim solution (and given that svn
tooling for CWS is almost done - if we need to invest half a year to
have that running, we could as well move to DSCM).
Just my 2 cents,
-- Thorsten
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] using a distributed SCM cross corporate networks
Jan Holesovsky <[EMAIL PROTECTED]> writes: > Should I enable http:// access to go-oo.org/git? > Dunno what went wrong for Martin - I was able to clone your repo from within Sun. Cheers, -- Thorsten - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] Problem compiling OO.org 2.0.1
peto <[EMAIL PROTECTED]> writes: > but maybe it is in gcc > > [EMAIL PROTECTED] peto]$ gcc --version > gcc (GCC) 3.2 (Mandrake Linux 9.0 3.2-1mdk) > Hi Peter, yes, that might indeed be the case - Sun builds with 3.4.1, the oldest version I've personally tried is 3.3. Please note that this code is in since October, and has probably been successfully built thousands of times. You might try to temporarily comment out the offending code, and see how far it takes you - if this remains the only problem, I'm willing to invest some time to sort it out. > I will try to find includes in header- I do not know why .hxx is > used instad of simple .h > Well, to distinguish it from pure c headers - not uncommon in c++ land. Cheers, -- Thorsten If you're not failing some of the time, you're not trying hard enough. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [tools-dev] Problem compiling OO.org 2.0.1
peto <[EMAIL PROTECTED]> writes: > It takes error > /mnt/umsd/OOA680_m1/slideshow/source/engine/animationnodes > -- > Making: ../../../unxlngi4.pro/slo/animationbasenode.obj > > Hi Peter, you're sure that you've checked out the correct version? Please verify that animationbasenode.cxx is rev. 1.8, and animationbasenode.hxx is rev. 1.4. HTH, -- Thorsten If you're not failing some of the time, you're not trying hard enough. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
