Re: Has the time come to get rid of the delivering of public headers?
On Thu, Feb 23, 2012 at 11:41:20AM +, Michael Meeks wrote: :-) Any chance of some more advanced 'easy hacks' on this ? and/or a Google Summer of Code idea: https://wiki.documentfoundation.org/Development/Gsoc/Ideas Done. Best, Bjoern ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On Thu, 2012-02-23 at 01:22 +0100, Bjoern Michaelsen wrote: It is neither an anachronism or has something to do with building over the network. Nobody sane did that on Windows anyway Surely that is what Hamburg release engineering did for years :-) But for things like sw or sc, the lack of encapsulation and the need to include 75% of all the public headers in each and every compile (e.g. via sw/inc/doc.hxx) and reparse those are a real killer. Ok - so, at least some better data. But this seems to suggest that the problem is not multiple include-path sub-directories (as was suggested) but badly structured includes - which is an orthogonal problem - right ? At least, I share Tor's view that we should have some real numbers about the cost of a given thing: multiple include paths, vs. single ones before doing the re-licensing then the code re-structuring ;-) ATB, Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On Thu, Feb 23, 2012 at 10:07:21AM +, Michael Meeks wrote: But for things like sw or sc, the lack of encapsulation and the need to include 75% of all the public headers in each and every compile (e.g. via sw/inc/doc.hxx) and reparse those are a real killer. Ok - so, at least some better data. As I cant provide numbers anymore it is also just anecdotal evidence. But this seems to suggest that the problem is not multiple include-path sub-directories (as was suggested) but badly structured includes - which is an orthogonal problem - right ? Both, in a way: The huge number of includes is the root problem, but not trivial to fix. The number of -I-switches to the compiler being performance relevant is a symtomn of that and reducing those might improve build time, but it is not fixing the root problem. Fixing the root problem might involve a git hook that gives you an angry stare for 30 seconds everytime you add more lines to a 1.5KLOC hxx than you remove from it. At least, I share Tor's view that we should have some real numbers about the cost of a given thing: multiple include paths, vs. single ones before doing the re-licensing then the code re-structuring ;-) Indeed. Best, Bjoern ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On Thu, 2012-02-23 at 11:24 +0100, Bjoern Michaelsen wrote: Fixing the root problem might involve a git hook that gives you an angry stare for 30 seconds everytime you add more lines to a 1.5KLOC hxx than you remove from it. Heh ;-) IMHO a weighted stare-length based on how many times this header is included might be more useful. Luckily we have that information in an easy to consume form in the dependency information. We even have the easy hack: https://bugs.freedesktop.org/show_bug.cgi?id=42949 :-) bytes to parseinclude count sizefilename 741439564 4454 166466 solver/unxlngi6.pro/inc/boost/preprocessor/seq/fold_left.hpp 416181480 6754 61620 solver/unxlngi6.pro/inc/rtl/ustring.hxx etc. ... I guess if we knew that it was discrete file accesses that cause windows performance issues, we could easily triage the list for small / trivial headers to merge. Failing that, boost-ification and gigantic 'left folding' headers [urk?] will no doubt continue to harm compile times. I guess we'd want to encourage someone to take over from August on the include-what-you-use front, but - that seems to suggest adding more includes to compiling code so ... ;-) HTH, Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On Thu, Feb 23, 2012 at 11:14:03AM +, Michael Meeks wrote: I guess we'd want to encourage someone to take over from August on the include-what-you-use front, but - that seems to suggest adding more includes to compiling code so ... ;-) ... and then the next step is to slice-and-dice god-objects like SwDoc in small little one object, one job parts so that some part of code concerning page layout doesnt have to care about how fields are maanged (and reparse all includes from that without ever using them). That can be done with one responsibility at a time. Incedentally, that will also improve code readability. Best, Bjoern ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On Thu, 2012-02-23 at 12:30 +0100, Bjoern Michaelsen wrote: ... and then the next step is to slice-and-dice god-objects like SwDoc in small little one object, one job parts so that some part of code concerning page layout doesnt have to care about how fields are maanged (and reparse all includes from that without ever using them). :-) Any chance of some more advanced 'easy hacks' on this ? and/or a Google Summer of Code idea: https://wiki.documentfoundation.org/Development/Gsoc/Ideas That can be done with one responsibility at a time. Incedentally, that will also improve code readability. Yep, seems like a very worthy goal. :-) Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
Maybe we need a new directory for exported header files. $MODULE/exported That way, it lives inside the module, but it's obvious that it's part of the external facing API. On 2012-02-21 16:34, Eike Rathke wrote: Hi Tor, On Tuesday, 2012-02-21 10:02:50 +0200, Tor Lillqvist wrote: Has the time come to get rid of the delivering step for public headers? I.e. for a headerMODULE/BAR.hxx, instead of having it in MODULE/inc/MODULE/BAR.hxx and then copying it to solver/INPATH/inc/MODULE/BAR.hxx as part of the make of MODULE, it could be directly in a new top-level inc directory, in inc/MODULE/BAR.hxx all the time. (Obviously -I$(SRC_ROOT)/inc would have to be added to the compilation flags.) I see no technical reason against it, but I wouldn't like it because then the headers belonging to a module wouldn't be located in the module anymore. Actions performed on a module would need to take that additional inc path into account, for example grep, module local tags and ID databases, and maybe more. Doxygen? Eike ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice Disclaimer: http://www.peralex.com/disclaimer.html ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On 22/02/12 15:09, Lubos Lunak wrote: On Tuesday 21 of February 2012, Michael Stahl wrote: I wonder how bad the alternative of adding -I$S/svx/inc -I$S/vcl/inc etc. etc. is for command-line length ? would need to try that out on Windows, but my guess is that even if it doesn't blow out command line length limits (which could be worked around with a response file) it'll make the Windows build a lot slower because compiler has to try out all include paths (well statistically half of them but that is bad enough) for every included files, and those file accesses take forever on Windows. Could not the windows build be helped by having only one location for include files where a script would at the beginning of the build create for each .hxx file a small wrapper with the same name only doing #include of the real one, with the right path to it? that sounds like a horrible hack (and it would still be necessary to open 2 files instead of 1 so it would still be slower). another simple alternative might be to have a single directory that contains symlinks to all module inc directories: inc/$MODULE - $SRC_ROOT/$MODULE/inc/$MODULE but i have doubts that that would actually work on windows, is there any kind of link that MSVC will follow? And, since I'm already asking, is the Windows performance the reason include files are scattered this way? E.g. in sw/ I find this actually rather annoying, some .hxx files are in inc/, some next to .cxx sources, Writer AFAIK does not really export anything anyway, and I always have to jump around just to find the right header file :(. in sw the include files are distributed to ensure that e.g. core implementation details are not available to UI code (though there is probably still a lot of stuff in sw/inc that really should be in sw/source/core/inc). one generally learns to use ctags and grep to look up stuff anyway, given the random 8 letter header names. ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On Tue, Feb 21, 2012 at 8:39 AM, Michael Stahl mst...@redhat.com wrote: On 21/02/12 11:59, Michael Meeks wrote: Hi Tor, On Tue, 2012-02-21 at 10:02 +0200, Tor Lillqvist wrote: solver/INPATH/inc/MODULE/BAR.hxx as part of the make of MODULE, it could be directly in a new top-level inc directory, in inc/MODULE/BAR.hxx all the time. (Obviously -I$(SRC_ROOT)/inc would have to be added to the compilation flags.) that's one way of doing it, which would require all headers to be under a new top-level directory. not sure whether we have other options as well, e.g. how would this work: - module/inc contains the public headers of the module - module/source/inc contains internal headers of the module - put top-level dir in include path I must say, of all the options so far, that one sound the most appealing to me. and it can be done 'incrementally', one module at the time... While we are at it, why not have just one public header file per module... that would seriously simplify the header inclusion stuff. sure the downside is that that header can be pretty big, and any change in a module's header would trigger a rebuild of all the dependent module.. on the other hand the include logic could be seriously simplified and clean, and the intermodule dependecy tracking become quite light (make wise) to a point where they can be expressed by static rules Norbert ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On 02/22/2012 04:23 PM, Norbert Thiebaud wrote: While we are at it, why not have just one public header file per module... that would seriously simplify the header inclusion stuff. sure the downside is that that header can be pretty big, and any change in a module's header would trigger a rebuild of all the dependent module.. I think I would not like that, given the constant flux those headers are in. Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
And, since I'm already asking, is the Windows performance the reason include files are scattered this way? Personally I take these claims about the horrible performance of Windows (itself, not just Cygwin) doing this or that with a grain of salt. After all, they come from people who have a vested interest in promoting Linux, no? And who in some cases might not have touched Windows for years. But oh well, on the other hand I certainly am not interested enough to do any actual benchmarking. Whatevers. --tml ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
I must say, of all the options so far, that one sound the most appealing to me. and it can be done 'incrementally', one module at the time... One could do the thing I suggested incrementally, too. While we are at it, why not have just one public header file per module... But isn't this module terminology mostly wishful thinking in many cases? I.e. some modules contain stuff that are not really related much at all from a functionality point of view, and in other cases stuff that one would imagine belongs together has been split into several modules for obscure and/or obsolete reasons. --tml ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On Wed, Feb 22, 2012 at 10:30 AM, Tor Lillqvist t...@iki.fi wrote: I must say, of all the options so far, that one sound the most appealing to me. and it can be done 'incrementally', one module at the time... One could do the thing I suggested incrementally, too. While we are at it, why not have just one public header file per module... But isn't this module terminology mostly wishful thinking in many cases? I.e. some modules contain stuff that are not really related much at all from a functionality point of view, and in other cases stuff that one would imagine belongs together has been split into several modules for obscure and/or obsolete reasons. be that as it may, that would just be an incentive to actually fix these case... Norbert ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
be that as it may, that would just be an incentive to actually fix these case... I am sure people would come up with good arguments why everything has to stay as it is. --tml ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On Wed, Feb 22, 2012 at 11:31 AM, Tor Lillqvist t...@iki.fi wrote: be that as it may, that would just be an incentive to actually fix these case... I am sure people would come up with good arguments why everything has to stay as it is. Point n’est besoin d’espérer pour entreprendre ni de réussir pour persévérer. “It is not necessary to hope in order to undertake, nor to succeed in order to persevere [Guillaume Ier d’Orange-Nassau] Norbert ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On Wed, 2012-02-22 at 18:26 +0200, Tor Lillqvist wrote: Personally I take these claims about the horrible performance of Windows (itself, not just Cygwin) doing this or that with a grain of salt. Agreed :-) I think a chunk of this folk wisdom may be a horrible anachronism from the time when StarDivision's build system would build over the network, such that every stat of a file in a directory was a network latency away, instead of some (hopefully fastish) local file-system lookup. But oh well, on the other hand I certainly am not interested enough to do any actual benchmarking. Whatevers. So - I suggest we do what is pretty and pleasant, and then profile it and re-think if it turns out to be extremely slow :-) ATB, Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On Wed, Feb 22, 2012 at 03:09:49PM +0100, Lubos Lunak wrote: Could not the windows build be helped by having only one location for include files where a script would at the beginning of the build create for each .hxx file a small wrapper with the same name only doing #include of the real one, with the right path to it? An _additional_ file to open? I highly doubt it will increase windows performance. I would assume the opposite, but of course, I am open to hard facts. And, since I'm already asking, is the Windows performance the reason include files are scattered this way? E.g. in sw/ I find this actually rather annoying, some .hxx files are in inc/, some next to .cxx sources, Writer AFAIK does not really export anything anyway, and I always have to jump around just to find the right header file :(. No its just historic mess. Never attribute to malice ... Best, Bjoern ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Has the time come to get rid of the delivering of public headers?
Has the time come to get rid of the delivering step for public headers? I.e. for a header MODULE/BAR.hxx, instead of having it in MODULE/inc/MODULE/BAR.hxx and then copying it to solver/INPATH/inc/MODULE/BAR.hxx as part of the make of MODULE, it could be directly in a new top-level inc directory, in inc/MODULE/BAR.hxx all the time. (Obviously -I$(SRC_ROOT)/inc would have to be added to the compilation flags.) Now with OneGit, there shouldn't be any technical reason not do do this. It would also speed up the build a little bit. Am I missing something, or can I JFDI? --tml ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
Hi Tor, On Tue, 2012-02-21 at 10:02 +0200, Tor Lillqvist wrote: solver/INPATH/inc/MODULE/BAR.hxx as part of the make of MODULE, it could be directly in a new top-level inc directory, in inc/MODULE/BAR.hxx all the time. (Obviously -I$(SRC_ROOT)/inc would have to be added to the compilation flags.) I wonder how bad the alternative of adding -I$S/svx/inc -I$S/vcl/inc etc. etc. is for command-line length ? Now with OneGit, there shouldn't be any technical reason not do do this. It would also speed up the build a little bit. Am I missing something, or can I JFDI? I'd prefer to get the MPL/LGPLv3+ on AL2 re-licensing done first, if possible - it's marginally more unpleasant if lots of files moved around; say a few months' time ? All the best, Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
I wonder how bad the alternative of adding -I$S/svx/inc -I$S/vcl/inc etc. etc. is for command-line length ? Probably not a problem considering that we compile just one file at a time anyway. But might it not be the case that we in some cases (or often even) have non-public headers in MODULE/inc (which are not delivered)? So this might then unintentionally make such headers public. If it matters... say a few months' time ? OK. --tml ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
Hi Tor, On Tuesday, 2012-02-21 10:02:50 +0200, Tor Lillqvist wrote: Has the time come to get rid of the delivering step for public headers? I.e. for a header MODULE/BAR.hxx, instead of having it in MODULE/inc/MODULE/BAR.hxx and then copying it to solver/INPATH/inc/MODULE/BAR.hxx as part of the make of MODULE, it could be directly in a new top-level inc directory, in inc/MODULE/BAR.hxx all the time. (Obviously -I$(SRC_ROOT)/inc would have to be added to the compilation flags.) I see no technical reason against it, but I wouldn't like it because then the headers belonging to a module wouldn't be located in the module anymore. Actions performed on a module would need to take that additional inc path into account, for example grep, module local tags and ID databases, and maybe more. Doxygen? Eike -- LibreOffice Calc developer. Number formatter stricken i18n transpositionizer. GnuPG key 0x293C05FD : 997A 4C60 CE41 0149 0DB3 9E96 2F1A D073 293C 05FD pgpbCS9x95UCH.pgp Description: PGP signature ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
Maybe we need a new directory for exported header files. $MODULE/exported Well, isn't that what $MODULE/inc is *supposed* to be? Are there many cases where also private headers (which don't get delivered) are in $MODULE/inc ? I honestly don't know; on one hand I am a pessimist and assume there are, but on the other hand I don't want to be pessimistic... --tml ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On 21/02/12 15:41, Tor Lillqvist wrote: Maybe we need a new directory for exported header files. $MODULE/exported Well, isn't that what $MODULE/inc is *supposed* to be? Are there many cases where also private headers (which don't get delivered) are in $MODULE/inc ? I honestly don't know; on one hand I am a pessimist and assume there are, but on the other hand I don't want to be pessimistic... currently the prevailing pattern is that $MODULE/inc/$MODULE contains the public headers, while headers directly in $MODULE/inc are private to the module. (in modules that still use deliver it's even worse and sometimes a complete mess...) ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On 02/21/2012 03:39 PM, Michael Stahl wrote: would need to try that out on Windows, but my guess is that even if it doesn't blow out command line length limits (which could be worked around with a response file) it'll make the Windows build a lot slower because compiler has to try out all include paths (well statistically half of them but that is bad enough) for every included files, and those file accesses take forever on Windows. Just want to add that response files can be a pain when you want to debug a build problem. So IMO the fewer places we use them the better. Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On Tue, Feb 21, 2012 at 03:39:25PM +0100, Michael Stahl wrote: On 21/02/12 11:59, Michael Meeks wrote: would need to try that out on Windows, but my guess is that even if it doesn't blow out command line length limits (which could be worked around with a response file) it'll make the Windows build a lot slower because compiler has to try out all include paths (well statistically half of them but that is bad enough) for every included files, and those file accesses take forever on Windows. ^^ This! IIRC, Mathias Bauer benchmarked that once and noted that on Windows each added path has a significant impact on buildtime, so you really dont what to go laissez-faire with those. Also the order of -I-switches can be quite important. Best, Bjoern ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Has the time come to get rid of the delivering of public headers?
On Tue, Feb 21, 2012 at 04:41:33PM +0200, Tor Lillqvist wrote: $MODULE/exported Well, isn't that what $MODULE/inc is *supposed* to be? yes. well: $MODULE/inc/$MODULE is exported headers. Best, Bjoern ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
