Re: about build environment development
fixed my issues. Jim, I had an install issue with latest trunk of your epm repo: make: INSTALL@: Befehl nicht gefunden (command not found) Makefile:153: die Regel für Ziel „install“ scheiterte (install failed) make: *** [install] Fehler 127 I think some sort of typo maybe? 4.4.2 works good. All the best Peter Am 23.04.20 um 15:48 schrieb Jim Jagielski: IMO, dmake is simply a build-and-compile dependency. Considering that we pull in lots of other external dependencies, keeping or dropping dmake will likely not make any real difference in the people we can attract to help develop, test and build AOO. At the very least, we know what dmake is good at, and what it is NOT good at; so I support porting as much as possible to gmake, and just using dmake as the last step. And, FWIW, I am keeping dmake up-to-date and even FreeBSD are using my repo now: https://github.com/jimjag/dmake On Apr 17, 2020, at 8:53 AM, Peter Kovacs wrote: The goal is to move away from dmake. I do struggle with your point of view, saying moving away from dmake is unimportant, and should be stopped. I disagree with this position. Am 16.04.20 um 23:04 schrieb Patricia Shanahan: On 4/15/2020 10:08 AM, Damjan Jovanovic wrote: On Wed, Apr 15, 2020 at 3:15 PM Jim Jagielski wrote: On Apr 15, 2020, at 3:01 AM, Damjan Jovanovic wrote: We are also thin on new contributors, and I recall you saying they're largely scared off by the current build system. Two points: 1. I doubt that by the time we finish porting to a whole new build system, we will even have anyone *wanting* to contribute. With each delay and push-out on releases, and more time spent working on the build system instead of AOO itself, we become less and less relevant. Is that really a priority we should be focusing on? Are the number of people knowledgeable around scons really greater than what we have now? But I could be wrong, which leads me to #2... What would you recommend we focus on instead then? I would recommend going for robustness, rather than new features. I know of some areas for potential improvement: Array bounds checking, especially during input processing. Memory allocation checking. Debug profile corruption. Ideally, new contributors wouldn't need to be knowledgeable about scons. The build should be easier to perform, hopefully just "./configure" followed by "scons" (and scons even implements features that can subsume ./configure too). Already, scons doesn't need the "source winenv.set.sh" and "cd instsetoo_native" steps to build its modules. 2. "The conversion from gbuild to scons would largely be automated, fast and correct." If that is the case, let's test that theory right now. This has been possible for some time. In the scons-build branch, you can do the following: $ cd gotoSCons $ mvn package $ java -cp target/gotoSCons-0.1-SNAPSHOT.jar org.apache.openoffice.gotoSCons.GBuildConverter parsingAnalysis ../main (per-module output) Could parse: [MathMLDTD, UnoControls, animations, cosv, cppcanvas, drawinglayer, eventattacher, fileaccess, i18nutil, idl, io, rdbmaker, registry, remotebridges, sane, store, svgio, twain, ucbhelper, unixODBC, xmlreader, xmlscript] 22 out of 105 gbuild modules are parseable That means 22 modules can already be converted, completely and correctly. As we add more features to the converter (AllLangResTarget, UnoApi, Junit, GoogleTest, etc.), that 22 will increase. The per-module gbuild files are easy to parse. Parsing the syntax takes only 3 methods and < 100 lines of Java. The non-deterministic ones with "ifeq ($(GUIBASE),aqua)" require some manual work, but even there, a lot can be automated. There is some more work involved in semantic conversion: understanding and converting specific gbuild commands, converting paths to scons format, etc. but even so, we're on just 1913 lines of code in total for the converter. The hard part is to convert gbuild functions in main/solenv/gbuild into scons, for example, the worst case scenario is AllLangResTarget, for which this monstrous dependency tree needs to be implemented, with 4 layers of intermediate targets and complex actions with side effects and header dependencies (not shown): # AllLangResTarget(name) # (meta-target; delivers an empty timestamp file) #^ ^ # / \ # / \ # / \ # ResTarget(nameen-US,name,en-US) ResTarget(nameen-GB,name,en-GB) # $(WORKDIR)/ResTarget/$(resName).res $(WORKDIR)/ResTarget/$(resName).res # $(WORKDIR)/ResTarget/nameen-US.res $(WORKDIR)/ResTarget/nameen-GB.res #^ ^^ #| rsc || #| || # SrsTarget SrsTarget SrsTarget # $(WORKDIR)/SrsTarget/$(srsName).srs #
Re: about build environment development
On 4/23/20 10:44 AM, Peter Kovacs wrote: Hi Jim, dmake is not simple. I do know nothing Jim Jagielski... Today the build system enjoys me with checking whether the found dmake is the right dmake... configure: WARNING: no or configure: error: no URL for dmake source code specified, either. Use --with-dmake-url to supply an URL; run configure with the --help option for a list of possible URLs. ./configure.sh: Zeile 22: --with-dmake-url=https://sourceforge.net/projects/oooextras.mirror/files/dmake-4.12.tar.bz2: Datei oder Verzeichnis nicht gefunden but: wget https://sourceforge.net/projects/oooextras.mirror/files/dmake-4.12.tar.bz2 dmake-4.12.tar.bz2 100%[==>] 490,31K 92,2KB/s in 5,3s I am frustrated and will continue, when I had a break. Maybe then I figure out what stupidity I do not see. All the Best Peter Maybe some useful information on dmake? https://docs.oracle.com/cd/E19422-01/819-3697/dmake.html Regards, Kay Am 23.04.20 um 15:48 schrieb Jim Jagielski: IMO, dmake is simply a build-and-compile dependency. Considering that we pull in lots of other external dependencies, keeping or dropping dmake will likely not make any real difference in the people we can attract to help develop, test and build AOO. At the very least, we know what dmake is good at, and what it is NOT good at; so I support porting as much as possible to gmake, and just using dmake as the last step. And, FWIW, I am keeping dmake up-to-date and even FreeBSD are using my repo now: https://github.com/jimjag/dmake On Apr 17, 2020, at 8:53 AM, Peter Kovacs wrote: The goal is to move away from dmake. I do struggle with your point of view, saying moving away from dmake is unimportant, and should be stopped. I disagree with this position. Am 16.04.20 um 23:04 schrieb Patricia Shanahan: On 4/15/2020 10:08 AM, Damjan Jovanovic wrote: On Wed, Apr 15, 2020 at 3:15 PM Jim Jagielski wrote: On Apr 15, 2020, at 3:01 AM, Damjan Jovanovic wrote: We are also thin on new contributors, and I recall you saying they're largely scared off by the current build system. Two points: 1. I doubt that by the time we finish porting to a whole new build system, we will even have anyone *wanting* to contribute. With each delay and push-out on releases, and more time spent working on the build system instead of AOO itself, we become less and less relevant. Is that really a priority we should be focusing on? Are the number of people knowledgeable around scons really greater than what we have now? But I could be wrong, which leads me to #2... What would you recommend we focus on instead then? I would recommend going for robustness, rather than new features. I know of some areas for potential improvement: Array bounds checking, especially during input processing. Memory allocation checking. Debug profile corruption. Ideally, new contributors wouldn't need to be knowledgeable about scons. The build should be easier to perform, hopefully just "./configure" followed by "scons" (and scons even implements features that can subsume ./configure too). Already, scons doesn't need the "source winenv.set.sh" and "cd instsetoo_native" steps to build its modules. 2. "The conversion from gbuild to scons would largely be automated, fast and correct." If that is the case, let's test that theory right now. This has been possible for some time. In the scons-build branch, you can do the following: $ cd gotoSCons $ mvn package $ java -cp target/gotoSCons-0.1-SNAPSHOT.jar org.apache.openoffice.gotoSCons.GBuildConverter parsingAnalysis ../main (per-module output) Could parse: [MathMLDTD, UnoControls, animations, cosv, cppcanvas, drawinglayer, eventattacher, fileaccess, i18nutil, idl, io, rdbmaker, registry, remotebridges, sane, store, svgio, twain, ucbhelper, unixODBC, xmlreader, xmlscript] 22 out of 105 gbuild modules are parseable That means 22 modules can already be converted, completely and correctly. As we add more features to the converter (AllLangResTarget, UnoApi, Junit, GoogleTest, etc.), that 22 will increase. The per-module gbuild files are easy to parse. Parsing the syntax takes only 3 methods and < 100 lines of Java. The non-deterministic ones with "ifeq ($(GUIBASE),aqua)" require some manual work, but even there, a lot can be automated. There is some more work involved in semantic conversion: understanding and converting specific gbuild commands, converting paths to scons format, etc. but even so, we're on just 1913 lines of code in total for the converter. The hard part is to convert gbuild functions in main/solenv/gbuild into scons, for example, the worst case scenario is AllLangResTarget, for which this monstrous dependency tree needs to be implemented, with 4 layers of intermediate ta
Re: about build environment development
Hi Jim, dmake is not simple. I do know nothing Jim Jagielski... Today the build system enjoys me with checking whether the found dmake is the right dmake... configure: WARNING: no or configure: error: no URL for dmake source code specified, either. Use --with-dmake-url to supply an URL; run configure with the --help option for a list of possible URLs. ./configure.sh: Zeile 22: --with-dmake-url=https://sourceforge.net/projects/oooextras.mirror/files/dmake-4.12.tar.bz2: Datei oder Verzeichnis nicht gefunden but: wget https://sourceforge.net/projects/oooextras.mirror/files/dmake-4.12.tar.bz2 dmake-4.12.tar.bz2 100%[==>] 490,31K 92,2KB/s in 5,3s I am frustrated and will continue, when I had a break. Maybe then I figure out what stupidity I do not see. All the Best Peter Am 23.04.20 um 15:48 schrieb Jim Jagielski: IMO, dmake is simply a build-and-compile dependency. Considering that we pull in lots of other external dependencies, keeping or dropping dmake will likely not make any real difference in the people we can attract to help develop, test and build AOO. At the very least, we know what dmake is good at, and what it is NOT good at; so I support porting as much as possible to gmake, and just using dmake as the last step. And, FWIW, I am keeping dmake up-to-date and even FreeBSD are using my repo now: https://github.com/jimjag/dmake On Apr 17, 2020, at 8:53 AM, Peter Kovacs wrote: The goal is to move away from dmake. I do struggle with your point of view, saying moving away from dmake is unimportant, and should be stopped. I disagree with this position. Am 16.04.20 um 23:04 schrieb Patricia Shanahan: On 4/15/2020 10:08 AM, Damjan Jovanovic wrote: On Wed, Apr 15, 2020 at 3:15 PM Jim Jagielski wrote: On Apr 15, 2020, at 3:01 AM, Damjan Jovanovic wrote: We are also thin on new contributors, and I recall you saying they're largely scared off by the current build system. Two points: 1. I doubt that by the time we finish porting to a whole new build system, we will even have anyone *wanting* to contribute. With each delay and push-out on releases, and more time spent working on the build system instead of AOO itself, we become less and less relevant. Is that really a priority we should be focusing on? Are the number of people knowledgeable around scons really greater than what we have now? But I could be wrong, which leads me to #2... What would you recommend we focus on instead then? I would recommend going for robustness, rather than new features. I know of some areas for potential improvement: Array bounds checking, especially during input processing. Memory allocation checking. Debug profile corruption. Ideally, new contributors wouldn't need to be knowledgeable about scons. The build should be easier to perform, hopefully just "./configure" followed by "scons" (and scons even implements features that can subsume ./configure too). Already, scons doesn't need the "source winenv.set.sh" and "cd instsetoo_native" steps to build its modules. 2. "The conversion from gbuild to scons would largely be automated, fast and correct." If that is the case, let's test that theory right now. This has been possible for some time. In the scons-build branch, you can do the following: $ cd gotoSCons $ mvn package $ java -cp target/gotoSCons-0.1-SNAPSHOT.jar org.apache.openoffice.gotoSCons.GBuildConverter parsingAnalysis ../main (per-module output) Could parse: [MathMLDTD, UnoControls, animations, cosv, cppcanvas, drawinglayer, eventattacher, fileaccess, i18nutil, idl, io, rdbmaker, registry, remotebridges, sane, store, svgio, twain, ucbhelper, unixODBC, xmlreader, xmlscript] 22 out of 105 gbuild modules are parseable That means 22 modules can already be converted, completely and correctly. As we add more features to the converter (AllLangResTarget, UnoApi, Junit, GoogleTest, etc.), that 22 will increase. The per-module gbuild files are easy to parse. Parsing the syntax takes only 3 methods and < 100 lines of Java. The non-deterministic ones with "ifeq ($(GUIBASE),aqua)" require some manual work, but even there, a lot can be automated. There is some more work involved in semantic conversion: understanding and converting specific gbuild commands, converting paths to scons format, etc. but even so, we're on just 1913 lines of code in total for the converter. The hard part is to convert gbuild functions in main/solenv/gbuild into scons, for example, the worst case scenario is AllLangResTarget, for which this monstrous dependency tree needs to be implemented, with 4 layers of intermediate targets and complex actions with side effects and header dependencies (not shown): # AllLangResTarget(name) # (meta-target; delivers an empty timestamp file
Re: about build environment development
IMO, dmake is simply a build-and-compile dependency. Considering that we pull in lots of other external dependencies, keeping or dropping dmake will likely not make any real difference in the people we can attract to help develop, test and build AOO. At the very least, we know what dmake is good at, and what it is NOT good at; so I support porting as much as possible to gmake, and just using dmake as the last step. And, FWIW, I am keeping dmake up-to-date and even FreeBSD are using my repo now: https://github.com/jimjag/dmake > On Apr 17, 2020, at 8:53 AM, Peter Kovacs wrote: > > The goal is to move away from dmake. I do struggle with your point of view, > saying moving away from dmake is unimportant, and should be stopped. > > I disagree with this position. > > > Am 16.04.20 um 23:04 schrieb Patricia Shanahan: >> >> >> On 4/15/2020 10:08 AM, Damjan Jovanovic wrote: >>> On Wed, Apr 15, 2020 at 3:15 PM Jim Jagielski wrote: >>> > On Apr 15, 2020, at 3:01 AM, Damjan Jovanovic wrote: > > > We are also thin on new contributors, and I recall you saying they're > largely scared off by the current build system. > Two points: 1. I doubt that by the time we finish porting to a whole new build system, we will even have anyone *wanting* to contribute. With each delay and push-out on releases, and more time spent working on the build system instead of AOO itself, we become less and less relevant. Is that really a priority we should be focusing on? Are the number of people knowledgeable around scons really greater than what we have now? But I could be wrong, which leads me to #2... >>> What would you recommend we focus on instead then? >> >> I would recommend going for robustness, rather than new features. I know of >> some areas for potential improvement: >> >> Array bounds checking, especially during input processing. >> >> Memory allocation checking. >> >> Debug profile corruption. >> >>> >>> Ideally, new contributors wouldn't need to be knowledgeable about scons. >>> The build should be easier to perform, hopefully just "./configure" >>> followed by "scons" (and scons even implements features that can subsume >>> ./configure too). Already, scons doesn't need the "source winenv.set.sh" >>> and "cd instsetoo_native" steps to build its modules. >>> >>> 2. "The conversion from gbuild to scons would largely be automated, fast and correct." If that is the case, let's test that theory right now. >>> >>> This has been possible for some time. In the scons-build branch, you can do >>> the following: >>> >>> $ cd gotoSCons >>> $ mvn package >>> $ java -cp target/gotoSCons-0.1-SNAPSHOT.jar >>> org.apache.openoffice.gotoSCons.GBuildConverter parsingAnalysis ../main >>> (per-module output) >>> Could parse: [MathMLDTD, UnoControls, animations, cosv, cppcanvas, >>> drawinglayer, eventattacher, fileaccess, i18nutil, idl, io, rdbmaker, >>> registry, remotebridges, sane, store, svgio, twain, ucbhelper, unixODBC, >>> xmlreader, xmlscript] >>> 22 out of 105 gbuild modules are parseable >>> >>> That means 22 modules can already be converted, completely and correctly. >>> As we add more features to the converter (AllLangResTarget, UnoApi, Junit, >>> GoogleTest, etc.), that 22 will increase. >>> >>> The per-module gbuild files are easy to parse. Parsing the syntax takes >>> only 3 methods and < 100 lines of Java. The non-deterministic ones with >>> "ifeq ($(GUIBASE),aqua)" require some manual work, but even there, a lot >>> can be automated. There is some more work involved in semantic conversion: >>> understanding and converting specific gbuild commands, converting paths to >>> scons format, etc. but even so, we're on just 1913 lines of code in total >>> for the converter. >>> >>> The hard part is to convert gbuild functions in main/solenv/gbuild into >>> scons, for example, the worst case scenario is AllLangResTarget, for which >>> this monstrous dependency tree needs to be implemented, with 4 layers of >>> intermediate targets and complex actions with side effects and header >>> dependencies (not shown): >>> >>> # AllLangResTarget(name) >>> # (meta-target; delivers an empty timestamp file) >>> #^ ^ >>> # / \ >>> # / \ >>> # / \ >>> # ResTarget(nameen-US,name,en-US) >>> ResTarget(nameen-GB,name,en-GB) >>> # $(WORKDIR)/ResTarget/$(resName).res >>> $(WORKDIR)/ResTarget/$(resName).res >>> # $(WORKDIR)/ResTarget/nameen-US.res >>> $(WORKDIR)/ResTarget/nameen-GB.res >>> #^ ^^ >>> #| rsc || >>> #| || >>> # SrsTarget SrsTarget >>> SrsTarge
Re: about build environment development
The goal is to move away from dmake. I do struggle with your point of view, saying moving away from dmake is unimportant, and should be stopped. I disagree with this position. Am 16.04.20 um 23:04 schrieb Patricia Shanahan: On 4/15/2020 10:08 AM, Damjan Jovanovic wrote: On Wed, Apr 15, 2020 at 3:15 PM Jim Jagielski wrote: On Apr 15, 2020, at 3:01 AM, Damjan Jovanovic wrote: We are also thin on new contributors, and I recall you saying they're largely scared off by the current build system. Two points: 1. I doubt that by the time we finish porting to a whole new build system, we will even have anyone *wanting* to contribute. With each delay and push-out on releases, and more time spent working on the build system instead of AOO itself, we become less and less relevant. Is that really a priority we should be focusing on? Are the number of people knowledgeable around scons really greater than what we have now? But I could be wrong, which leads me to #2... What would you recommend we focus on instead then? I would recommend going for robustness, rather than new features. I know of some areas for potential improvement: Array bounds checking, especially during input processing. Memory allocation checking. Debug profile corruption. Ideally, new contributors wouldn't need to be knowledgeable about scons. The build should be easier to perform, hopefully just "./configure" followed by "scons" (and scons even implements features that can subsume ./configure too). Already, scons doesn't need the "source winenv.set.sh" and "cd instsetoo_native" steps to build its modules. 2. "The conversion from gbuild to scons would largely be automated, fast and correct." If that is the case, let's test that theory right now. This has been possible for some time. In the scons-build branch, you can do the following: $ cd gotoSCons $ mvn package $ java -cp target/gotoSCons-0.1-SNAPSHOT.jar org.apache.openoffice.gotoSCons.GBuildConverter parsingAnalysis ../main (per-module output) Could parse: [MathMLDTD, UnoControls, animations, cosv, cppcanvas, drawinglayer, eventattacher, fileaccess, i18nutil, idl, io, rdbmaker, registry, remotebridges, sane, store, svgio, twain, ucbhelper, unixODBC, xmlreader, xmlscript] 22 out of 105 gbuild modules are parseable That means 22 modules can already be converted, completely and correctly. As we add more features to the converter (AllLangResTarget, UnoApi, Junit, GoogleTest, etc.), that 22 will increase. The per-module gbuild files are easy to parse. Parsing the syntax takes only 3 methods and < 100 lines of Java. The non-deterministic ones with "ifeq ($(GUIBASE),aqua)" require some manual work, but even there, a lot can be automated. There is some more work involved in semantic conversion: understanding and converting specific gbuild commands, converting paths to scons format, etc. but even so, we're on just 1913 lines of code in total for the converter. The hard part is to convert gbuild functions in main/solenv/gbuild into scons, for example, the worst case scenario is AllLangResTarget, for which this monstrous dependency tree needs to be implemented, with 4 layers of intermediate targets and complex actions with side effects and header dependencies (not shown): # AllLangResTarget(name) # (meta-target; delivers an empty timestamp file) # ^ ^ # / \ # / \ # / \ # ResTarget(nameen-US,name,en-US) ResTarget(nameen-GB,name,en-GB) # $(WORKDIR)/ResTarget/$(resName).res $(WORKDIR)/ResTarget/$(resName).res # $(WORKDIR)/ResTarget/nameen-US.res $(WORKDIR)/ResTarget/nameen-GB.res # ^ ^ ^ # | rsc | | # | | | # SrsTarget SrsTarget SrsTarget # $(WORKDIR)/SrsTarget/$(srsName).srs # $(WORKDIR)/SrsTarget/uui/res.srs # ^ # | concatenate # +--+ # | \ # | \ # SrcPartTarget SrcPartTarget # $(WORKDIR)/SrsPartTarget/$(srsPartName) # $(WORKDIR)/SrsPartTarget/uui/source/ids.src # ^ ^ # | rsc | rsc # | | # (when not translating) | | (when translating) # | | # | SrcPartMergeTarget # | $(WORKDIR)/SrsPartMergeTarget/$(1) # | $(WORKDIR)/SrsPartMergeTarget/uui/source/ids.src # | ^ # | / # | / transex3 # | / (when
Re: about build environment development
On 4/15/2020 10:08 AM, Damjan Jovanovic wrote: On Wed, Apr 15, 2020 at 3:15 PM Jim Jagielski wrote: On Apr 15, 2020, at 3:01 AM, Damjan Jovanovic wrote: We are also thin on new contributors, and I recall you saying they're largely scared off by the current build system. Two points: 1. I doubt that by the time we finish porting to a whole new build system, we will even have anyone *wanting* to contribute. With each delay and push-out on releases, and more time spent working on the build system instead of AOO itself, we become less and less relevant. Is that really a priority we should be focusing on? Are the number of people knowledgeable around scons really greater than what we have now? But I could be wrong, which leads me to #2... What would you recommend we focus on instead then? I would recommend going for robustness, rather than new features. I know of some areas for potential improvement: Array bounds checking, especially during input processing. Memory allocation checking. Debug profile corruption. Ideally, new contributors wouldn't need to be knowledgeable about scons. The build should be easier to perform, hopefully just "./configure" followed by "scons" (and scons even implements features that can subsume ./configure too). Already, scons doesn't need the "source winenv.set.sh" and "cd instsetoo_native" steps to build its modules. 2. "The conversion from gbuild to scons would largely be automated, fast and correct." If that is the case, let's test that theory right now. This has been possible for some time. In the scons-build branch, you can do the following: $ cd gotoSCons $ mvn package $ java -cp target/gotoSCons-0.1-SNAPSHOT.jar org.apache.openoffice.gotoSCons.GBuildConverter parsingAnalysis ../main (per-module output) Could parse: [MathMLDTD, UnoControls, animations, cosv, cppcanvas, drawinglayer, eventattacher, fileaccess, i18nutil, idl, io, rdbmaker, registry, remotebridges, sane, store, svgio, twain, ucbhelper, unixODBC, xmlreader, xmlscript] 22 out of 105 gbuild modules are parseable That means 22 modules can already be converted, completely and correctly. As we add more features to the converter (AllLangResTarget, UnoApi, Junit, GoogleTest, etc.), that 22 will increase. The per-module gbuild files are easy to parse. Parsing the syntax takes only 3 methods and < 100 lines of Java. The non-deterministic ones with "ifeq ($(GUIBASE),aqua)" require some manual work, but even there, a lot can be automated. There is some more work involved in semantic conversion: understanding and converting specific gbuild commands, converting paths to scons format, etc. but even so, we're on just 1913 lines of code in total for the converter. The hard part is to convert gbuild functions in main/solenv/gbuild into scons, for example, the worst case scenario is AllLangResTarget, for which this monstrous dependency tree needs to be implemented, with 4 layers of intermediate targets and complex actions with side effects and header dependencies (not shown): # AllLangResTarget(name) # (meta-target; delivers an empty timestamp file) #^ ^ # / \ # / \ # / \ # ResTarget(nameen-US,name,en-US) ResTarget(nameen-GB,name,en-GB) # $(WORKDIR)/ResTarget/$(resName).res $(WORKDIR)/ResTarget/$(resName).res # $(WORKDIR)/ResTarget/nameen-US.res $(WORKDIR)/ResTarget/nameen-GB.res #^ ^^ #| rsc || #| || # SrsTarget SrsTarget SrsTarget # $(WORKDIR)/SrsTarget/$(srsName).srs # $(WORKDIR)/SrsTarget/uui/res.srs #^ #| concatenate #+--+ #| \ #|\ # SrcPartTarget SrcPartTarget # $(WORKDIR)/SrsPartTarget/$(srsPartName) # $(WORKDIR)/SrsPartTarget/uui/source/ids.src #^ ^ #| rsc | rsc #| | # (when not translating) | | (when translating) #| | #|SrcPartMergeTarget #|$(WORKDIR)/SrsPartMergeTarget/$(1) #| $(WORKDIR)/SrsPartMergeTarget/uui/source/ids.src #| ^ #|/ #| / transex3 #| / (when translating) # $(srsPartName) / # uuid/source/ids.src # D
Re: about build environment development
On 4/14/20 9:46 PM, Peter Kovacs wrote: If one wants to tap in our build system he needs to understand Perl, shell, make, ant, XML, configure, ... This is just way to complicated, especially if you want to bring in an IDE to ease code development. Damjan is not very happy with the features gmake offers. I am not sure where exactly the Issue is. He is opting for SCONs, with the option to extend the build system with python. And IMHO on Damjan Side he is quite serious about it. Everyone else has not expressed any opinion on this development, so I am not sure everyone is aware. The last discussion on this topic, consent has been strongly to make gmake work. Another objection is that we got some heavy negative experience report from the serf community about SCONS. Which are switching from, SCONS to cmake. HA! I guess I missed this. I was wondering if cmake might be acceptable option, and I guess it is! :) On Carl's original question. There WERE dependency specs in dmake. Since I'm not a gbuild guru, I couldn't figure how this was handled in gbuild. So in the end we do not have Consent where we want to go. And currently it is heavily influenced by Damjan. And this is imho very thin. I am still like the Idea most to go in the direction of ant / maven, despite its flaws. But I am not negative on SCONS either. My main point is we need something that offers a better architecture and is easier to handled and maintained. Also what we could try is making use of something like portage. Portage is pretty easy to use repostory manager used by gentoo, whioch also had a community prior to homebrew on mac. It is not very difficult to setup. But it is build to make different build system work together. So we could have a build repository, that builds our dependencies. We reconstruct our monolith in smaller build libraries, like UNOcore, OOFrame, UNOGUI, OOapp, OOpython, StarBasic, OOwizards, extentionXYZ (Just saying something), and pick the best build system (cmake, gmake, ant, maven, SCONS) for each library. Also we could think on incubating Starbasic or UNO, as own Project if they become more interesting. Since Portage is made for source build, but can also handle binaries, maybe we could add some features that will make it easy to export towards specific distributions, making it easy for distributors to export to their system. BTW, portage is build on python, so it should work on all systems we target. Sorry if this Idea is to crazy for you. It is only an idea. Maybe it is a good time now to bring this topic up in everyones mind. All the Best Peter Am 15.04.20 um 01:14 schrieb Carl Marcum: On 4/14/20 5:53 PM, Kay Schenk wrote: On 4/14/20 1:46 PM, Carl Marcum wrote: On 4/14/20 3:57 PM, Peter Kovacs wrote: You could try to build only the module, by going into the folder and execute make directly. Hi Peter, Yes but that doesn't solve my problem with targets not running in order or how I can enforce it if possible. I don't want to break the build if my change ever makes it to trunk. Eventually if I can get Ant to build the Jar exactly as gbuild does I can use that one and my problem goes away. But until then I was wanting to use the current one that gbuild builds. Thanks, Carl Hi Carl -- From your first post in this thread... When I build with "build --all" everything works as expected. When I build with "build --all -P2 -- -P2" the file copy fails because the juh.jar file isn't completed. I recall having issues with the second part -- -P2. You might try omitting that, and just use "build --all -P2" or maybe "build --all -P4" ref... https://wiki.openoffice.org/wiki/Documentation/Building_Guide_AOO#Building_2 The build will take a longer but you shouldn't run into the "non-completion" issue you're having. Hope this helps, Kay Hi Kay, Thanks for pointing out what the second parameter meant :) It would still be good to know if it's possible to declare dependencies between targets in gbuild somehow like you can with Ant builds. I'm guessing any final solution that gets into trunk has to build with multiple threads per module. My best option is probably to do the jar build along with the other tasks in Ant so I can control when it happens. Were already using Ant to build java jars in ridljar, jurt, officebean, and probably other modules. I didn't mention it but I'm working on creating the necessary javadoc, source, and library jars for distribution through Apache Nexus [1]. But during the build process to avoid the need for a separate Vote next time around. [1] https://wiki.openoffice.org/wiki/Uno/Java/MavenBundles Thanks again, Carl - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org -
Re: about build environment development (was: declaring gbuild target dependencies)
On Wed, Apr 15, 2020 at 3:15 PM Jim Jagielski wrote: > > > > On Apr 15, 2020, at 3:01 AM, Damjan Jovanovic wrote: > > > > > > We are also thin on new contributors, and I recall you saying they're > > largely scared off by the current build system. > > > > Two points: > > 1. I doubt that by the time we finish porting to a whole new build > system, we will even have anyone *wanting* to contribute. With each delay > and push-out on releases, and more time spent working on the build system > instead of AOO itself, we become less and less relevant. Is that really a > priority we should be focusing on? Are the number of people knowledgeable > around scons really greater than what we have now? But I could be wrong, > which leads me to #2... > > What would you recommend we focus on instead then? Ideally, new contributors wouldn't need to be knowledgeable about scons. The build should be easier to perform, hopefully just "./configure" followed by "scons" (and scons even implements features that can subsume ./configure too). Already, scons doesn't need the "source winenv.set.sh" and "cd instsetoo_native" steps to build its modules. > 2. "The conversion from gbuild to scons would largely be automated, fast > and correct." If that is the case, let's test that theory right now. > This has been possible for some time. In the scons-build branch, you can do the following: $ cd gotoSCons $ mvn package $ java -cp target/gotoSCons-0.1-SNAPSHOT.jar org.apache.openoffice.gotoSCons.GBuildConverter parsingAnalysis ../main (per-module output) Could parse: [MathMLDTD, UnoControls, animations, cosv, cppcanvas, drawinglayer, eventattacher, fileaccess, i18nutil, idl, io, rdbmaker, registry, remotebridges, sane, store, svgio, twain, ucbhelper, unixODBC, xmlreader, xmlscript] 22 out of 105 gbuild modules are parseable That means 22 modules can already be converted, completely and correctly. As we add more features to the converter (AllLangResTarget, UnoApi, Junit, GoogleTest, etc.), that 22 will increase. The per-module gbuild files are easy to parse. Parsing the syntax takes only 3 methods and < 100 lines of Java. The non-deterministic ones with "ifeq ($(GUIBASE),aqua)" require some manual work, but even there, a lot can be automated. There is some more work involved in semantic conversion: understanding and converting specific gbuild commands, converting paths to scons format, etc. but even so, we're on just 1913 lines of code in total for the converter. The hard part is to convert gbuild functions in main/solenv/gbuild into scons, for example, the worst case scenario is AllLangResTarget, for which this monstrous dependency tree needs to be implemented, with 4 layers of intermediate targets and complex actions with side effects and header dependencies (not shown): # AllLangResTarget(name) # (meta-target; delivers an empty timestamp file) #^ ^ # / \ # / \ # / \ # ResTarget(nameen-US,name,en-US) ResTarget(nameen-GB,name,en-GB) # $(WORKDIR)/ResTarget/$(resName).res $(WORKDIR)/ResTarget/$(resName).res # $(WORKDIR)/ResTarget/nameen-US.res $(WORKDIR)/ResTarget/nameen-GB.res #^ ^^ #| rsc || #| || # SrsTarget SrsTarget SrsTarget # $(WORKDIR)/SrsTarget/$(srsName).srs # $(WORKDIR)/SrsTarget/uui/res.srs #^ #| concatenate #+--+ #| \ #|\ # SrcPartTarget SrcPartTarget # $(WORKDIR)/SrsPartTarget/$(srsPartName) # $(WORKDIR)/SrsPartTarget/uui/source/ids.src #^ ^ #| rsc | rsc #| | # (when not translating) | | (when translating) #| | #|SrcPartMergeTarget #|$(WORKDIR)/SrsPartMergeTarget/$(1) #| $(WORKDIR)/SrsPartMergeTarget/uui/source/ids.src #| ^ #|/ #| / transex3 #| / (when translating) # $(srsPartName) / # uuid/source/ids.src # Damjan
Re: about build environment development (was: declaring gbuild target dependencies)
> On Apr 15, 2020, at 3:01 AM, Damjan Jovanovic wrote: > > > We are also thin on new contributors, and I recall you saying they're > largely scared off by the current build system. > Two points: 1. I doubt that by the time we finish porting to a whole new build system, we will even have anyone *wanting* to contribute. With each delay and push-out on releases, and more time spent working on the build system instead of AOO itself, we become less and less relevant. Is that really a priority we should be focusing on? Are the number of people knowledgeable around scons really greater than what we have now? But I could be wrong, which leads me to #2... 2. "The conversion from gbuild to scons would largely be automated, fast and correct." If that is the case, let's test that theory right now. - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: about build environment development
Hi All, Damjan, thanks for providing the feature list on SCONs. I did not have that on the able. For me it is a promising Idea with a lot of questions attached to it. And I am not 100% convinced it can solve the Issues where we got stuck on Gmake. I would also like to know which arguments speak for using something we already have? That is not clear to me Patricia. I am sorry. But I believe except Damjan no one has bought in to SCONs completely. For me there are to many questions open and ant is still in the race. It does not have the great build in c++ support, but it is easier to build in in IDEs and we can sign our stuff at least on windows with it. Probably on Mac too. It has also great support to be extended, and it has as SCONS a dependency support. I think we have to think on the following: # Build bots ## support our own ## support for Azure (they have mac server free for OpenSource Projects!) # Maven development support for 3rd party contributors # Various different Platform and distribution support. ## Distribution we might need distribution specific patch support. # support of building our own dependencies. ## automated Signing support, for our targets. maybe # easy to add other architectures. (I know that we might have an Issue in UNO here in general) In the end all we need is Plan. Maybe we could combine SCONS and ANT through Jython -Ant as Frontend and SCONS as workhorse? I do not know. But we should deliver answers to our needs. And doing PoCs is good. All the Best Peter Am 15.04.20 um 09:01 schrieb Damjan Jovanovic: On Wed, Apr 15, 2020 at 6:46 AM Peter Kovacs wrote: If one wants to tap in our build system he needs to understand Perl, shell, make, ant, XML, configure, ... This is just way to complicated, especially if you want to bring in an IDE to ease code development. Damjan is not very happy with the features gmake offers. I am not sure where exactly the Issue is. He is opting for SCONs, with the option to extend the build system with python. And IMHO on Damjan Side he is quite serious about it. Everyone else has not expressed any opinion on this development, so I am not sure everyone is aware. The last discussion on this topic, consent has been strongly to make gmake work. We already took that approach to its limit, and I don't believe we can go much further. Most of gmake works by luck. The simplest things break, make no sense, and cannot be debugged, eg. sometimes *_add_files breaks, but *_add_file works, despite the fact the former just calls the latter. There are already some hacks in modules to work around gmake's brokenness... Another objection is that we got some heavy negative experience report from the serf community about SCONS. It wasn't the entire "serf community", just brane@ on 30 Oct 2019: ---snip--- As a far-too-long-time user of Scons (in Serf), let me just add a caution: Scons is very, very broken and not at all well maintained. Writing a truly cross-platform, cross-toolchain SConstruct file for anything other than really trivial cases amounts to writing a *lot* of platform-specific Python code to make the builds work. If you're already moving to gmake (without autotools, and I expect using Cygwin on Windows), then I suggest you finish the transition, then leave well enough alone. ---snip--- My experience so far has been exactly the opposite: it is far easier to get scons building cross-platform, than GNU make + countless shell tools which also drag in the whole of Cygwin. To summarize quickly: I tried to avoid scons before and use gmake, and we already got as far as we could. scons is not a decision I made lightly, it's a decision I made because it's that good. Python isn't my favorite language, but what we gain from scons is significant enough for me to want to learn more Python. By using scons, Cygwin left the building without me even trying. scons has a 20 year history, supports OS/2 and numerous other platforms, supports Python 2 and 3, supports more MSVC versions than we do (including Visual Studio 2019), would allow us to build almost anywhere. It is packed full of many advanced features we need, like symlinking libX.so -> libX.so.2, automated header dependency scanning, flex, yacc, Java, Objective C, precompiled headers on MSVC, fully parallelizes builds across module boundaries, can work out minimal rebuilds using checksums of file contents instead of inode timestamps, is extensible by design, has readable source code, can be debugged, and is under a liberal license (MIT). I've looked, and nothing else really comes close; every other build tool would require considerable further development (AOO has already tried building big build systems around both dmake and GNU make; my better experience with Ant/Maven makes me more hopeful about a richer higher-level build tool that automates more of the work internally). The conversion from gbuild to scons would largely be automated, fast and cor
Re: about build environment development (was: declaring gbuild target dependencies)
On Wed, Apr 15, 2020 at 6:46 AM Peter Kovacs wrote: > If one wants to tap in our build system he needs to understand Perl, > shell, make, ant, XML, configure, ... > > This is just way to complicated, especially if you want to bring in an > IDE to ease code development. > > > Damjan is not very happy with the features gmake offers. I am not sure > where exactly the Issue is. > > He is opting for SCONs, with the option to extend the build system with > python. And IMHO on Damjan > > Side he is quite serious about it. > > > Everyone else has not expressed any opinion on this development, so I am > not sure everyone is aware. The last discussion on this topic, > > consent has been strongly to make gmake work. > We already took that approach to its limit, and I don't believe we can go much further. Most of gmake works by luck. The simplest things break, make no sense, and cannot be debugged, eg. sometimes *_add_files breaks, but *_add_file works, despite the fact the former just calls the latter. There are already some hacks in modules to work around gmake's brokenness... > > Another objection is that we got some heavy negative experience report > from the serf community about SCONS. > > It wasn't the entire "serf community", just brane@ on 30 Oct 2019: ---snip--- As a far-too-long-time user of Scons (in Serf), let me just add a caution: Scons is very, very broken and not at all well maintained. Writing a truly cross-platform, cross-toolchain SConstruct file for anything other than really trivial cases amounts to writing a *lot* of platform-specific Python code to make the builds work. If you're already moving to gmake (without autotools, and I expect using Cygwin on Windows), then I suggest you finish the transition, then leave well enough alone. ---snip--- My experience so far has been exactly the opposite: it is far easier to get scons building cross-platform, than GNU make + countless shell tools which also drag in the whole of Cygwin. To summarize quickly: I tried to avoid scons before and use gmake, and we already got as far as we could. scons is not a decision I made lightly, it's a decision I made because it's that good. Python isn't my favorite language, but what we gain from scons is significant enough for me to want to learn more Python. By using scons, Cygwin left the building without me even trying. scons has a 20 year history, supports OS/2 and numerous other platforms, supports Python 2 and 3, supports more MSVC versions than we do (including Visual Studio 2019), would allow us to build almost anywhere. It is packed full of many advanced features we need, like symlinking libX.so -> libX.so.2, automated header dependency scanning, flex, yacc, Java, Objective C, precompiled headers on MSVC, fully parallelizes builds across module boundaries, can work out minimal rebuilds using checksums of file contents instead of inode timestamps, is extensible by design, has readable source code, can be debugged, and is under a liberal license (MIT). I've looked, and nothing else really comes close; every other build tool would require considerable further development (AOO has already tried building big build systems around both dmake and GNU make; my better experience with Ant/Maven makes me more hopeful about a richer higher-level build tool that automates more of the work internally). The conversion from gbuild to scons would largely be automated, fast and correct. (We could also keep the scons files in a format that would allow easier automated conversion to something else later.) > Which are switching from, SCONS to cmake. > > > So in the end we do not have Consent where we want to go. And currently > it is heavily influenced by Damjan. And this is imho very thin. > > Then we were always very thin on gbuild too. You've also done some scons development. We are also thin on new contributors, and I recall you saying they're largely scared off by the current build system. > I am still like the Idea most to go in the direction of ant / maven, > despite its flaws. But I am not negative on SCONS either. My main point > is we need something that > > offers a better architecture and is easier to handled and maintained. > > > Also what we could try is making use of something like portage. Portage > is pretty easy to use repostory manager used by gentoo, whioch also had > a community prior to homebrew on mac. It is not very difficult to > setup. But it is build to make different build system work together. So > we could have a build repository, that builds our dependencies. We > reconstruct our monolith in smaller build libraries, like UNOcore, > OOFrame, UNOGUI, OOapp, OOpython, StarBasic, OOwizards, extentionXYZ > (Just saying something), and pick the best build system (cmake, gmake, > ant, maven, SCONS) for each library. Also we could think on incubating > Starbasic or UNO, as own Project if they become more interesting. Since > Portage is made for source build, but can also handle binaries, maybe we
Re: about build environment development
Without knowing enough about the merits of the different build systems we got where we are through a history in which a new build system is selected every N years, but it takes more than N years to fully switch. My recommendation is to pick one of the existing build systems, and get everything on to it. On 4/14/2020 9:46 PM, Peter Kovacs wrote: If one wants to tap in our build system he needs to understand Perl, shell, make, ant, XML, configure, ... This is just way to complicated, especially if you want to bring in an IDE to ease code development. Damjan is not very happy with the features gmake offers. I am not sure where exactly the Issue is. He is opting for SCONs, with the option to extend the build system with python. And IMHO on Damjan Side he is quite serious about it. Everyone else has not expressed any opinion on this development, so I am not sure everyone is aware. The last discussion on this topic, consent has been strongly to make gmake work. Another objection is that we got some heavy negative experience report from the serf community about SCONS. Which are switching from, SCONS to cmake. So in the end we do not have Consent where we want to go. And currently it is heavily influenced by Damjan. And this is imho very thin. I am still like the Idea most to go in the direction of ant / maven, despite its flaws. But I am not negative on SCONS either. My main point is we need something that offers a better architecture and is easier to handled and maintained. Also what we could try is making use of something like portage. Portage is pretty easy to use repostory manager used by gentoo, whioch also had a community prior to homebrew on mac. It is not very difficult to setup. But it is build to make different build system work together. So we could have a build repository, that builds our dependencies. We reconstruct our monolith in smaller build libraries, like UNOcore, OOFrame, UNOGUI, OOapp, OOpython, StarBasic, OOwizards, extentionXYZ (Just saying something), and pick the best build system (cmake, gmake, ant, maven, SCONS) for each library. Also we could think on incubating Starbasic or UNO, as own Project if they become more interesting. Since Portage is made for source build, but can also handle binaries, maybe we could add some features that will make it easy to export towards specific distributions, making it easy for distributors to export to their system. BTW, portage is build on python, so it should work on all systems we target. Sorry if this Idea is to crazy for you. It is only an idea. Maybe it is a good time now to bring this topic up in everyones mind. All the Best Peter Am 15.04.20 um 01:14 schrieb Carl Marcum: On 4/14/20 5:53 PM, Kay Schenk wrote: On 4/14/20 1:46 PM, Carl Marcum wrote: On 4/14/20 3:57 PM, Peter Kovacs wrote: You could try to build only the module, by going into the folder and execute make directly. Hi Peter, Yes but that doesn't solve my problem with targets not running in order or how I can enforce it if possible. I don't want to break the build if my change ever makes it to trunk. Eventually if I can get Ant to build the Jar exactly as gbuild does I can use that one and my problem goes away. But until then I was wanting to use the current one that gbuild builds. Thanks, Carl Hi Carl -- From your first post in this thread... When I build with "build --all" everything works as expected. When I build with "build --all -P2 -- -P2" the file copy fails because the juh.jar file isn't completed. I recall having issues with the second part -- -P2. You might try omitting that, and just use "build --all -P2" or maybe "build --all -P4" ref... https://wiki.openoffice.org/wiki/Documentation/Building_Guide_AOO#Building_2 The build will take a longer but you shouldn't run into the "non-completion" issue you're having. Hope this helps, Kay Hi Kay, Thanks for pointing out what the second parameter meant :) It would still be good to know if it's possible to declare dependencies between targets in gbuild somehow like you can with Ant builds. I'm guessing any final solution that gets into trunk has to build with multiple threads per module. My best option is probably to do the jar build along with the other tasks in Ant so I can control when it happens. Were already using Ant to build java jars in ridljar, jurt, officebean, and probably other modules. I didn't mention it but I'm working on creating the necessary javadoc, source, and library jars for distribution through Apache Nexus [1]. But during the build process to avoid the need for a separate Vote next time around. [1] https://wiki.openoffice.org/wiki/Uno/Java/MavenBundles Thanks again, Carl - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org --
about build environment development (was: declaring gbuild target dependencies)
If one wants to tap in our build system he needs to understand Perl, shell, make, ant, XML, configure, ... This is just way to complicated, especially if you want to bring in an IDE to ease code development. Damjan is not very happy with the features gmake offers. I am not sure where exactly the Issue is. He is opting for SCONs, with the option to extend the build system with python. And IMHO on Damjan Side he is quite serious about it. Everyone else has not expressed any opinion on this development, so I am not sure everyone is aware. The last discussion on this topic, consent has been strongly to make gmake work. Another objection is that we got some heavy negative experience report from the serf community about SCONS. Which are switching from, SCONS to cmake. So in the end we do not have Consent where we want to go. And currently it is heavily influenced by Damjan. And this is imho very thin. I am still like the Idea most to go in the direction of ant / maven, despite its flaws. But I am not negative on SCONS either. My main point is we need something that offers a better architecture and is easier to handled and maintained. Also what we could try is making use of something like portage. Portage is pretty easy to use repostory manager used by gentoo, whioch also had a community prior to homebrew on mac. It is not very difficult to setup. But it is build to make different build system work together. So we could have a build repository, that builds our dependencies. We reconstruct our monolith in smaller build libraries, like UNOcore, OOFrame, UNOGUI, OOapp, OOpython, StarBasic, OOwizards, extentionXYZ (Just saying something), and pick the best build system (cmake, gmake, ant, maven, SCONS) for each library. Also we could think on incubating Starbasic or UNO, as own Project if they become more interesting. Since Portage is made for source build, but can also handle binaries, maybe we could add some features that will make it easy to export towards specific distributions, making it easy for distributors to export to their system. BTW, portage is build on python, so it should work on all systems we target. Sorry if this Idea is to crazy for you. It is only an idea. Maybe it is a good time now to bring this topic up in everyones mind. All the Best Peter Am 15.04.20 um 01:14 schrieb Carl Marcum: On 4/14/20 5:53 PM, Kay Schenk wrote: On 4/14/20 1:46 PM, Carl Marcum wrote: On 4/14/20 3:57 PM, Peter Kovacs wrote: You could try to build only the module, by going into the folder and execute make directly. Hi Peter, Yes but that doesn't solve my problem with targets not running in order or how I can enforce it if possible. I don't want to break the build if my change ever makes it to trunk. Eventually if I can get Ant to build the Jar exactly as gbuild does I can use that one and my problem goes away. But until then I was wanting to use the current one that gbuild builds. Thanks, Carl Hi Carl -- From your first post in this thread... When I build with "build --all" everything works as expected. When I build with "build --all -P2 -- -P2" the file copy fails because the juh.jar file isn't completed. I recall having issues with the second part -- -P2. You might try omitting that, and just use "build --all -P2" or maybe "build --all -P4" ref... https://wiki.openoffice.org/wiki/Documentation/Building_Guide_AOO#Building_2 The build will take a longer but you shouldn't run into the "non-completion" issue you're having. Hope this helps, Kay Hi Kay, Thanks for pointing out what the second parameter meant :) It would still be good to know if it's possible to declare dependencies between targets in gbuild somehow like you can with Ant builds. I'm guessing any final solution that gets into trunk has to build with multiple threads per module. My best option is probably to do the jar build along with the other tasks in Ant so I can control when it happens. Were already using Ant to build java jars in ridljar, jurt, officebean, and probably other modules. I didn't mention it but I'm working on creating the necessary javadoc, source, and library jars for distribution through Apache Nexus [1]. But during the build process to avoid the need for a separate Vote next time around. [1] https://wiki.openoffice.org/wiki/Uno/Java/MavenBundles Thanks again, Carl - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
