Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On 10/27/11 8:28 PM, Pedro Giffuni wrote: --- On Thu, 10/27/11, Jürgen Schmidtjogischm...@googlemail.com wrote: In any case, yes.. I think this is the way to go. I am just hoping there will be a way to opt out those components in favor of the system libraries when those available. me too but we should move forward and we can change it at any time when we have a better solution. I am OK with that, but let me attempt to dump what I think: 1) you are not bringing in *anything* copyleft, that directory will only be for the non-restrictive stuff that we need: ICU, Boost, etc. 2) This will all have to be registered in the NOTICE file, but since this is transitory and not really stuff we use in base, we should start a new section there to separate it from the stuff we do use in the core system. 3) We should probably move some of the stuff in soltools there too (mkdepend). 4) I know you want ucpp there too, but since that stuff is used in idlc, I think I'd prefer it in idlc/source/preproc/ as it was before. No idea if we can use the system cpp for the rest but that would probably make sense. mmh, i would prefer to put it under the ext-sources to make clear that it comes from external. Juergen All just IMHO, I am pretty sure whatever you do is better than what we have now :). Pedro.
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
--- On Fri, 10/28/11, Jürgen Schmidt jogischm...@googlemail.com wrote: snip mental dump 4) I know you want ucpp there too, but since that stuff is used in idlc, I think I'd prefer it in idlc/source/preproc/ as it was before. No idea if we can use the system cpp for the rest but that would probably make sense. mmh, i would prefer to put it under the ext-sources to make clear that it comes from external. That is pretty well covered by SVN and the NOTICE file, but I was only brainstorming. Just have fun :). Pedro.
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On 9/22/11 1:19 PM, Jürgen Schmidt wrote: ok, we have several arguments for and against but no decision how we want to move forward. Let us take again a look on it 1. we have a working mechanism to get the externals from somewhere, check md5 sum, unpack, patch, build 1.1 somewhere is configurable during the configure step, initially the externals are downloaded from http://hg.services.openoffice.org/binaries 2. having the externals in the repository (SVN) won't be a big issue because in case of a checkout always the tip version is downloaded 2.1 the SCM can be used to track the used version of the externals for a specific OO version - simply checkout the version tag and everything is in place ... 3. in a DSCM it would be a real problem over time because of the increasing space of all versions 4. we need a replacement http://hg.services.openoffice.org/binaries asap (who knows how long the server will be available) 5. many developers probably work with a local clone of the repository using for example git svn or something else - disadvantage of the increasing space but probably acceptable if a clean local trunk will be kept and updated Proposed way to move forward 1. put the externals under .../trunk/ext_sources .../trunk/ext_sources .../trunk/main .../trunk/extras 2. adapt configure to use this as default, disable the download (maybe reactivate it later if we move to a DSCM) 3. keep the process with checking the md5 sum as it is (for potential later use) Any opinions or suggestions? i think we still haven't finished on this topic but it is somewhat important to move forward with our IP clearance and the whole development work. So if nobody has real objections i would like to move forward with this proposal but would also like to change the proposed directory name from ext_sources to 3rdparty. Keep in mind that we use this directory to keep the current state working and with our ongoing work we will remove more and more stuff from there. The adapted bootstrap mechanism will download the libraries from this new place. Juergen
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
2011/10/27 Jürgen Schmidt jogischm...@googlemail.com: On 9/22/11 1:19 PM, Jürgen Schmidt wrote: ok, we have several arguments for and against but no decision how we want to move forward. Let us take again a look on it 1. we have a working mechanism to get the externals from somewhere, check md5 sum, unpack, patch, build 1.1 somewhere is configurable during the configure step, initially the externals are downloaded from http://hg.services.openoffice.org/binaries 2. having the externals in the repository (SVN) won't be a big issue because in case of a checkout always the tip version is downloaded 2.1 the SCM can be used to track the used version of the externals for a specific OO version - simply checkout the version tag and everything is in place ... 3. in a DSCM it would be a real problem over time because of the increasing space of all versions 4. we need a replacement http://hg.services.openoffice.org/binaries asap (who knows how long the server will be available) 5. many developers probably work with a local clone of the repository using for example git svn or something else - disadvantage of the increasing space but probably acceptable if a clean local trunk will be kept and updated Proposed way to move forward 1. put the externals under .../trunk/ext_sources .../trunk/ext_sources .../trunk/main .../trunk/extras 2. adapt configure to use this as default, disable the download (maybe reactivate it later if we move to a DSCM) 3. keep the process with checking the md5 sum as it is (for potential later use) Any opinions or suggestions? i think we still haven't finished on this topic but it is somewhat important to move forward with our IP clearance and the whole development work. So if nobody has real objections i would like to move forward with this proposal but would also like to change the proposed directory name from ext_sources to 3rdparty. Keep in mind that we use this directory to keep the current state working and with our ongoing work we will remove more and more stuff from there. So keep the current approach with tarballs with MD5 hashnames, etc., just as before but on Apache servers? That sounds good to me. The adapted bootstrap mechanism will download the libraries from this new place. Juergen
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
--- On Thu, 10/27/11, Jürgen Schmidt jogischm...@googlemail.com wrote: ... i think we still haven't finished on this topic but it is somewhat important to move forward with our IP clearance and the whole development work. So if nobody has real objections i would like to move forward with this proposal but would also like to change the proposed directory name from ext_sources to 3rdparty. Keep in mind that we use this directory to keep the current state working and with our ongoing work we will remove more and more stuff from there. I was about to bring in support for FreeBSD's fetch command (somewhat like curl) in fetch-tarballs.sh and it looks like now you are obsoleting it :-P . In any case, yes.. I think this is the way to go. I am just hoping there will be a way to opt out those components in favor of the system libraries when those available. Pedro.
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On 10/27/11 6:13 PM, Pedro Giffuni wrote: --- On Thu, 10/27/11, Jürgen Schmidtjogischm...@googlemail.com wrote: ... i think we still haven't finished on this topic but it is somewhat important to move forward with our IP clearance and the whole development work. So if nobody has real objections i would like to move forward with this proposal but would also like to change the proposed directory name from ext_sources to 3rdparty. Keep in mind that we use this directory to keep the current state working and with our ongoing work we will remove more and more stuff from there. I was about to bring in support for FreeBSD's fetch command (somewhat like curl) in fetch-tarballs.sh and it looks like now you are obsoleting it :-P . In any case, yes.. I think this is the way to go. I am just hoping there will be a way to opt out those components in favor of the system libraries when those available. me too but we should move forward and we can change it at any time when we have a better solution. Juergen Pedro.
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
--- On Thu, 10/27/11, Jürgen Schmidt jogischm...@googlemail.com wrote: In any case, yes.. I think this is the way to go. I am just hoping there will be a way to opt out those components in favor of the system libraries when those available. me too but we should move forward and we can change it at any time when we have a better solution. I am OK with that, but let me attempt to dump what I think: 1) you are not bringing in *anything* copyleft, that directory will only be for the non-restrictive stuff that we need: ICU, Boost, etc. 2) This will all have to be registered in the NOTICE file, but since this is transitory and not really stuff we use in base, we should start a new section there to separate it from the stuff we do use in the core system. 3) We should probably move some of the stuff in soltools there too (mkdepend). 4) I know you want ucpp there too, but since that stuff is used in idlc, I think I'd prefer it in idlc/source/preproc/ as it was before. No idea if we can use the system cpp for the rest but that would probably make sense. All just IMHO, I am pretty sure whatever you do is better than what we have now :). Pedro.
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On Thu, Oct 27, 2011 at 2:28 PM, Pedro Giffuni p...@apache.org wrote: --- On Thu, 10/27/11, Jürgen Schmidt jogischm...@googlemail.com wrote: In any case, yes.. I think this is the way to go. I am just hoping there will be a way to opt out those components in favor of the system libraries when those available. me too but we should move forward and we can change it at any time when we have a better solution. I am OK with that, but let me attempt to dump what I think: 1) you are not bringing in *anything* copyleft, that directory will only be for the non-restrictive stuff that we need: ICU, Boost, etc. I think it is like the SVN trunk. We initially bring it all in, and then remove the copyleft parts. Of course if we can remove them before hand, that is good as well. But whatever order we do the work, we cannot release until we've done the IP review. The files are currently hosted here: http://hg.services.openoffice.org/binaries/ Since the build currently depends on that, I think we want to move those files now, to Apache, rather than wait too long. -Rob 2) This will all have to be registered in the NOTICE file, but since this is transitory and not really stuff we use in base, we should start a new section there to separate it from the stuff we do use in the core system. 3) We should probably move some of the stuff in soltools there too (mkdepend). 4) I know you want ucpp there too, but since that stuff is used in idlc, I think I'd prefer it in idlc/source/preproc/ as it was before. No idea if we can use the system cpp for the rest but that would probably make sense. All just IMHO, I am pretty sure whatever you do is better than what we have now :). Pedro.
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
Hi Matthias; --- On Thu, 10/27/11, Mathias Bauer mathias_ba...@gmx.net wrote: ... In any case, yes.. I think this is the way to go. I am just hoping there will be a way to opt out those I am OK with that, but let me attempt to dump what I think: 1) you are not bringing in *anything* copyleft, that directory will only be for the non-restrictive stuff that we need: ICU, Boost, etc. That should be doable. OTOH I'm wondering whether we should keep the copyleft tarballs at Apache Extras - it would allow to still build with them (something that can be done outside the ASF infrastructure and is still appreciated (if I understood correctly). I don't like that but we will have to do it as a temporary solution to avoid breaking the build until we replace everything. I think on the long run this is only interesting for windows binaries, due to the difficulties of getting those packages from different places. On linux/BSD distributions it makes sense to use the prepackaged mozilla, etc. 3) We should probably move some of the stuff in soltools there too (mkdepend). That's something for later, ATM we should move the ext_src stuff into a secure place. Yes. Also for later, the simpleICC library is used to generate a color profile required for pdf. I think we should just generate the color profile somewhere outside the main build and use it, avoiding the extra build cycles. Another thing is we are excluding by default with extreme prejudice both LGPL and MPL but it will be convenient to reevaluate that since we will have to use the prepackaged hunspell. If nobody else wants to do it, I can invest some time into that, but it might take some days. I won't do it because of principles... I want them to just go away ;-). FWIW, Rob and I are trying to use an ooo- prefix on Apache Extras. ooo-external-sources ? It seems that the consensus is that we check in the binary tarballs into trunk/ext_sources?! I am not sure on that, I think lazy consensus by whomever does it first will win :). Pedro.
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
Am 01.10.2011 00:17, schrieb Michael Stahl: On 30.09.2011 21:24, Mathias Bauer wrote: On 28.09.2011 17:32, Pedro F. Giffuni wrote: Another advantage of unpacking the tarballs: the patches will become *real* patches that just contain changes of the original source code. Often the patches nowadays contain additional files that we just need to build the stuff in OOo (e.g. dmake makefiles) - they could be checked in as regular files. Currently keeping them as regular files is awkward because then they need to be copied to the place the tarballs are unpacked to. but this is just because dmake can only build source files in the same directory; imagine a more flexible gbuild external build target where the makefiles are in the source tree while the tarball gets unpacked in the workdir... Sure, but until we aren't there... I didn't talk about the dmake makefiles that are used to unpack and patch, I was talking about using dmake for building the external modules that come with their own build system. The makefile.mk in the root directory of the external modules are not part of the patch, but some patches contain makefile.mk files that are necessary to build the stuff, either on all or only on some platforms. Regards, Mathias
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On 28.09.2011 17:32, Pedro F. Giffuni wrote: FWIW; I don't like the patches because I can't really examine well the code, besides this is something the VCS handles acceptably: commit the original sourcecode and then apply the patches in a different commit. If we start with up to date versions there would not be much trouble. I'm not against unpacking the tarballs and applying the patches, but we should keep the patches somewhere so that updates could be done with the same effort as today. Another advantage of unpacking the tarballs: the patches will become *real* patches that just contain changes of the original source code. Often the patches nowadays contain additional files that we just need to build the stuff in OOo (e.g. dmake makefiles) - they could be checked in as regular files. Currently keeping them as regular files is awkward because then they need to be copied to the place the tarballs are unpacked to. Regards, Mathias
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On 30.09.2011 21:24, Mathias Bauer wrote: On 28.09.2011 17:32, Pedro F. Giffuni wrote: Another advantage of unpacking the tarballs: the patches will become *real* patches that just contain changes of the original source code. Often the patches nowadays contain additional files that we just need to build the stuff in OOo (e.g. dmake makefiles) - they could be checked in as regular files. Currently keeping them as regular files is awkward because then they need to be copied to the place the tarballs are unpacked to. but this is just because dmake can only build source files in the same directory; imagine a more flexible gbuild external build target where the makefiles are in the source tree while the tarball gets unpacked in the workdir... Regards, Mathias
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On 20.09.2011 16:36, Pavel Janík wrote: Have we ever considered using version control to...uh...manage file versions? Just an idea. Maybe Heiner will say more, but in the past, we have had the external tarballs in the VCS, but then we moved them out and it worked very well. There never was a reason to track external.tar.gz files in VCS, because we do not change them. What might be the best way to handle 3rd party code in AOOo probably will depend on the needs of the developers as well as on legal requirements. We had these tarballs plus patches IIRC because Sun Legal required that all used 3rd party stuff should be preserved in our repos in its original form. As a developer I always had preferred to have 3rd party code treated in the *build* like the internal source code. So if there wasn't a requirement to have unpatched sources in the repository, the most natural way to keep 3rd party stuff would be to have a third sub-repo 3rdparty next to main and extras with the 3rd party stuff checked in. Not the tarballs, just the unpacked content. I wouldn't give up the patches, as they allow to handle updates better. This would cause a problem, as direct changes to the 3rd party stuff without additional authorization (means: changing the source code must not happen accidently, only when the 3rd party code gets an update from upstream) must be prevented, while still patch files must be allowed to added, removed, or changed, not the original source code. If that wasn't possible or too cumbersome, checking in the tarballs in 3rdparty would be better. As svn users never download the complete history as DSCM users do, the pain of binary files in the repo isn't that hard. In case AOOo moved to a DSCM again later, the tarballs could be moved out again easily. Regards, Mathias
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
FWIW; I don't like the patches because I can't really examine well the code, besides this is something the VCS handles acceptably: commit the original sourcecode and then apply the patches in a different commit. If we start with up to date versions there would not be much trouble. just my $0.02, not an objection. Pedro. --- On Wed, 9/28/11, Jürgen Schmidt jogischm...@googlemail.com wrote: ... I wouldn't give up the patches, as they allow to handle updates better. This would cause a problem, as direct changes to the 3rd party stuff without additional authorization (means: changing the source code must not happen accidently, only when the 3rd party code gets an update from upstream) must be prevented, while still patch files must be allowed to added, removed, or changed, not the original source code. If that wasn't possible or too cumbersome, checking in the tarballs in 3rdparty would be better. i also wouldn't give up the patches and for that reason i would like to move forward for now with keeping the tarballs as proposed. But i like the name 3rdparty for the directory and we can later on change it from the tarballs to the unpacked code it we see demand for it. At the moment it's just easier to keep the tarballs and focus on other work. As svn users never download the complete history as DSCM users do, the pain of binary files in the repo isn't that hard. In case AOOo moved to a DSCM again later, the tarballs could be moved out again easily. agree, we don't really loose anything, can change if necessary and can continue with our work Juergen
RE: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
The problem with bringing the 3rd party software completely into the SVN tree and modifying it in the tree has to do with the license the updated software is under. In that case, there *is* a code provenance issue and I believe it crosses a line that the Apache Software Foundation is unwilling to cross with regard to the integrity of its code bases. The current patches to Boost, for example, do not change the license on the code and preserve the Boost license. But since this is ephemeral and the source is never in the SVN tree (is that correct?) the derivative use disappears at the end of a build. It is sufficient then to include the dependency in the NOTICE for the release and not worry further. Also, the current dependency is several releases behind the current Boost release. This might not matter - the specific Boost libraries that are used might not be effected. But there is a release synchronization issue. A fork would have to be maintained. Also, the dependencies are managed better now, rather than having the entire Boost library installed for cherry picking. (This will all change at some point, since Boost is being incorporated into ISO C++. It is probably best to wait for that to ripple out into the compiler distributions.) - Dennis -Original Message- From: Pedro F. Giffuni [mailto:giffu...@tutopia.com] Sent: Wednesday, September 28, 2011 08:32 To: ooo-dev@incubator.apache.org Subject: Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?] FWIW; I don't like the patches because I can't really examine well the code, besides this is something the VCS handles acceptably: commit the original sourcecode and then apply the patches in a different commit. If we start with up to date versions there would not be much trouble. just my $0.02, not an objection. Pedro. --- On Wed, 9/28/11, Jürgen Schmidt jogischm...@googlemail.com wrote: ... I wouldn't give up the patches, as they allow to handle updates better. This would cause a problem, as direct changes to the 3rd party stuff without additional authorization (means: changing the source code must not happen accidently, only when the 3rd party code gets an update from upstream) must be prevented, while still patch files must be allowed to added, removed, or changed, not the original source code. If that wasn't possible or too cumbersome, checking in the tarballs in 3rdparty would be better. i also wouldn't give up the patches and for that reason i would like to move forward for now with keeping the tarballs as proposed. But i like the name 3rdparty for the directory and we can later on change it from the tarballs to the unpacked code it we see demand for it. At the moment it's just easier to keep the tarballs and focus on other work. As svn users never download the complete history as DSCM users do, the pain of binary files in the repo isn't that hard. In case AOOo moved to a DSCM again later, the tarballs could be moved out again easily. agree, we don't really loose anything, can change if necessary and can continue with our work Juergen
RE: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
The idea (not originally mine) is to have keep only compatible licensed code under an isolated (3rdparty) directory. I think on the long run we should try to use the system versions of such software when available, and every linux/bsd distribution is probably doing that for LO already. Pedro. --- On Wed, 9/28/11, Dennis E. Hamilton dennis.hamil...@acm.org wrote: The problem with bringing the 3rd party software completely into the SVN tree and modifying it in the tree has to do with the license the updated software is under. In that case, there *is* a code provenance issue and I believe it crosses a line that the Apache Software Foundation is unwilling to cross with regard to the integrity of its code bases. The current patches to Boost, for example, do not change the license on the code and preserve the Boost license. But since this is ephemeral and the source is never in the SVN tree (is that correct?) the derivative use disappears at the end of a build. It is sufficient then to include the dependency in the NOTICE for the release and not worry further. Also, the current dependency is several releases behind the current Boost release. This might not matter - the specific Boost libraries that are used might not be effected. But there is a release synchronization issue. A fork would have to be maintained. Also, the dependencies are managed better now, rather than having the entire Boost library installed for cherry picking. (This will all change at some point, since Boost is being incorporated into ISO C++. It is probably best to wait for that to ripple out into the compiler distributions.) - Dennis -Original Message- From: Pedro F. Giffuni [mailto:giffu...@tutopia.com] Sent: Wednesday, September 28, 2011 08:32 To: ooo-dev@incubator.apache.org Subject: Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?] FWIW; I don't like the patches because I can't really examine well the code, besides this is something the VCS handles acceptably: commit the original sourcecode and then apply the patches in a different commit. If we start with up to date versions there would not be much trouble. just my $0.02, not an objection. Pedro. --- On Wed, 9/28/11, Jürgen Schmidt jogischm...@googlemail.com wrote: ... I wouldn't give up the patches, as they allow to handle updates better. This would cause a problem, as direct changes to the 3rd party stuff without additional authorization (means: changing the source code must not happen accidently, only when the 3rd party code gets an update from upstream) must be prevented, while still patch files must be allowed to added, removed, or changed, not the original source code. If that wasn't possible or too cumbersome, checking in the tarballs in 3rdparty would be better. i also wouldn't give up the patches and for that reason i would like to move forward for now with keeping the tarballs as proposed. But i like the name 3rdparty for the directory and we can later on change it from the tarballs to the unpacked code it we see demand for it. At the moment it's just easier to keep the tarballs and focus on other work. As svn users never download the complete history as DSCM users do, the pain of binary files in the repo isn't that hard. In case AOOo moved to a DSCM again later, the tarballs could be moved out again easily. agree, we don't really loose anything, can change if necessary and can continue with our work Juergen
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On 28.09.2011 17:32, Pedro F. Giffuni wrote: FWIW; I don't like the patches because I can't really examine well the code, besides this is something the VCS handles acceptably: commit the original sourcecode and then apply the patches in a different commit. If we start with up to date versions there would not be much trouble. if we didn't have many thousands of lines of patches to rebase, then upgrading to less outdated versions wouldn't be such a PITA. sadly in many cases upstreaming patches was never sufficiently high on the priority list to actually get done... -- Dealing with failure is easy: Work hard to improve. Success is also easy to handle: You've solved the wrong problem. Work hard to improve. -- Alan Perlis
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On Thu, Sep 22, 2011 at 12:40 AM, Jens-Heiner Rechtien jhrecht...@web.dewrote: On 09/20/2011 05:26 PM, Rob Weir wrote: Ai2011/9/20 Pavel Janíkpa...@janik.cz: Have we ever considered using version control to...uh...manage file versions? Just an idea. Maybe Heiner will say more, but in the past, we have had the external tarballs in the VCS, but then we moved them out and it worked very well. There never was a reason to track external.tar.gz files in VCS, because we do not change them. -- That's fine. If they don't change, then doing a svn update will not bring them down each time. Aside from being useful for version control, SVN is useful also very useful as an audit trail. So in the rare occasions when one of these files does change, we know who changed it and why. This is important for ensuring the IP cleanliness of the project. Is your main concern performance? Even as individual tarballs, ext-sources is 86 files, 250MB. ooo/extras is 243 files and 822 MB. And ooo/main is 76,295 files for over 900MB. So ext-sources is not a huge contributor to download time. Placing all the external tarballs in the VCS is a real killer if using a distributed SCM like git or Mercurial, thats why we had moved them out. As Pavel said, it worked quite nice. As for the audit possibility, we referenced the external tar balls in the source tree by file name and a md5 check sum, which works just as reliantly as putting them directly into the repository. Nowadays the DSCM have some alternative methods which deal with such blobs but in essence they also keep them separate. If AOOo ever plans to go back to a DSCM I would keep the source tree and the external blobs strictly separated. All in all the general SCM tooling community opinion trend seems to be that a S(ource)CM system is for, well, source and external dependencies are better handled with other mechanism, like Maven or so. With SVN all this is less of a concern, naturally. ok, we have several arguments for and against but no decision how we want to move forward. Let us take again a look on it 1. we have a working mechanism to get the externals from somewhere, check md5 sum, unpack, patch, build 1.1 somewhere is configurable during the configure step, initially the externals are downloaded from http://hg.services.openoffice.org/binaries 2. having the externals in the repository (SVN) won't be a big issue because in case of a checkout always the tip version is downloaded 2.1 the SCM can be used to track the used version of the externals for a specific OO version - simply checkout the version tag and everything is in place ... 3. in a DSCM it would be a real problem over time because of the increasing space of all versions 4. we need a replacement http://hg.services.openoffice.org/binaries asap (who knows how long the server will be available) 5. many developers probably work with a local clone of the repository using for example git svn or something else - disadvantage of the increasing space but probably acceptable if a clean local trunk will be kept and updated Proposed way to move forward 1. put the externals under .../trunk/ext_sources .../trunk/ext_sources .../trunk/main .../trunk/extras 2. adapt configure to use this as default, disable the download (maybe reactivate it later if we move to a DSCM) 3. keep the process with checking the md5 sum as it is (for potential later use) Any opinions or suggestions? Juergen
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
Proposed way to move forward 1. put the externals under .../trunk/ext_sources .../trunk/ext_sources .../trunk/main .../trunk/extras 2. adapt configure to use this as default, disable the download (maybe reactivate it later if we move to a DSCM) 3. keep the process with checking the md5 sum as it is (for potential later use) Any opinions or suggestions? +1. And one more question: If we put something into SVN into .../trunk/ext_sources, do we have some URL that can replace http://hg so users don't have to check out everything? Ie. do we have a URL where we have real checkout of the SVN? Some SVN web interface? Don't know Apache infra well yet... That would be real killer solution! -- Pavel Janík
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On 22.09.2011 13:19, Jürgen Schmidt wrote: On Thu, Sep 22, 2011 at 12:40 AM, Jens-Heiner Rechtienjhrecht...@web.dewrote: On 09/20/2011 05:26 PM, Rob Weir wrote: ... Placing all the external tarballs in the VCS is a real killer if using a distributed SCM like git or Mercurial, thats why we had moved them out. As Pavel said, it worked quite nice. As for the audit possibility, we referenced the external tar balls in the source tree by file name and a md5 check sum, which works just as reliantly as putting them directly into the repository. Nowadays the DSCM have some alternative methods which deal with such blobs but in essence they also keep them separate. If AOOo ever plans to go back to a DSCM I would keep the source tree and the external blobs strictly separated. All in all the general SCM tooling community opinion trend seems to be that a S(ource)CM system is for, well, source and external dependencies are better handled with other mechanism, like Maven or so. With SVN all this is less of a concern, naturally. ok, we have several arguments for and against but no decision how we want to move forward. Let us take again a look on it 1. we have a working mechanism to get the externals from somewhere, check md5 sum, unpack, patch, build 1.1 somewhere is configurable during the configure step, initially the externals are downloaded from http://hg.services.openoffice.org/binaries 2. having the externals in the repository (SVN) won't be a big issue because in case of a checkout always the tip version is downloaded 2.1 the SCM can be used to track the used version of the externals for a specific OO version - simply checkout the version tag and everything is in place ... 3. in a DSCM it would be a real problem over time because of the increasing space of all versions 4. we need a replacement http://hg.services.openoffice.org/binaries asap (who knows how long the server will be available) 5. many developers probably work with a local clone of the repository using for example git svn or something else - disadvantage of the increasing space but probably acceptable if a clean local trunk will be kept and updated Proposed way to move forward 1. put the externals under .../trunk/ext_sources .../trunk/ext_sources .../trunk/main .../trunk/extras 2. adapt configure to use this as default, disable the download (maybe reactivate it later if we move to a DSCM) 3. keep the process with checking the md5 sum as it is (for potential later use) Any opinions or suggestions? +1 Best current solution: Added to SVN where it does not really matter, and a way to get back when we may change to a DSCM in the future. Juergen sincerely, Armin -- ALG
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
2011/9/22 Pavel Janík pa...@janik.cz Proposed way to move forward 1. put the externals under .../trunk/ext_sources .../trunk/ext_sources .../trunk/main .../trunk/extras 2. adapt configure to use this as default, disable the download (maybe reactivate it later if we move to a DSCM) 3. keep the process with checking the md5 sum as it is (for potential later use) Any opinions or suggestions? +1. And one more question: If we put something into SVN into .../trunk/ext_sources, do we have some URL that can replace http://hg so users don't have to check out everything? Ie. do we have a URL where we have real checkout of the SVN? Some SVN web interface? Don't know Apache infra well yet... That would be real killer solution! don't know if it is what you are looking for but wget http://svn.apache.org/viewvc/incubator/ooo/trunk/main/ filename?view=co should download the head version. Juergen -- Pavel Janík
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
don't know if it is what you are looking for but wget http://svn.apache.org/viewvc/incubator/ooo/trunk/main/ filename?view=co should download the head version. Then we should be able to have both things solved - files in SVN and with a relatively small change in the download script also the remote fetching of the files if we do not have ext_sources local checkout. -- Pavel Janík
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
2011/9/22 Pavel Janík pa...@janik.cz: Proposed way to move forward 1. put the externals under .../trunk/ext_sources .../trunk/ext_sources .../trunk/main .../trunk/extras 2. adapt configure to use this as default, disable the download (maybe reactivate it later if we move to a DSCM) 3. keep the process with checking the md5 sum as it is (for potential later use) Any opinions or suggestions? +1. And one more question: If we put something into SVN into .../trunk/ext_sources, do we have some URL that can replace http://hg so users don't have to check out everything? Ie. do we have a URL where we have real checkout of the SVN? Some SVN web interface? Don't know Apache infra well yet... That would be real killer solution! -- I was thinking something similar. We only need to use the SVN interface to the files when we're adding or updating. But we can have bootstrap continue to download via http. The location, using Juergen's proposed location, would be http://svn.apache.org/repos/asf/incubator/ooo/trunk/ext-sources This would save having a duplicate local SVN working copy of the file, right? -Rob
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On Thu, Sep 22, 2011 at 2:23 PM, Rob Weir robw...@apache.org wrote: I was thinking something similar. We only need to use the SVN interface to the files when we're adding or updating. But we can have bootstrap continue to download via http. The location, using Juergen's proposed location, would be http://svn.apache.org/repos/asf/incubator/ooo/trunk/ext-sources yes, this is the correct URL, the URL that i have posted wouldn't work Juergen This would save having a duplicate local SVN working copy of the file, right? -Rob
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
2011/9/22 Jürgen Schmidt jogischm...@googlemail.com On Thu, Sep 22, 2011 at 2:23 PM, Rob Weir robw...@apache.org wrote: I was thinking something similar. We only need to use the SVN interface to the files when we're adding or updating. But we can have bootstrap continue to download via http. The location, using Juergen's proposed location, would be http://svn.apache.org/repos/asf/incubator/ooo/trunk/ext-sources yes, this is the correct URL, the URL that i have posted wouldn't work Juergen This would save having a duplicate local SVN working copy of the file, right? mmh, no or i understand something wrong. People checkout .../trunk and would get ext_sources, main and extras. To benefit from the modified script we have to put ext_sources besides trunk .../ooo/ext_sources .../ooo/trunk/main .../ooo/trunk/extras Means back to my initial proposal, right? Juergen
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
2011/9/22 Jürgen Schmidt jogischm...@googlemail.com: 2011/9/22 Jürgen Schmidt jogischm...@googlemail.com On Thu, Sep 22, 2011 at 2:23 PM, Rob Weir robw...@apache.org wrote: I was thinking something similar. We only need to use the SVN interface to the files when we're adding or updating. But we can have bootstrap continue to download via http. The location, using Juergen's proposed location, would be http://svn.apache.org/repos/asf/incubator/ooo/trunk/ext-sources yes, this is the correct URL, the URL that i have posted wouldn't work Juergen This would save having a duplicate local SVN working copy of the file, right? mmh, no or i understand something wrong. People checkout .../trunk and would get ext_sources, main and extras. To benefit from the modified script we have to put ext_sources besides trunk .../ooo/ext_sources .../ooo/trunk/main .../ooo/trunk/extras Means back to my initial proposal, right? I think the idea is this: Everything under ooo represents what goes into a release. It can be tagged and branched. trunk/ is a peer to a tags/ and branches/ directory. It is possible that we have this wrong. Adding in site/ and ooo-site/ brings in a different convention. They have are set up to have trunk/tags/branches underneath them. That is fine, because the website does not release in synch with an OOo release. It makes sense for them to be able to tag and branch independently. We should also consider how the project grows going forward. We know that other code bases will be checked in, like Symphony. And there are other, small, but disjoint contributions that I'm working on as well. So it might make sense to move trunk down one level: /ooo/ooo-src/trunk/main /ooo/ooo-src/trunk/extras /ooo/ooo-src/trunk/ext-sources /ooo/ooo-src/tags /ooo/ooo-src/branches That would make more sense then, as a unit, since we would want to tag the across all of /ooo/ooo-src/ to define a release. I assume a developer still just checks out ooo/ooo-src/trunk/main. If they need the additional extras then they check that out separately. I don't think most users will want to check out the entire trunk all the time. We should consider also how we want this tree to grow over time, as other related In the end, I think we want to preserve the ability to: 1) Preserve an audit trail of all changes that went into a release 2) Do be able to tag and branch a release and everything that is in the release 3) Restore the exact state of a previous tagged release, including the exact ext-sources used in that release I'm certain that my proposal will enable this. There may be other approaches that do as well. Another thing to keep in mind is the SVN support for externals: http://svnbook.red-bean.com/en/1.0/ch07s03.html This might make some things easier. -Rob Juergen
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
hi, Based on this result, an other trunk will be like the following if IBM symphony checked in: /ooo/symphony-src/trunk/main /ooo/symphony-src/trunk/extras /ooo/symphony-src/tags /ooo/symphony-src/branches thus it introduces a problem: How to merge the two trunks of symphony-src and ooo-src? thanks mail:zhaos...@cn.ibm.com Address:2/F,Ring Bldg. No.28 Building, Zhong Guan Cun Software Park, No.8, Dong Bei Wang West Road, ShangDi, Haidian District, Beijing 100193, P.R.China Rob Weir robweir@apache.o rgTo ooo-dev@incubator.apache.org, 2011-09-22 21:18 cc Subject Please respond to Re: handling of ext_sources - ooo-dev@incubator Juergen's suggestion [was: Re: A .apache.orgsystematic approach to IP review?] 2011/9/22 Jürgen Schmidt jogischm...@googlemail.com: 2011/9/22 Jürgen Schmidt jogischm...@googlemail.com On Thu, Sep 22, 2011 at 2:23 PM, Rob Weir robw...@apache.org wrote: I was thinking something similar. We only need to use the SVN interface to the files when we're adding or updating. But we can have bootstrap continue to download via http. The location, using Juergen's proposed location, would be http://svn.apache.org/repos/asf/incubator/ooo/trunk/ext-sources yes, this is the correct URL, the URL that i have posted wouldn't work Juergen This would save having a duplicate local SVN working copy of the file, right? mmh, no or i understand something wrong. People checkout .../trunk and would get ext_sources, main and extras. To benefit from the modified script we have to put ext_sources besides trunk .../ooo/ext_sources .../ooo/trunk/main .../ooo/trunk/extras Means back to my initial proposal, right? I think the idea is this: Everything under ooo represents what goes into a release. It can be tagged and branched. trunk/ is a peer to a tags/ and branches/ directory. It is possible that we have this wrong. Adding in site/ and ooo-site/ brings in a different convention. They have are set up to have trunk/tags/branches underneath them. That is fine, because the website does not release in synch with an OOo release. It makes sense for them to be able to tag and branch independently. We should also consider how the project grows going forward. We know that other code bases will be checked in, like Symphony. And there are other, small, but disjoint contributions that I'm working on as well. So it might make sense to move trunk down one level: /ooo/ooo-src/trunk/main /ooo/ooo-src/trunk/extras /ooo/ooo-src/trunk/ext-sources /ooo/ooo-src/tags /ooo/ooo-src/branches That would make more sense then, as a unit, since we would want to tag the across all of /ooo/ooo-src/ to define a release. I assume a developer still just checks out ooo/ooo-src/trunk/main. If they need the additional extras then they check that out separately. I don't think most users will want to check out the entire trunk all the time. We should consider also how we want this tree to grow over time, as other related In the end, I think we want to preserve the ability to: 1) Preserve an audit trail of all changes that went into a release 2) Do be able to tag and branch a release and everything that is in the release 3) Restore the exact state of a previous tagged release, including the exact ext-sources used in that release I'm certain that my proposal will enable this. There may be other approaches that do as well. Another thing to keep in mind is the SVN support for externals: http://svnbook.red-bean.com/en/1.0/ch07s03.html This might make some things easier. -Rob Juergen
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On Thu, Sep 22, 2011 at 3:18 PM, Rob Weir robw...@apache.org wrote: It is possible that we have this wrong. Adding in site/ and ooo-site/ brings in a different convention. They have are set up to have trunk/tags/branches underneath them. That is fine, because the website does not release in synch with an OOo release. It makes sense for them to be able to tag and branch independently. agree We should also consider how the project grows going forward. We know that other code bases will be checked in, like Symphony. And there are other, small, but disjoint contributions that I'm working on as well. So it might make sense to move trunk down one level: /ooo/ooo-src/trunk/main /ooo/ooo-src/trunk/extras /ooo/ooo-src/trunk/ext-sources /ooo/ooo-src/tags /ooo/ooo-src/branches That would make more sense then, as a unit, since we would want to tag the across all of /ooo/ooo-src/ to define a release. agree, from this perspective it make sense. The question then is when we want to introduce this further level? I assume a developer still just checks out ooo/ooo-src/trunk/main. If they need the additional extras then they check that out separately. I don't think most users will want to check out the entire trunk all the time. We should consider also how we want this tree to grow over time, as other related i assumed that a developer will check out trunk, maybe a wrong assumption In the end, I think we want to preserve the ability to: 1) Preserve an audit trail of all changes that went into a release 2) Do be able to tag and branch a release and everything that is in the release 3) Restore the exact state of a previous tagged release, including the exact ext-sources used in that release I'm certain that my proposal will enable this. There may be other approaches that do as well. i think so too. And with my changed mindset to not always check out trunk completely i am fine with this approach. Another thing to keep in mind is the SVN support for externals: http://svnbook.red-bean.com/en/1.0/ch07s03.html interesting, i didn't know that before Juergen
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On Thu, Sep 22, 2011 at 9:40 AM, Shao Zhi Zhao zhaos...@cn.ibm.com wrote: hi, Based on this result, an other trunk will be like the following if IBM symphony checked in: /ooo/symphony-src/trunk/main /ooo/symphony-src/trunk/extras /ooo/symphony-src/tags /ooo/symphony-src/branches thus it introduces a problem: How to merge the two trunks of symphony-src and ooo-src? I don't think moving the tree down one level introduces any new problems for Symphony, so long as the directories within */main. remain the same. Of course, merging code from Symphony into AOOo will be difficult in general. The problem is how do we establish a common ancestor revision to do a 3-way merge with? This will really depend on whether Symphony has a good record of what the corresponding OOo revision was for each of its initial files. If not, then you can do a text diff and do some merging without trouble. But dealing with renamed files, or moved files, or deleted files, these are trickier to process automatically. If you don't have that history, then in theory it could be reestablished by taking the initial revision of each file in Symphony and comparing it to each revision of the same file in OOo Mercurial, and find which revision matches. It might be possible to establish enough context for a 3-way merge that way. -Rob thanks mail:zhaos...@cn.ibm.com Address:2/F,Ring Bldg. No.28 Building, Zhong Guan Cun Software Park, No.8, Dong Bei Wang West Road, ShangDi, Haidian District, Beijing 100193, P.R.China [image: Inactive hide details for Rob Weir ---2011-09-22 21:20:44---Rob Weir robw...@apache.org]Rob Weir ---2011-09-22 21:20:44---Rob Weir robw...@apache.org *Rob Weir robw...@apache.org* 2011-09-22 21:18 Please respond to ooo-dev@incubator.apache.org To ooo-dev@incubator.apache.org, cc Subject Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?] 2011/9/22 Jürgen Schmidt jogischm...@googlemail.com: 2011/9/22 Jürgen Schmidt jogischm...@googlemail.com On Thu, Sep 22, 2011 at 2:23 PM, Rob Weir robw...@apache.org wrote: I was thinking something similar. We only need to use the SVN interface to the files when we're adding or updating. But we can have bootstrap continue to download via http. The location, using Juergen's proposed location, would be http://svn.apache.org/repos/asf/incubator/ooo/trunk/ext-sources yes, this is the correct URL, the URL that i have posted wouldn't work Juergen This would save having a duplicate local SVN working copy of the file, right? mmh, no or i understand something wrong. People checkout .../trunk and would get ext_sources, main and extras. To benefit from the modified script we have to put ext_sources besides trunk .../ooo/ext_sources .../ooo/trunk/main .../ooo/trunk/extras Means back to my initial proposal, right? I think the idea is this: Everything under ooo represents what goes into a release. It can be tagged and branched. trunk/ is a peer to a tags/ and branches/ directory. It is possible that we have this wrong. Adding in site/ and ooo-site/ brings in a different convention. They have are set up to have trunk/tags/branches underneath them. That is fine, because the website does not release in synch with an OOo release. It makes sense for them to be able to tag and branch independently. We should also consider how the project grows going forward. We know that other code bases will be checked in, like Symphony. And there are other, small, but disjoint contributions that I'm working on as well. So it might make sense to move trunk down one level: /ooo/ooo-src/trunk/main /ooo/ooo-src/trunk/extras /ooo/ooo-src/trunk/ext-sources /ooo/ooo-src/tags /ooo/ooo-src/branches That would make more sense then, as a unit, since we would want to tag the across all of /ooo/ooo-src/ to define a release. I assume a developer still just checks out ooo/ooo-src/trunk/main. If they need the additional extras then they check that out separately. I don't think most users will want to check out the entire trunk all the time. We should consider also how we want this tree to grow over time, as other related In the end, I think we want to preserve the ability to: 1) Preserve an audit trail of all changes that went into a release 2) Do be able to tag and branch a release and everything that is in the release 3) Restore the exact state of a previous tagged release, including the exact ext-sources used in that release I'm certain that my proposal will enable this. There may be other approaches that do as well. Another thing to keep in mind is the SVN support for externals: http://svnbook.red-bean.com/en/1.0/ch07s03.html This might make some things easier. -Rob Juergen
RE: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
You can get anything off of the web interface of SVN at the individual level without it being in a working copy, though of course it has to be somewhere local while it is being processed in a build. But if you check-out the trunk, you get everything that is in the trunk HEAD (or a specified) version. As far as I know, you can do a check-out anywhere deeper in the tree and avoid everything not at that node [and below]. For example, just checkout trunk/main. It takes some consideration of SVN organization to have the desired flavors in convenient chunks that people can work with without having to eat the whole thing (with regard to SVN checkout, SVN update and, of course, SVN commits). I can testify that an SVN UDPATE of the working copy of the entire incubator/ooo/ subtree is a painful experience, even when there is nothing to update. - Dennis PS: I find it an interesting characteristic of SVN that trunk, tags, and branches are just names of folders and don't mean anything special to SVN. The nomenclature and it is use is a matter of custom, like code indentation rules for ( ... }. -Original Message- From: Rob Weir [mailto:robw...@apache.org] Sent: Thursday, September 22, 2011 05:24 To: ooo-dev@incubator.apache.org Subject: Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?] 2011/9/22 Pavel Janík pa...@janik.cz: Proposed way to move forward 1. put the externals under .../trunk/ext_sources .../trunk/ext_sources .../trunk/main .../trunk/extras 2. adapt configure to use this as default, disable the download (maybe reactivate it later if we move to a DSCM) 3. keep the process with checking the md5 sum as it is (for potential later use) Any opinions or suggestions? +1. And one more question: If we put something into SVN into .../trunk/ext_sources, do we have some URL that can replace http://hg so users don't have to check out everything? Ie. do we have a URL where we have real checkout of the SVN? Some SVN web interface? Don't know Apache infra well yet... That would be real killer solution! -- I was thinking something similar. We only need to use the SVN interface to the files when we're adding or updating. But we can have bootstrap continue to download via http. The location, using Juergen's proposed location, would be http://svn.apache.org/repos/asf/incubator/ooo/trunk/ext-sources This would save having a duplicate local SVN working copy of the file, right? -Rob
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On 09/20/2011 05:26 PM, Rob Weir wrote: Ai2011/9/20 Pavel Janíkpa...@janik.cz: Have we ever considered using version control to...uh...manage file versions? Just an idea. Maybe Heiner will say more, but in the past, we have had the external tarballs in the VCS, but then we moved them out and it worked very well. There never was a reason to track external.tar.gz files in VCS, because we do not change them. -- That's fine. If they don't change, then doing a svn update will not bring them down each time. Aside from being useful for version control, SVN is useful also very useful as an audit trail. So in the rare occasions when one of these files does change, we know who changed it and why. This is important for ensuring the IP cleanliness of the project. Is your main concern performance? Even as individual tarballs, ext-sources is 86 files, 250MB. ooo/extras is 243 files and 822 MB. And ooo/main is 76,295 files for over 900MB. So ext-sources is not a huge contributor to download time. Placing all the external tarballs in the VCS is a real killer if using a distributed SCM like git or Mercurial, thats why we had moved them out. As Pavel said, it worked quite nice. As for the audit possibility, we referenced the external tar balls in the source tree by file name and a md5 check sum, which works just as reliantly as putting them directly into the repository. Nowadays the DSCM have some alternative methods which deal with such blobs but in essence they also keep them separate. If AOOo ever plans to go back to a DSCM I would keep the source tree and the external blobs strictly separated. All in all the general SCM tooling community opinion trend seems to be that a S(ource)CM system is for, well, source and external dependencies are better handled with other mechanism, like Maven or so. With SVN all this is less of a concern, naturally. Heiner -- Jens-Heiner Rechtien
handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
Hi, On 20.09.2011 14:37, Jürgen Schmidt wrote: On Mon, Sep 19, 2011 at 1:59 PM, Rob Weirrobw...@apache.org wrote: 2011/9/19 Jürgen Schmidtjogischm...@googlemail.com: On Mon, Sep 19, 2011 at 2:27 AM, Rob Weirrobw...@apache.org wrote: ... Suggestions: 1) We need to get all files needed for the build into SVN. Right now there are some that are copied down from the OpenOffice.org website during the build's bootstrap process. Until we get the files all in one place it is hard to get a comprehensive view of our dependencies. do you mean to check in the files under ext_source into svn and remove it later on when we have cleaned up the code. Or do you mean to put it somehwere on apache extras? I would prefer to save these binary files under apache extra if possible. Why not just keep in in SVN? Moving things to Apache-Extras does not help us with the IP review. In other words, if we have a dependency on a OSS module that has an incompatible license, then moving that module to Apache Extras does not make that dependency go away. We still need to understand the nature of the dependency: a build tool, a dynamic runtime dependency, a statically linked library, an optional extensions, a necessary core module. If we find out, for example, that something in ext-sources is only used as a build tool, and is not part of the release, then there is nothing that prevents us from hosting it in SVN. But if something is a necessary library and it is under GPL, then this is a problem even if we store it on Apache-Extras, i am not really happy with all the binaries in the trunk tree because of the large binary blobs and i don't expect too many changes of these dependencies. And i would like to avoid to check them out every time. What do others think about a structure where we have ext_sources besides trunk. incubator/ooo/trunk incubator/ooo/ext_source ... I like this idea. From a developer point of view I only have to checkout ext_sources once and reference it from all my trunks using the already existing configure-switch 'with-external-tar=path to ext_sources' Best regards, Oliver.
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
Hi, I like this idea. From a developer point of view I only have to checkout ext_sources once and reference it from all my trunks using the already existing configure-switch 'with-external-tar=path to ext_sources' when we will have such repository, we will surely modify the current sources so you don't have to add such switch because ../ext_sources will be auto-checked. BTW - welcome! :-) -- Pavel Janík
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On Tue, Sep 20, 2011 at 9:48 AM, Armin Le Grand armin.le.gr...@me.com wrote: On 20.09.2011 15:33, Oliver-Rainer Wittmann wrote: Hi, On 20.09.2011 14:37, Jürgen Schmidt wrote: ... What do others think about a structure where we have ext_sources besides trunk. incubator/ooo/trunk incubator/ooo/ext_source ... So are we saying we would never need to branch or tag these files? For example, suppose we release AOOo 3.4.0, and then later we release AOOo 4.0. Then someone finds a serious security flaw in AOOo 3.4.0, and we decide to release an AOOo 3.4.1 as well as a AOOo 4.0.1. Would we be able to do this? What if the flaw was related to code in ext_sources? And if not us, in the project, what if some downstream consumer of AOOo 3.4.0 wants to rebuild 3.4.0 later, for a patch or whatever. But we've already updated ext_sources for AOOo 4.0? In other words, how do we track, in SVN, a compatible set of matching trunk/ and ext_source/ revisions, so we (or someone else) can recreate any released version of AOOo? -Rob I like this idea. From a developer point of view I only have to checkout ext_sources once and reference it from all my trunks using the already existing configure-switch 'with-external-tar=path to ext_sources' +1 Also, hopefully ext_sources will not change too much (after a consolidation phase) and it's mostly binaries, thus not too well suited for a repository. Let's not extend our main repository with those binaries, please. Best regards, Oliver. Regards, Armin -- ALG
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
Would we be able to do this? What if the flaw was related to code in ext_sources? Then we patch it. Patch will be in the trunk/main, as always. And if not us, in the project, what if some downstream consumer of AOOo 3.4.0 wants to rebuild 3.4.0 later, for a patch or whatever. But we've already updated ext_sources for AOOo 4.0? Versions - we can and will have more tarballs of one external source. This all is already solved. -- Pavel Janík
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
On 20.09.2011 15:58, Rob Weir wrote: On Tue, Sep 20, 2011 at 9:48 AM, Armin Le Grandarmin.le.gr...@me.com wrote: On 20.09.2011 15:33, Oliver-Rainer Wittmann wrote: Hi, On 20.09.2011 14:37, Jürgen Schmidt wrote: ... What do others think about a structure where we have ext_sources besides trunk. incubator/ooo/trunk incubator/ooo/ext_source ... So are we saying we would never need to branch or tag these files? For example, suppose we release AOOo 3.4.0, and then later we release AOOo 4.0. Then someone finds a serious security flaw in AOOo 3.4.0, and we decide to release an AOOo 3.4.1 as well as a AOOo 4.0.1. Would we be able to do this? What if the flaw was related to code in ext_sources? And if not us, in the project, what if some downstream consumer of AOOo 3.4.0 wants to rebuild 3.4.0 later, for a patch or whatever. But we've already updated ext_sources for AOOo 4.0? In other words, how do we track, in SVN, a compatible set of matching trunk/ and ext_source/ revisions, so we (or someone else) can recreate any released version of AOOo? Good point. Thus, it should be part of incubator/ooo/trunk, something like: incubator/ooo/trunk/main incubator/ooo/trunk/extras incubator/ooo/trunk/ext_sources It could be in an own repro, but this would just bring up the risk to not use the same tags in both (by purpose or by error). Indeed, looks as if it has to be a part of trunk somehow. Not very nice for binaries. Maybe we could find a intermediate place for them as long as we will need to do changes pretty often. Currently we will have to do some add/remove/changes to it. It could be good to add them to trunk after it has stabilized a little more. -Rob I like this idea. From a developer point of view I only have to checkout ext_sources once and reference it from all my trunks using the already existing configure-switch 'with-external-tar=path to ext_sources' +1 Also, hopefully ext_sources will not change too much (after a consolidation phase) and it's mostly binaries, thus not too well suited for a repository. Let's not extend our main repository with those binaries, please. Best regards, Oliver. Regards, Armin -- ALG
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
+1 - This will make it easier to update the BSD/MIT unrestricted stuff. - Hopefully it also means we will eventually stop depending on GNU patch for the build. Welcome Oliver! Great job Juergen: it's the first code replacement and a very necessary one for OO forks too (unless they want to carry lcc's copyright;) ). cheers, Pedro. On Tue, 20 Sep 2011 15:44:59 +0200, Pavel Janík pa...@janik.cz wrote: Hi, I like this idea. From a developer point of view I only have to checkout ext_sources once and reference it from all my trunks using the already existing configure-switch 'with-external-tar=path to ext_sources' when we will have such repository, we will surely modify the current sources so you don't have to add such switch because ../ext_sources will be auto-checked. BTW - welcome! :-)
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
Have we ever considered using version control to...uh...manage file versions? Just an idea. Maybe Heiner will say more, but in the past, we have had the external tarballs in the VCS, but then we moved them out and it worked very well. There never was a reason to track external.tar.gz files in VCS, because we do not change them. -- Pavel Janík
Re: handling of ext_sources - Juergen's suggestion [was: Re: A systematic approach to IP review?]
Ai2011/9/20 Pavel Janík pa...@janik.cz: Have we ever considered using version control to...uh...manage file versions? Just an idea. Maybe Heiner will say more, but in the past, we have had the external tarballs in the VCS, but then we moved them out and it worked very well. There never was a reason to track external.tar.gz files in VCS, because we do not change them. -- That's fine. If they don't change, then doing a svn update will not bring them down each time. Aside from being useful for version control, SVN is useful also very useful as an audit trail. So in the rare occasions when one of these files does change, we know who changed it and why. This is important for ensuring the IP cleanliness of the project. Is your main concern performance? Even as individual tarballs, ext-sources is 86 files, 250MB. ooo/extras is 243 files and 822 MB. And ooo/main is 76,295 files for over 900MB. So ext-sources is not a huge contributor to download time. Pavel Janík
