Re: possible to filter the output to commits@ list????
Hoss, you and I should see if we can wrangle some of the SVN guys at ApacheCon and see if we can get this stuff straightened out. On Oct 24, 2010, at 8:33 PM, Yonik Seeley wrote: Another downside to the current situation - svn log shows all these updates to the mergeprops, so we're losing the real updates in the noise. Maybe we should try to merge only from trunk to branch_3x, so at least trunk says relatively clean. Making sense of branch_3x is getting very difficult. Example: yo...@wolverine /cygdrive/c/code/lusolr_3x $ svn log lucene/src/test/org/apache/lucene/search/spans/TestPayloadSpans.java r1026921 | yonik | 2010-10-24 20:23:17 -0400 (Sun, 24 Oct 2010) | 1 line tests: fix resource leaks (merge from trunk) r1026885 | yonik | 2010-10-24 16:56:54 -0400 (Sun, 24 Oct 2010) | 1 line SOLR-2192: sync runners in StreamingUpdateSolrServer.blockUntilFinished r1026873 | yonik | 2010-10-24 15:52:34 -0400 (Sun, 24 Oct 2010) | 1 line SOLR-2160: tests - use classloader from core resource loader r1026869 | yonik | 2010-10-24 15:24:39 -0400 (Sun, 24 Oct 2010) | 1 line SOLR-2162: tests should use multi-threaded connection manager r1026739 | koji | 2010-10-24 00:42:39 -0400 (Sun, 24 Oct 2010) | 1 line fix indent in DIH sample r1026593 | mikemccand | 2010-10-23 06:23:02 -0400 (Sat, 23 Oct 2010) | 1 line LUCENE-2618: allow optimize to complete during IW.close, take 2 r1026432 | mikemccand | 2010-10-22 13:59:16 -0400 (Fri, 22 Oct 2010) | 1 line LUCENE-2618: revert r1026337 | mikemccand | 2010-10-22 10:17:10 -0400 (Fri, 22 Oct 2010) | 1 line LUCENE-2618: make sure optimize merge complete even if a close is pending r1026170 | uschindler | 2010-10-21 18:39:44 -0400 (Thu, 21 Oct 2010) | 1 line merge: remove warning r1026053 | yonik | 2010-10-21 12:16:22 -0400 (Thu, 21 Oct 2010) | 1 line SOLR-2180: close request in finally block r1026035 | yonik | 2010-10-21 11:49:45 -0400 (Thu, 21 Oct 2010) | 1 line tests: fix resource leaks, convert to Junit4 (merged from trunk) r1026024 | yonik | 2010-10-21 11:09:32 -0400 (Thu, 21 Oct 2010) | 1 line axe extra distrib test (merge trunk 1022805) r1026000 | gsingers | 2010-10-21 09:48:34 -0400 (Thu, 21 Oct 2010) | 1 line SOLR-2010, including Yonik's fix, SOLR-2181 -- hope I did this merge correctly r1025931 | mikemccand | 2010-10-21 06:29:01 -0400 (Thu, 21 Oct 2010) | 1 line fix MTQ.CutOffTermCollector to check limits after adding term, not before r1025606 | rmuir | 2010-10-20 10:51:12 -0400 (Wed, 20 Oct 2010) | 1 line LUCENE-1938: Precedence query parser using the contrib/queryparser framework r1024481 | yonik | 2010-10-19 21:27:57 -0400 (Tue, 19 Oct 2010) | 1 line SOLR-2197: wait for search executor to close before closing main server (merge trunk 1024476) r1024409 | uschindler | 2010-10-19 16:57:52 -0400 (Tue, 19 Oct 2010) | 1 line LUCENE-2556: Improve memory usage after cloning TermAttribute r1024405 | rmuir | 2010-10-19 16:46:23 -0400 (Tue, 19 Oct 2010) | 1 line LUCENE-1937: add methods to manipulate QueryNodeProcessorPipeline elements r1024258 | rmuir | 2010-10-19 11:04:11 -0400 (Tue, 19 Oct 2010) | 1 line SOLR-1794: Dataimport of CLOB fields fails when getCharacterStream() is defined in a superclass r1024223 | rmuir | 2010-10-19 08:59:48 -0400 (Tue, 19 Oct 2010) | 1 line LUCENE-2652: remove obselete runners, they are inconsistent with @beforeClass and obselete in the build today
Re: possible to filter the output to commits@ list????
Another downside to the current situation - svn log shows all these updates to the mergeprops, so we're losing the real updates in the noise. Maybe we should try to merge only from trunk to branch_3x, so at least trunk says relatively clean. Making sense of branch_3x is getting very difficult. Example: yo...@wolverine /cygdrive/c/code/lusolr_3x $ svn log lucene/src/test/org/apache/lucene/search/spans/TestPayloadSpans.java r1026921 | yonik | 2010-10-24 20:23:17 -0400 (Sun, 24 Oct 2010) | 1 line tests: fix resource leaks (merge from trunk) r1026885 | yonik | 2010-10-24 16:56:54 -0400 (Sun, 24 Oct 2010) | 1 line SOLR-2192: sync runners in StreamingUpdateSolrServer.blockUntilFinished r1026873 | yonik | 2010-10-24 15:52:34 -0400 (Sun, 24 Oct 2010) | 1 line SOLR-2160: tests - use classloader from core resource loader r1026869 | yonik | 2010-10-24 15:24:39 -0400 (Sun, 24 Oct 2010) | 1 line SOLR-2162: tests should use multi-threaded connection manager r1026739 | koji | 2010-10-24 00:42:39 -0400 (Sun, 24 Oct 2010) | 1 line fix indent in DIH sample r1026593 | mikemccand | 2010-10-23 06:23:02 -0400 (Sat, 23 Oct 2010) | 1 line LUCENE-2618: allow optimize to complete during IW.close, take 2 r1026432 | mikemccand | 2010-10-22 13:59:16 -0400 (Fri, 22 Oct 2010) | 1 line LUCENE-2618: revert r1026337 | mikemccand | 2010-10-22 10:17:10 -0400 (Fri, 22 Oct 2010) | 1 line LUCENE-2618: make sure optimize merge complete even if a close is pending r1026170 | uschindler | 2010-10-21 18:39:44 -0400 (Thu, 21 Oct 2010) | 1 line merge: remove warning r1026053 | yonik | 2010-10-21 12:16:22 -0400 (Thu, 21 Oct 2010) | 1 line SOLR-2180: close request in finally block r1026035 | yonik | 2010-10-21 11:49:45 -0400 (Thu, 21 Oct 2010) | 1 line tests: fix resource leaks, convert to Junit4 (merged from trunk) r1026024 | yonik | 2010-10-21 11:09:32 -0400 (Thu, 21 Oct 2010) | 1 line axe extra distrib test (merge trunk 1022805) r1026000 | gsingers | 2010-10-21 09:48:34 -0400 (Thu, 21 Oct 2010) | 1 line SOLR-2010, including Yonik's fix, SOLR-2181 -- hope I did this merge correctly r1025931 | mikemccand | 2010-10-21 06:29:01 -0400 (Thu, 21 Oct 2010) | 1 line fix MTQ.CutOffTermCollector to check limits after adding term, not before r1025606 | rmuir | 2010-10-20 10:51:12 -0400 (Wed, 20 Oct 2010) | 1 line LUCENE-1938: Precedence query parser using the contrib/queryparser framework r1024481 | yonik | 2010-10-19 21:27:57 -0400 (Tue, 19 Oct 2010) | 1 line SOLR-2197: wait for search executor to close before closing main server (merge trunk 1024476) r1024409 | uschindler | 2010-10-19 16:57:52 -0400 (Tue, 19 Oct 2010) | 1 line LUCENE-2556: Improve memory usage after cloning TermAttribute r1024405 | rmuir | 2010-10-19 16:46:23 -0400 (Tue, 19 Oct 2010) | 1 line LUCENE-1937: add methods to manipulate QueryNodeProcessorPipeline elements r1024258 | rmuir | 2010-10-19 11:04:11 -0400 (Tue, 19 Oct 2010) | 1 line SOLR-1794: Dataimport of CLOB fields fails when getCharacterStream() is defined in a superclass r1024223 | rmuir | 2010-10-19 08:59:48 -0400 (Tue, 19 Oct 2010) | 1 line LUCENE-2652: remove obselete runners, they are inconsistent with @beforeClass and obselete in the build today r1024197 | mikemccand | 2010-10-19 06:27:19 -0400 (Tue, 19 Oct 2010) | 1 line add some missing super.setParams so alg.toString shows the params r1023877 | uschindler | 2010-10-18
Re: possible to filter the output to commits@ list????
On Oct 19, 2010, at 4:14 PM, Robert Muir wrote: On Tue, Oct 19, 2010 at 3:51 PM, Michael McCandless luc...@mikemccandless.com wrote: Hmm but how does this explain how the number of mergeprops has grown so much recently? I think recent merges have generally been toplevel? i introduced a lot of them by combining trunk's analyzers into modules/analysis, because things were reorganized, there are many, but still there are more than necessary and this could be cleaned up. Couldn't 3.x move contrib/analysis to modules? We could just move it to contrib as part of the release packaging if the goal is to be consistent, but otherwise who cares where it is located? Would that reduce some of this? still, if we are going to refactor lucene+lucene contrib+solr in other ways (i hope we will!) I would imagine we would see more of this in the future. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem docs using Solr/Lucene: http://www.lucidimagination.com/search - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: possible to filter the output to commits@ list????
No. Once the files have a property they have it irreversible. This is why single-file merges are a no-go. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Grant Ingersoll [mailto:gsing...@apache.org] Sent: Wednesday, October 20, 2010 1:01 PM To: dev@lucene.apache.org Subject: Re: possible to filter the output to commits@ list On Oct 19, 2010, at 4:14 PM, Robert Muir wrote: On Tue, Oct 19, 2010 at 3:51 PM, Michael McCandless luc...@mikemccandless.com wrote: Hmm but how does this explain how the number of mergeprops has grown so much recently? I think recent merges have generally been toplevel? i introduced a lot of them by combining trunk's analyzers into modules/analysis, because things were reorganized, there are many, but still there are more than necessary and this could be cleaned up. Couldn't 3.x move contrib/analysis to modules? We could just move it to contrib as part of the release packaging if the goal is to be consistent, but otherwise who cares where it is located? Would that reduce some of this? still, if we are going to refactor lucene+lucene contrib+solr in other ways (i hope we will!) I would imagine we would see more of this in the future. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem docs using Solr/Lucene: http://www.lucidimagination.com/search - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
On Wed, Oct 20, 2010 at 7:01 AM, Grant Ingersoll gsing...@apache.org wrote: Couldn't 3.x move contrib/analysis to modules? We could just move it to contrib as part of the release packaging if the goal is to be consistent, but otherwise who cares where it is located? Would that reduce some of this? I don't think it would help so much, because the analyzers in modules did not come just from contrib/analyzers... but from lucene's core (individual files cherrypicked, things like TokenStream still stay), contrib/icu, and from Solr (the factories stay at the moment though). thats just what i mean though, if we will make lucene more modular i think this is just a very typical case of consolidating things from different places in lucene/solr and modularizing it, so thats why I think a lot of it is unavoidable. Uwe says single-file merges are a no-go, but (hypothetically) if we moved just the concrete Queries from o.a.l.search (but not everything else!), and the Queries in contrib/queries into a modules/queries, then you would end up with the same situation... if you fix a bug in modules/queries/TermQuery you have to merge the individual file in 3.x's src/java because the entire o.a.l.search did not move, only a few cherrypicked files (the queries). - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
On Mon, Oct 18, 2010 at 3:03 PM, Grant Ingersoll gsing...@apache.org wrote: It was just a bit of a shocker for me the first time I did it and I see like 30 files changed when I only changed one file. Me too. In fact I think it's ridiculous -- violates principle of least surprise. I shouldn't have to see such details of the source control system's impl Furthermore, I think it may eventually turn into a serious perf issue, since it seems to be an O(N^2) growth. We are at 7 emails today, which was only 3 emails not long ago. Where will be be a few months from now? (Though I guess it is bounded by the total number of source files we have in 3.x...). Maybe svn is trying to tell us to release 4.0, heh ;) Mike - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
FWIW, the commit notices are just an SVN post-commit hook that uses the svn-mailer tool [http://opensource.perlig.de/svnmailer/]. I believe Grant has commit rights to that file - it is in the infra SVN right next to the old asf-auth file. Right now I have no idea whether svn-mailer can support the kind of filtering you are talking about, but there's no harm looking! Upayavira On Tue, 19 Oct 2010 06:45 -0400, Michael McCandless luc...@mikemccandless.com wrote: On Mon, Oct 18, 2010 at 3:03 PM, Grant Ingersoll gsing...@apache.org wrote: It was just a bit of a shocker for me the first time I did it and I see like 30 files changed when I only changed one file. Me too. In fact I think it's ridiculous -- violates principle of least surprise. I shouldn't have to see such details of the source control system's impl Furthermore, I think it may eventually turn into a serious perf issue, since it seems to be an O(N^2) growth. We are at 7 emails today, which was only 3 emails not long ago. Where will be be a few months from now? (Though I guess it is bounded by the total number of source files we have in 3.x...). Maybe svn is trying to tell us to release 4.0, heh ;) Mike - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
On Oct 19, 2010, at 7:06 AM, Upayavira wrote: FWIW, the commit notices are just an SVN post-commit hook that uses the svn-mailer tool [http://opensource.perlig.de/svnmailer/]. I believe Grant has commit rights to that file - it is in the infra SVN right next to the old asf-auth file. Yes, I do, but it isn't clear if this is something that could be handled through simple routing. I suppose one thing we could do is filter it through a dummy GMail account somehow. Send all commits to the GMail group and then have it send a digest email a few times a day to comm...@l.a.o. It's ugly, but it might work. FWIW, I get the sense that a lot of other projects deal with merges. What do they do? Right now I have no idea whether svn-mailer can support the kind of filtering you are talking about, but there's no harm looking! Upayavira On Tue, 19 Oct 2010 06:45 -0400, Michael McCandless luc...@mikemccandless.com wrote: On Mon, Oct 18, 2010 at 3:03 PM, Grant Ingersoll gsing...@apache.org wrote: It was just a bit of a shocker for me the first time I did it and I see like 30 files changed when I only changed one file. Me too. In fact I think it's ridiculous -- violates principle of least surprise. I shouldn't have to see such details of the source control system's impl Furthermore, I think it may eventually turn into a serious perf issue, since it seems to be an O(N^2) growth. We are at 7 emails today, which was only 3 emails not long ago. Where will be be a few months from now? (Though I guess it is bounded by the total number of source files we have in 3.x...). Maybe svn is trying to tell us to release 4.0, heh ;) Mike - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem docs using Solr/Lucene: http://www.lucidimagination.com/search - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
: FWIW, I get the sense that a lot of other projects deal with merges. What do they do? I suspect they do merges properly and avoid this problem entirely. Bottom Line: if *all* merges happen at the top level, then this problem won't exist -- mergeinfo props get added to individual files only when individual file merges take place, or with merges from mixed working revisions. Once a file gets those merge props, all future merges in that tree interact with those individual file props even if the merge has nothing to do with those files... http://svnbook.red-bean.com/en/1.5/svn.branchmerge.advanced.html#svn.branchmerge.advanced.finalword For long-lived release branches (as described in the section called “Common Branching Patterns”), perform merges only on the root of the branch, not on subdirectories. -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
On Oct 19, 2010, at 3:04 PM, Chris Hostetter wrote: : FWIW, I get the sense that a lot of other projects deal with merges. What do they do? I suspect they do merges properly and avoid this problem entirely. Bottom Line: if *all* merges happen at the top level, then this problem won't exist -- mergeinfo props get added to individual files only when individual file merges take place, or with merges from mixed working revisions. Once a file gets those merge props, all future merges in that tree interact with those individual file props even if the merge has nothing to do with those files... http://svnbook.red-bean.com/en/1.5/svn.branchmerge.advanced.html#svn.branchmerge.advanced.finalword I will go take a look at it, but as far as I could tell, I merged at the top level, but maybe IntelliJ does individual file merges and I wasn't aware of it (I believe I said merge trunk to branch_3x and then picked which revisions I wanted to apply). Perhaps we should put together some examples in Lucene's context that would give Lucene developers a deeper understanding. -Grant - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: possible to filter the output to commits@ list????
Hi Grant, The individual merges are there since long time. And the additional ones came in later. As I said before (answer to first mail on this thread): The problem is that once a file gets a separate merge property not inherited from its parent folder, this merge prop must be updated on every later merge. So whenever you merge a file separately you create one more problem :-) Some merges in trunk cannot be done completely top-level, but if they are always done in the same way, the number of files/subdirs with separate mergeprops is limited. One problem is the new modules folder as the analysis folder in it must be merged to contrib. Also the moved tokenstreams and analyzers. But all other touched files are results of one-file merges. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Grant Ingersoll [mailto:gsing...@apache.org] Sent: Tuesday, October 19, 2010 9:30 PM To: dev@lucene.apache.org Subject: Re: possible to filter the output to commits@ list On Oct 19, 2010, at 3:04 PM, Chris Hostetter wrote: : FWIW, I get the sense that a lot of other projects deal with merges. What do they do? I suspect they do merges properly and avoid this problem entirely. Bottom Line: if *all* merges happen at the top level, then this problem won't exist -- mergeinfo props get added to individual files only when individual file merges take place, or with merges from mixed working revisions. Once a file gets those merge props, all future merges in that tree interact with those individual file props even if the merge has nothing to do with those files... http://svnbook.red-bean.com/en/1.5/svn.branchmerge.advanced.html#svn.b ranchmerge.advanced.finalword I will go take a look at it, but as far as I could tell, I merged at the top level, but maybe IntelliJ does individual file merges and I wasn't aware of it (I believe I said merge trunk to branch_3x and then picked which revisions I wanted to apply). Perhaps we should put together some examples in Lucene's context that would give Lucene developers a deeper understanding. -Grant - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
On Tue, Oct 19, 2010 at 3:29 PM, Grant Ingersoll gsing...@apache.org wrote: I will go take a look at it, but as far as I could tell, I merged at the top level, but maybe IntelliJ does individual file merges and I wasn't aware of it (I believe I said merge trunk to branch_3x and then picked which revisions I wanted to apply). Perhaps we should put together some examples in Lucene's context that would give Lucene developers a deeper understanding. It's not just you... it's any of us in the past who merged singleton files (I'm pretty sure I did so before I understood this must merge from root mantra). Hmm but how does this explain how the number of mergeprops has grown so much recently? I think recent merges have generally been toplevel? Mike - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
On Tue, Oct 19, 2010 at 3:51 PM, Michael McCandless luc...@mikemccandless.com wrote: Hmm but how does this explain how the number of mergeprops has grown so much recently? I think recent merges have generally been toplevel? i introduced a lot of them by combining trunk's analyzers into modules/analysis, because things were reorganized, there are many, but still there are more than necessary and this could be cleaned up. still, if we are going to refactor lucene+lucene contrib+solr in other ways (i hope we will!) I would imagine we would see more of this in the future. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: possible to filter the output to commits@ list????
Hi Upayavira, Thanks for the hint. Indeed with changing the config file (which allows special configs for specific subtrees of the svn, so we can do it only for Lucene), we can do it very easy: http://opensource.perlig.de/svnmailer/doc-1.0/#groups-generate-diffs The generate_diffs option defines which actions diffs are generated for. It takes a space or tab separated list of one or more of the following tokens: add, modify, copy, delete, propchange and none. If the add token is given and a new file is added to the repository, the svnmailer generates a diff between an empty file and the newly added one. If the modify token is given and the content of an already existing file is changed, a diff between the old revision and the new revision of that file is generated. The copy token only worries about files, that are copied and modified during one commit. The delete token generates a diff between the previous revision of the file and an empty file, if a file was deleted. If the propchange token is given, the svnmailer also takes care of changes in versioned properties. Whether it should actually generate diffs for the property change action depends on the other tokens of the generate_diffs list. The same rules as for files apply, except that the svnmailer never generates property diffs for deleted files If we change that config option and remove propchange, then the diffs would not contain propchanges anymore. It would it only list as modified files, but with that we can live. Grant: Can you send me a copy of the current config file of that tool? I could create a patch! (I am allowed to see it). Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Upayavira [mailto:u...@odoko.co.uk] Sent: Tuesday, October 19, 2010 1:07 PM To: dev@lucene.apache.org Subject: Re: possible to filter the output to commits@ list FWIW, the commit notices are just an SVN post-commit hook that uses the svn- mailer tool [http://opensource.perlig.de/svnmailer/]. I believe Grant has commit rights to that file - it is in the infra SVN right next to the old asf-auth file. Right now I have no idea whether svn-mailer can support the kind of filtering you are talking about, but there's no harm looking! Upayavira On Tue, 19 Oct 2010 06:45 -0400, Michael McCandless luc...@mikemccandless.com wrote: On Mon, Oct 18, 2010 at 3:03 PM, Grant Ingersoll gsing...@apache.org wrote: It was just a bit of a shocker for me the first time I did it and I see like 30 files changed when I only changed one file. Me too. In fact I think it's ridiculous -- violates principle of least surprise. I shouldn't have to see such details of the source control system's impl Furthermore, I think it may eventually turn into a serious perf issue, since it seems to be an O(N^2) growth. We are at 7 emails today, which was only 3 emails not long ago. Where will be be a few months from now? (Though I guess it is bounded by the total number of source files we have in 3.x...). Maybe svn is trying to tell us to release 4.0, heh ;) Mike - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: possible to filter the output to commits@ list????
(I am allowed to see it). I mean: (if I am allowed to see it.) - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Tuesday, October 19, 2010 11:21 PM To: dev@lucene.apache.org Subject: RE: possible to filter the output to commits@ list Hi Upayavira, Thanks for the hint. Indeed with changing the config file (which allows special configs for specific subtrees of the svn, so we can do it only for Lucene), we can do it very easy: http://opensource.perlig.de/svnmailer/doc-1.0/#groups-generate-diffs The generate_diffs option defines which actions diffs are generated for. It takes a space or tab separated list of one or more of the following tokens: add, modify, copy, delete, propchange and none. If the add token is given and a new file is added to the repository, the svnmailer generates a diff between an empty file and the newly added one. If the modify token is given and the content of an already existing file is changed, a diff between the old revision and the new revision of that file is generated. The copy token only worries about files, that are copied and modified during one commit. The delete token generates a diff between the previous revision of the file and an empty file, if a file was deleted. If the propchange token is given, the svnmailer also takes care of changes in versioned properties. Whether it should actually generate diffs for the property change action depends on the other tokens of the generate_diffs list. The same rules as for files apply, except that the svnmailer never generates property diffs for deleted files If we change that config option and remove propchange, then the diffs would not contain propchanges anymore. It would it only list as modified files, but with that we can live. Grant: Can you send me a copy of the current config file of that tool? I could create a patch! (I am allowed to see it). Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Upayavira [mailto:u...@odoko.co.uk] Sent: Tuesday, October 19, 2010 1:07 PM To: dev@lucene.apache.org Subject: Re: possible to filter the output to commits@ list FWIW, the commit notices are just an SVN post-commit hook that uses the svn- mailer tool [http://opensource.perlig.de/svnmailer/]. I believe Grant has commit rights to that file - it is in the infra SVN right next to the old asf-auth file. Right now I have no idea whether svn-mailer can support the kind of filtering you are talking about, but there's no harm looking! Upayavira On Tue, 19 Oct 2010 06:45 -0400, Michael McCandless luc...@mikemccandless.com wrote: On Mon, Oct 18, 2010 at 3:03 PM, Grant Ingersoll gsing...@apache.org wrote: It was just a bit of a shocker for me the first time I did it and I see like 30 files changed when I only changed one file. Me too. In fact I think it's ridiculous -- violates principle of least surprise. I shouldn't have to see such details of the source control system's impl Furthermore, I think it may eventually turn into a serious perf issue, since it seems to be an O(N^2) growth. We are at 7 emails today, which was only 3 emails not long ago. Where will be be a few months from now? (Though I guess it is bounded by the total number of source files we have in 3.x...). Maybe svn is trying to tell us to release 4.0, heh ;) Mike - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
On Oct 17, 2010, at 10:58 AM, Robert Muir wrote: Lets say i change a single line of code, and merge it back to the 3x_branch. Currently we get 6 or 7 emails of mergeproperty changes to the commits@ list... this is making it difficult or impossible for backports to be reviewed at all... I think this is terrible. I find it onerous that one need do a merge for this kind of thing period. Why not just apply the patch a second time? Sure, something is lost in SVN, but it's covered elsewhere. Of course, the flip side is that by not doing it, it becomes all that much harder to merge in the future. -Grant - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
What commands do you want me to run? On Oct 17, 2010, at 11:27 AM, Uwe Schindler wrote: Hi Daniel, I already proposed something similar to our developers list and also explained, why it is a bad idea to merge single files (see attached mail). Still our/my opinion: It may be better, to simply provide optionally plain diffs without metadata for commit mails (on request of a project, e.g. Grant Ingersoll could configure that for the Lucene project in the SVN config file). Uwe - Uwe Schindler uschind...@apache.org Apache Lucene PMC Member / Committer Bremen, Germany http://lucene.apache.org/ -Original Message- From: Daniel Shahaf [mailto:d...@daniel.shahaf.name] Sent: Sunday, October 17, 2010 5:17 PM To: Uwe Schindler Cc: dev@lucene.apache.org; 'Apache Infrastructure' Subject: Re: possible to filter the output to commits@ list Or you could try to see if some of the existing mergeinfo can be removed, and try to have less subtree mergeinfo in the first place; these are common topics on us...@subversion. Uwe Schindler wrote on Sun, Oct 17, 2010 at 17:06:44 +0200: Hi all, There are configuration options in SVN to let the commit mails pipe through a filter. Other svn hosting providers like Sourceforge provide some filters to be applied. Ideally, you would use the patchutils package (it's called like this in Debian/Ubuntu) and use svn diff | filterdiff --clean and pipe that into an eMail. Maybe we should open an issue at infra? Uwe - Uwe Schindler uschind...@apache.org Apache Lucene PMC Member / Committer Bremen, Germany http://lucene.apache.org/ -Original Message- From: Robert Muir [mailto:rcm...@gmail.com] Sent: Sunday, October 17, 2010 4:58 PM To: dev@lucene.apache.org Subject: possible to filter the output to commits@ list Lets say i change a single line of code, and merge it back to the 3x_branch. Currently we get 6 or 7 emails of mergeproperty changes to the commits@ list... this is making it difficult or impossible for backports to be reviewed at all... I think this is terrible. How can we get this fixed so that these mergeprop-changes only are filtered in the emails? Someone could always click the link to viewvc or whatever if they are interested in this... - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org Mail Attachment.eml -- Grant Ingersoll http://www.lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
On Mon, Oct 18, 2010 at 12:47 PM, Grant Ingersoll gsing...@apache.org wrote: I find it onerous that one need do a merge for this kind of thing period. Why not just apply the patch a second time? Sure, something is lost in SVN, but it's covered elsewhere. Of course, the flip side is that by not doing it, it becomes all that much harder to merge in the future. why use version control at all? by merging, the practical benefit to me is it tracks what has and hasn't been merged (my ide has a nice interface that lets me quickly see eligible revs by my username, etc) but if you just patch and commit twice, it looks to svn from the mergeinfo that you never merged that change back to branch_3x... if you are going to do this, then at least mark the revision as merged so it doesnt mislead people who are using version control. (separately merging makes life easier to me for anything non-trivial, a patch wouldn't apply cleanly anyway to both our branches, i'd rather just resolve the conflicting parts) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
: I find it onerous that one need do a merge for this kind of thing : period. Why not just apply the patch a second time? Sure, something is : lost in SVN, but it's covered elsewhere. Of course, the flip side is : that by not doing it, it becomes all that much harder to merge in the : future. Exactly -- the amount of work required to apply a one line patch to two distinct branches is about the same as the amount of work to apply that patch to trunk and then svn merge it to the branch -- but the impact on other people who have much more complicated merges is huge. when you use svn merge, you make all of those future (big) merges much simpler to deal with -- but every time you apply a patch directly to both branches and commit individually, you introduce a conflict that has to be resolved manually by every person who ever tries an svn merge on that file in the future. -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: possible to filter the output to commits@ list????
Hi Grant, Not yet decided/investigated, this was more a question to infra, if it would be possible to set this up. :-) I have to check a config from an SVN server that supports merging. Where are the Apach config files for SVN? Can I view them? Else, I agree with Robert: I also want to track all merges! Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Grant Ingersoll [mailto:gsing...@apache.org] Sent: Monday, October 18, 2010 6:48 PM Cc: dev@lucene.apache.org Subject: Re: possible to filter the output to commits@ list What commands do you want me to run? On Oct 17, 2010, at 11:27 AM, Uwe Schindler wrote: Hi Daniel, I already proposed something similar to our developers list and also explained, why it is a bad idea to merge single files (see attached mail). Still our/my opinion: It may be better, to simply provide optionally plain diffs without metadata for commit mails (on request of a project, e.g. Grant Ingersoll could configure that for the Lucene project in the SVN config file). Uwe - Uwe Schindler uschind...@apache.org Apache Lucene PMC Member / Committer Bremen, Germany http://lucene.apache.org/ -Original Message- From: Daniel Shahaf [mailto:d...@daniel.shahaf.name] Sent: Sunday, October 17, 2010 5:17 PM To: Uwe Schindler Cc: dev@lucene.apache.org; 'Apache Infrastructure' Subject: Re: possible to filter the output to commits@ list Or you could try to see if some of the existing mergeinfo can be removed, and try to have less subtree mergeinfo in the first place; these are common topics on us...@subversion. Uwe Schindler wrote on Sun, Oct 17, 2010 at 17:06:44 +0200: Hi all, There are configuration options in SVN to let the commit mails pipe through a filter. Other svn hosting providers like Sourceforge provide some filters to be applied. Ideally, you would use the patchutils package (it's called like this in Debian/Ubuntu) and use svn diff | filterdiff --clean and pipe that into an eMail. Maybe we should open an issue at infra? Uwe - Uwe Schindler uschind...@apache.org Apache Lucene PMC Member / Committer Bremen, Germany http://lucene.apache.org/ -Original Message- From: Robert Muir [mailto:rcm...@gmail.com] Sent: Sunday, October 17, 2010 4:58 PM To: dev@lucene.apache.org Subject: possible to filter the output to commits@ list Lets say i change a single line of code, and merge it back to the 3x_branch. Currently we get 6 or 7 emails of mergeproperty changes to the commits@ list... this is making it difficult or impossible for backports to be reviewed at all... I think this is terrible. How can we get this fixed so that these mergeprop-changes only are filtered in the emails? Someone could always click the link to viewvc or whatever if they are interested in this... --- - - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org Mail Attachment.eml -- Grant Ingersoll http://www.lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: possible to filter the output to commits@ list????
On Oct 18, 2010, at 2:07 PM, Uwe Schindler wrote: Hi Grant, Not yet decided/investigated, this was more a question to infra, if it would be possible to set this up. :-) I have to check a config from an SVN server that supports merging. Where are the Apach config files for SVN? Can I view them? I think you will need to go through infra for the config, as even I don't have that. I don't believe there is a public URL (or even PMC level) Else, I agree with Robert: I also want to track all merges! I agree. It was just a bit of a shocker for me the first time I did it and I see like 30 files changed when I only changed one file. Once you get used to it, though, not so bad and the IDE definitely helps. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: possible to filter the output to commits@ list????
Hi Daniel, I already proposed something similar to our developers list and also explained, why it is a bad idea to merge single files (see attached mail). Still our/my opinion: It may be better, to simply provide optionally plain diffs without metadata for commit mails (on request of a project, e.g. Grant Ingersoll could configure that for the Lucene project in the SVN config file). Uwe - Uwe Schindler uschind...@apache.org Apache Lucene PMC Member / Committer Bremen, Germany http://lucene.apache.org/ -Original Message- From: Daniel Shahaf [mailto:d...@daniel.shahaf.name] Sent: Sunday, October 17, 2010 5:17 PM To: Uwe Schindler Cc: dev@lucene.apache.org; 'Apache Infrastructure' Subject: Re: possible to filter the output to commits@ list Or you could try to see if some of the existing mergeinfo can be removed, and try to have less subtree mergeinfo in the first place; these are common topics on us...@subversion. Uwe Schindler wrote on Sun, Oct 17, 2010 at 17:06:44 +0200: Hi all, There are configuration options in SVN to let the commit mails pipe through a filter. Other svn hosting providers like Sourceforge provide some filters to be applied. Ideally, you would use the patchutils package (it's called like this in Debian/Ubuntu) and use svn diff | filterdiff --clean and pipe that into an eMail. Maybe we should open an issue at infra? Uwe - Uwe Schindler uschind...@apache.org Apache Lucene PMC Member / Committer Bremen, Germany http://lucene.apache.org/ -Original Message- From: Robert Muir [mailto:rcm...@gmail.com] Sent: Sunday, October 17, 2010 4:58 PM To: dev@lucene.apache.org Subject: possible to filter the output to commits@ list Lets say i change a single line of code, and merge it back to the 3x_branch. Currently we get 6 or 7 emails of mergeproperty changes to the commits@ list... this is making it difficult or impossible for backports to be reviewed at all... I think this is terrible. How can we get this fixed so that these mergeprop-changes only are filtered in the emails? Someone could always click the link to viewvc or whatever if they are interested in this... - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org ---BeginMessage--- I cc'ed my previous eMail also to the in...@ao list. In general, if we cannot filter we could think about the following: - remove all mergepros recursively for trunk, but we would lose history for merges. - Alternatively merge all mergeprops up to the root folder and remove it from the separate files/subfolders. This would lose some changes, but if all are merged up, that’s fine (only some files may appear as merged from different branch, although they aren't). In general when merging to reduce the amount of mergeprops: - Don't merge single files or subdirectories! Always merge the top folder! Excluded from this is trunk/modules, as this needs a separate, merging step between trunk/3x. So a merge would then only add mergeprops to root folder and modules/contrib (in branch). This is the explanation for the behavior: SVN needs to add the mergeprops to all single files that *already* contain mergeprops on each merge (SVN is not able to only store diffs in per file props). This is the reason why we have so many mergeprops at single files that every time gets updated: Once a file has a merge property, it does no longer inherit it from the parent folder. Whenever you merge something, the new rev numbers get added to the parent folder, as expected; but as this single file also have mergeprops and those are not differential/separate to parent, the merge prop of this single file also needs to be updated. This leads to the large modification email. We cannot change that anymore, so by reducing the number of single-file merges, we can stop this behavior from getting worse. To get rid of it completely and start new, see above. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Robert Muir [mailto:rcm...@gmail.com] Sent: Sunday, October 17, 2010 4:58 PM To: dev@lucene.apache.org Subject: possible to filter the output to commits@ list Lets say i change a single line of code, and merge it back to the 3x_branch. Currently we get 6 or 7 emails of mergeproperty changes to the commits@ list... this is making it difficult or impossible for backports to be reviewed at all... I think this is terrible. How can we get this fixed so that these mergeprop-changes only are filtered in the emails? Someone could always click the link to viewvc or whatever if they are interested in this... - To unsubscribe, e-mail:
