[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238510#comment-14238510 ] Michael McCandless commented on LUCENE-4335: I think it's really important that we test ant regenerate in Jenkins: we have many, many places that generate complex code, and users are finding problems (LUCENE-6102). Maybe for this issue, it's too ambitious for now to insist on no svn diffs after 'ant regenerate' ... can we relax this to say ant test should pass after ant regenerate and ignore any svn diffs (for a new, special Jenkins build) for starters? Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238514#comment-14238514 ] Robert Muir commented on LUCENE-4335: - +1 ! We have to make some small progress. Just the other day i fixed 'ant regenerate' to actually create code for lucene/core that will actually compile. I think a job that just runs 'regenerate test' (but no svn status check afterwards) is a good interim solution. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238643#comment-14238643 ] Mike Drob commented on LUCENE-4335: --- I might be missing some of the more historical context, but why are generated sources part of SVN? I've seen other projects make compile depend on generate and that way there is no risk of attempting to edit generated files. The disadvantages of this approach are increased build time and possibly not as portable across platforms, but I think those can both be worked through given enough time. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238655#comment-14238655 ] Robert Muir commented on LUCENE-4335: - There are a ton of reasons why: * actually working with a modern IDE is important. * some are very slow, for example kuromoji compacts a segmentation model into data files. * others require tools like icu4c, python, mercurial, all kinds of stuff which must be installed on the machine. Its not feasible, and losing portability is not an option. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238681#comment-14238681 ] Robert Muir commented on LUCENE-4335: - Well the easiest way would be to speed up the current regeneration? It currently takes 47 seconds. But honestly I don't have a lot of confidence this stuff can be that much faster faster. Its not like we are making it intentionally ineffecient. Look at what some of the regenerators are doing... packing large FSTs of tokenization models, minimizing large jflex DFAs, etc. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[
https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945198#comment-13945198
]
Steve Rowe commented on LUCENE-4335:
As of the JFlex 1.5.1 upgrade (LUCENE-5552), the only changes I see after
running {{ant regenerate}} at the top level are in the queryparser module:
{noformat}
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/CharStream.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/ParseException.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/QueryParser.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/Token.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/TokenMgrError.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/parser/CharStream.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/parser/ParseException.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/parser/StandardSyntaxParser.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/parser/Token.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/parser/TokenMgrError.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/surround/parser/CharStream.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/surround/parser/ParseException.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/surround/parser/QueryParser.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/surround/parser/Token.java
M
lucene/queryparser/src/java/org/apache/lucene/queryparser/surround/parser/TokenMgrError.java
{noformat}
Most of these are diamond operator issues: the generated source was manually
converted to use the diamond operator, but the corresponding {{.jj}} files were
not. I removed the appropriate explicit types in the {{.jj}} files and ran
{{ant regenerate}}, but JavaCC 5.0 doesn't like it:
{noformat}
javacc-QueryParser:
[javacc] Java Compiler Compiler Version 5.0 (Parser Generator)
[javacc] (type javacc with no arguments for help)
[javacc] Reading from file
/Users/sarowe/svn/lucene/dev/trunk7/lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/QueryParser.jj
. . .
[javacc] org.javacc.parser.ParseException: Encountered at line
225, column 47.
[javacc] Was expecting one of:
[javacc] boolean ...
[javacc] byte ...
[javacc] char ...
[javacc] double ...
[javacc] float ...
[javacc] int ...
[javacc] long ...
[javacc] short ...
[javacc] ? ...
[javacc] IDENTIFIER ...
[javacc]
[javacc] Detected 1 errors and 0 warnings.
{noformat}
I see JavaCC 6.0 was recently released - maybe it can handle the diamond
operator?
One other problem with some JavaCC-generated sources: the checksum seems to
have somehow changed, even though nothing else has? - e.g. for the classic
queryparser's {{CharStream.java}}:
{noformat}
Index:
lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/CharStream.java
===
---
lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/CharStream.java
(revision 1580832)
+++
lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/CharStream.java
(working copy)
@@ -112,4 +112,4 @@
void Done();
}
-/* JavaCC - OriginalChecksum=c847dd1920bf7901125a7244125682ad (do not edit
this line) */
+/* JavaCC - OriginalChecksum=30b94cad7b10d0d81e3a59a1083939d0 (do not edit
this line) */
{noformat}
One last thing: I accidentally ran {{ant regenerate}} using Java8, and the
supplementary character jflex macro files output by the icu module changed, and
this caused the JFlex-generated scanner classes to change too. On cursory
inspection, some lines are reordered, but I wouldn't think that would trigger
scanner class changes. At a minimum, the output should be changed to have a
fixed ordering.
Builds should regenerate all generated sources
--
Key: LUCENE-4335
URL: https://issues.apache.org/jira/browse/LUCENE-4335
Project: Lucene - Core
Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch
We have more and more sources that are generated programmatically (query
parsers, fuzzy levN tables from Moman, packed ints specialized decoders,
etc.), and it's dangerous because developers may directly edit the generated
sources and forget to edit the meta-source. It's
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945214#comment-13945214 ] Uwe Schindler commented on LUCENE-4335: --- bq. One other problem with some JavaCC-generated sources: the checksum seems to have somehow changed, even though nothing else has? - e.g. for the classic queryparser's CharStream.java: This is because the checksum is generated on the binary input file. As *I* regenerated the files the last time and I have Windows CR-LF as line separator, the checksum was different. If you run JavaCC on Linux afterwards, the file checksum changes, unfortunately. I know about this problem, but I have no idea how to fix. I would remove the checkum from the files completely after regenerating (using a regex). We already have many regex replaces, this is just one more. bq. I see JavaCC 6.0 was recently released - maybe it can handle the diamond operator? I would simply let JavaCC use old-style generics. We have no must to use diamonds. If generated code uses conventional declarations, it is no problem at all. If we want to upgrade to JavaCC 6.0, we should carefully compare its output. If its identical, I have no problem with upgrading (if its available in Maven Central). Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945215#comment-13945215 ] Uwe Schindler commented on LUCENE-4335: --- bq. One last thing: I accidentally ran ant regenerate using Java8, and the supplementary character jflex macro files output by the icu module changed, and this caused the JFlex-generated scanner classes to change too. On cursory inspection, some lines are reordered, but I wouldn't think that would trigger scanner class changes. At a minimum, the output should be changed to have a fixed ordering. Java 8 has a different hashing algorithm for string keys... The usual problem. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13817683#comment-13817683 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1540187 from [~rjernst] in branch 'dev/trunk' [ https://svn.apache.org/r1540187 ] LUCENE-4335: Add Namespaces to Expressions Javascript Compiler Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13722974#comment-13722974 ] Steve Rowe commented on LUCENE-4335: bq. I don't want to setup a fixed JFlex on Jenkins, I want to download it with IVY, so before resolving this issue we should have a JFlex version available. If Steve Rowe is not able to relaese the version on Maven, we should maybe fork jflex on Google Code and make a release including the ANT task. I can't promise I'll release JFlex anytime soon, sorry. If you want to fork, you can certainly do that. FYI, Gerwin Klein, the JFlex founder, has done some work (maybe all that needs to be done? not sure at this point) to convert JFlex to a BSD license. I'll review the source and see what state that effort is in - BSD licensing should simplify forking, I think. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718148#comment-13718148 ] Uwe Schindler commented on LUCENE-4335: --- bq. But now I think we need to edit Uwe Schindler's groovy script to be a macro that fails also if any files were modified. If we change the top-level task that runs on ant validate, the problem with that is that you are then no longer be able to run validate on a modified checkout before committing. But I think you are thinking of running this check only on the generated files? We could create a separate svnkit macro that does a before/after check. I am thinking about a groovy script that runs a check for modified files, saves that information in a Set?, then calls a subant with the regenerate task and runs the macro code again, this time adding to a different set. If the Sets are not identical something has changed :-) I can assist with that! Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718237#comment-13718237 ] Robert Muir commented on LUCENE-4335: - Uwe take a look at the branch. I didn't change the top-level task that runs on validate. i only changed the jenkins task. precommit still does the same checks as before. jenkins should not have modified files in any way... Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718239#comment-13718239 ] Robert Muir commented on LUCENE-4335: - The thing stopping us from merging this branch to trunk right now is the jenkins configuration. To run regenerate, jenkins needs to have the correct versions installed of: * mercurial (hg) * javacc * jflex * icu4c Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718251#comment-13718251 ] Uwe Schindler commented on LUCENE-4335: --- Hi Robert, that was an alternattive way to fix the precommit Task by a more intelligent approach. In general my current problem is: I don't want to setup a fixed JFlex on Jenkins, I want to download it with IVY, so before resolving this issue we should have a JFlex version available. If [~steve_rowe] is not able to relaese the version on Maven, we should maybe fork jflex on Google Code and make a release including the ANT task. The second problem in: Moman - I dont want to have Mercurial (hg) on Jenkins, this makes the setup much worse. Python on Windows is terrible already. Ideally, the sources generated by Python should be converted to a ant script task using jython. This would make setup easier. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[
https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718255#comment-13718255
]
Uwe Schindler commented on LUCENE-4335:
---
{quote}
- javacc
{quote}
JavaCC is already downloaded from IVY in ANT
Builds should regenerate all generated sources
--
Key: LUCENE-4335
URL: https://issues.apache.org/jira/browse/LUCENE-4335
Project: Lucene - Core
Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
Attachments: LUCENE-4335.patch, LUCENE-4335.patch
We have more and more sources that are generated programmatically (query
parsers, fuzzy levN tables from Moman, packed ints specialized decoders,
etc.), and it's dangerous because developers may directly edit the generated
sources and forget to edit the meta-source. It's happened to me several
times ... most recently just after landing the BlockPostingsFormat branch.
I think we should re-gen all of these in our builds and fail the build if
this creates a difference. I know some generators (eg JavaCC) embed
timestamps and so always create mods ... we can leave them out of this for
starters (or maybe post-process the sources to remove the timestamps) ...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718256#comment-13718256 ] Robert Muir commented on LUCENE-4335: - Uwe: right, well then we can disable the jenkins task and merge this to trunk without the check. I dont think we should block this issue on shit like jflex releases (I look at my mailing list, last discussion about this was in *2009*) and this isnt my fault. Ill back out the jenkins check, merge this to trunk, and open a new issue. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718272#comment-13718272 ] Uwe Schindler commented on LUCENE-4335: --- bq. Ill back out the jenkins check, merge this to trunk, and open a new issue. Can you provide a patch here, I just wanted to have a quick look! Otherwise if we leave out jenkins from the game at the moment and fix that in later issues and talk with [~steve_rowe] about releasing or forking JFlex. The Moan stuff should maybe downloaded as a ZIP file from the specific HG version (e.g. from Bitbucket as ZIP file using the commit hash) and unzipped. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718284#comment-13718284 ] Robert Muir commented on LUCENE-4335: - That can be in a new issue too. Its unrelated to what we are doing here. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718295#comment-13718295 ] Uwe Schindler commented on LUCENE-4335: --- +1 looks good We can fix the remaing stuff on trunk. I am currently working on removing the HG clone. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718299#comment-13718299 ] Michael McCandless commented on LUCENE-4335: +1, patch looks great! Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718302#comment-13718302 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1506516 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1506516 ] LUCENE-4335: ant regenerate Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718328#comment-13718328 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1506533 from [~thetaphi] in branch 'dev/trunk' [ https://svn.apache.org/r1506533 ] LUCENE-4335: Make moman not use HG anymore. Just download as ZIP from bitbucket and unzip Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718347#comment-13718347 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1506542 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1506542 ] LUCENE-4335: ant regenerate Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718364#comment-13718364 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1506548 from [~thetaphi] in branch 'dev/trunk' [ https://svn.apache.org/r1506548 ] LUCENE-4335: Fix the bug with modifications on the SVN root folder Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718367#comment-13718367 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1506549 from [~thetaphi] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1506549 ] Merged revision(s) 1506533 from lucene/dev/trunk: LUCENE-4335: Make moman not use HG anymore. Just download as ZIP from bitbucket and unzip Merged revision(s) 1506548 from lucene/dev/trunk: LUCENE-4335: Fix the bug with modifications on the SVN root folder Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718368#comment-13718368 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1506549 from [~thetaphi] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1506549 ] Merged revision(s) 1506533 from lucene/dev/trunk: LUCENE-4335: Make moman not use HG anymore. Just download as ZIP from bitbucket and unzip Merged revision(s) 1506548 from lucene/dev/trunk: LUCENE-4335: Fix the bug with modifications on the SVN root folder Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717513#comment-13717513 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1506240 from [~mikemccand] in branch 'dev/branches/lucene4335' [ https://svn.apache.org/r1506240 ] LUCENE-4335: commit current patch Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717511#comment-13717511 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1506234 from [~mikemccand] in branch 'dev/branches/lucene4335' [ https://svn.apache.org/r1506234 ] LUCENE-4335: make branch Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717515#comment-13717515 ] Michael McCandless commented on LUCENE-4335: OK I made a branch https://svn.apache.org/repos/asf/lucene/dev/branches/lucene4335 and committed the last (broken, but a starting point) patch ... Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717539#comment-13717539 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1506248 from [~mikemccand] in branch 'dev/branches/lucene4335' [ https://svn.apache.org/r1506248 ] LUCENE-4335: add empty target in common-build.xml Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717567#comment-13717567 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1506258 from [~mikemccand] in branch 'dev/branches/lucene4335' [ https://svn.apache.org/r1506258 ] LUCENE-4335: fix generators to match recent code changes to the gen'd files Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717620#comment-13717620 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1506281 from [~mikemccand] in branch 'dev/branches/lucene4335' [ https://svn.apache.org/r1506281 ] LUCENE-4335: add -r 623 to instructions for checking out jflex Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717625#comment-13717625 ] Robert Muir commented on LUCENE-4335: - Cool Mike: regenerate seems to be working! But now I think we need to edit [~thetaphi]'s groovy script to be a macro that fails also if any files were modified. We should use this for verifying the regenerated sources have not changed. I think we should also use this in jenkins after running tests. The precommit test can keep it off as it does now, but jenkins can be more strict. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717627#comment-13717627 ] ASF subversion and git services commented on LUCENE-4335: - Commit 1506284 from [~mikemccand] in branch 'dev/branches/lucene4335' [ https://svn.apache.org/r1506284 ] LUCENE-4335: don't regenerate for precommit Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-4335.patch, LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[
https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717740#comment-13717740
]
Robert Muir commented on LUCENE-4335:
-
{code}
regenerateAndCheck:
BUILD SUCCESSFUL
Total time: 57 seconds
{code}
Builds should regenerate all generated sources
--
Key: LUCENE-4335
URL: https://issues.apache.org/jira/browse/LUCENE-4335
Project: Lucene - Core
Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
Attachments: LUCENE-4335.patch, LUCENE-4335.patch
We have more and more sources that are generated programmatically (query
parsers, fuzzy levN tables from Moman, packed ints specialized decoders,
etc.), and it's dangerous because developers may directly edit the generated
sources and forget to edit the meta-source. It's happened to me several
times ... most recently just after landing the BlockPostingsFormat branch.
I think we should re-gen all of these in our builds and fail the build if
this creates a difference. I know some generators (eg JavaCC) embed
timestamps and so always create mods ... we can leave them out of this for
starters (or maybe post-process the sources to remove the timestamps) ...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13610903#comment-13610903 ] Commit Tag Bot commented on LUCENE-4335: [branch_4x commit] Michael McCandless http://svn.apache.org/viewvc?view=revisionrevision=1381702 LUCENE-4335: checkout Moman under build dir Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Attachments: LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13445966#comment-13445966 ] Steven Rowe commented on LUCENE-4335: - bq. I've seen JFlex reorder cases in switch statements, even when there are no .jflex source changes, when run under different JVM versions. I recall seeing this specifically when generating under Java5 and Java6, both Oracle JVMs on Windows. Dawid sent me a patch to use LinkedHashMaps instead of HashMaps for the data structures emitted as switch cases, so insertion order will be the emit order. I committed Dawid's patch to JFlex trunk r614. Please update and rebuild to get the change. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Attachments: LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13445978#comment-13445978 ] Robert Muir commented on LUCENE-4335: - is there a possibility of a jflex release in the future? It would be nice to regenerate it via IVY like javacc. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Attachments: LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13446083#comment-13446083 ] Steven Rowe commented on LUCENE-4335: - bq. is there a possibility of a jflex release in the future? It would be nice to regenerate it via IVY like javacc. I agree, it would be nice for several reasons to make JFlex downloadable via IVY. I had planned on working toward a JFlex release this summer, but haven't done any work on it yet. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Attachments: LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13446089#comment-13446089 ] Robert Muir commented on LUCENE-4335: - Cool, i think it would be convenient for the long term: in the short term I think we can still try to make some progress here towards something along the lines of an 'ant regenerate'. I am thinking the easiest way is to just add a 'regenerate' task to common-build.xml that is a no-op by default, and then each module can do what it needs to do? Then we would just call it across the build. I think we should try to regenerate as much as possible (data too) when we do this: e.g. kuromoji would call 'build-dict', icu module would regenerate its stuff, and so on. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Attachments: LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13446096#comment-13446096 ] Steven Rowe commented on LUCENE-4335: - Generation of content that's static should be fine, but generation based on external content we don't control, e.g. rules matching TLDs generated for UAX29URLEmailTokenizer, shouldn't be a regular part of the build. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Attachments: LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13446101#comment-13446101 ] Robert Muir commented on LUCENE-4335: - Right: in cases like that we should just not include it in 'ant regenerate' I think, unless we can find a way to make it static. root TLDs have a registration date in the IANA database, and they don't ever get deleted right? Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Attachments: LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13446104#comment-13446104 ] Steven Rowe commented on LUCENE-4335: - Sorry, I don't know how the IANA database works. I'll look into it though. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Attachments: LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13446180#comment-13446180 ] Robert Muir commented on LUCENE-4335: - I'm not an expert either: I could be wrong about how it works. But still I think we can make progress, excluding things like this from 'regenerate' and just try to have regenerate regenerate as much as possible, hooking this into the nightly build or something like that. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Attachments: LUCENE-4335.patch We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13443427#comment-13443427 ] Robert Muir commented on LUCENE-4335: - I think we should use replaceRegexp commands (like that are already there) to remove the various system information (time, paths, etc) that jflex/javacc/etc add from the generated code. then we could have an 'ant regenerate' command that regens all sources, and our usual 'svn status' check would ensure nothing changed. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13443455#comment-13443455 ] Uwe Schindler commented on LUCENE-4335: --- Thats a good idea, there is one problem with one of the tools, not sure if jflex or javacc. It happens that one of these tools reorders the switch statement's case XX: labels and so creating different source. This seems to depend on JDK version used, if you regen again its the same, but often i changed the metafile (like fixing /** to /* for license) and regened, it was different order. The pattern looks like one of these tools use a HashSet/HashMap of case statements, where the order is undefined. We should check what causes this. bq. then we could have an 'ant regenerate' command that regens all sources, and our usual 'svn status' check would ensure nothing changed. We have to extend that one to also detect modifications. The current checker task only looks for unversioned files and checks properties. By this you can run it before commit. This one would need to check for mods, too. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[
https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13443464#comment-13443464
]
Robert Muir commented on LUCENE-4335:
-
{quote}
We should check what causes this.
{quote}
I agree, this is always scary when it happens. It makes it harder to tell if
something really changed.
Builds should regenerate all generated sources
--
Key: LUCENE-4335
URL: https://issues.apache.org/jira/browse/LUCENE-4335
Project: Lucene - Core
Issue Type: Improvement
Reporter: Michael McCandless
We have more and more sources that are generated programmatically (query
parsers, fuzzy levN tables from Moman, packed ints specialized decoders,
etc.), and it's dangerous because developers may directly edit the generated
sources and forget to edit the meta-source. It's happened to me several
times ... most recently just after landing the BlockPostingsFormat branch.
I think we should re-gen all of these in our builds and fail the build if
this creates a difference. I know some generators (eg JavaCC) embed
timestamps and so always create mods ... we can leave them out of this for
starters (or maybe post-process the sources to remove the timestamps) ...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4335) Builds should regenerate all generated sources
[ https://issues.apache.org/jira/browse/LUCENE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13443560#comment-13443560 ] Steven Rowe commented on LUCENE-4335: - I'm not sure about Javacc, but I've seen JFlex reorder cases in switch statements, even when there are no .jflex source changes, when run under different JVM versions. I recall seeing this specifically when generating under Java5 and Java6, both Oracle JVMs on Windows. I'll look into the generator to see how to fix the output order. Builds should regenerate all generated sources -- Key: LUCENE-4335 URL: https://issues.apache.org/jira/browse/LUCENE-4335 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless We have more and more sources that are generated programmatically (query parsers, fuzzy levN tables from Moman, packed ints specialized decoders, etc.), and it's dangerous because developers may directly edit the generated sources and forget to edit the meta-source. It's happened to me several times ... most recently just after landing the BlockPostingsFormat branch. I think we should re-gen all of these in our builds and fail the build if this creates a difference. I know some generators (eg JavaCC) embed timestamps and so always create mods ... we can leave them out of this for starters (or maybe post-process the sources to remove the timestamps) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
