[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17318375#comment-17318375 ] ASF subversion and git services commented on LUCENE-9705: - Commit c5876771503e22968d7cf9be37a56eca68e5e4a5 in lucene's branch refs/heads/main from Julie Tibshirani [ https://gitbox.apache.org/repos/asf?p=lucene.git;h=c587677 ] LUCENE-9705: Correct the format names in Lucene90StoredFieldsFormat (#74) We accidentally kept the old names when creating the new format. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Assignee: Ignacio Vera >Priority: Major > Fix For: main (9.0) > > Time Spent: 11.5h > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314453#comment-17314453 ] ASF subversion and git services commented on LUCENE-9705: - Commit 67a0bd4b6df53b81d1108b8349854d400ff27f22 in lucene's branch refs/heads/main from Ignacio Vera [ https://gitbox.apache.org/repos/asf?p=lucene.git;h=67a0bd4 ] LUCENE-9705: Final clean-up and entry in CHANGES.txt (#59) > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 11h 20m > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312863#comment-17312863 ] ASF subversion and git services commented on LUCENE-9705: - Commit 8c9b9546ccf63ba8433a096ed0712fc33b5d529d in lucene's branch refs/heads/main from Ignacio Vera [ https://gitbox.apache.org/repos/asf?p=lucene.git;h=8c9b954 ] LUCENE-9705: Create Lucene90PointsFormat (#52) > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 10h 40m > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17311249#comment-17311249 ] ASF subversion and git services commented on LUCENE-9705: - Commit 00e57f8c8a42b9814aada2005457c6f1a00eeea4 in lucene's branch refs/heads/main from Ignacio Vera [ https://gitbox.apache.org/repos/asf?p=lucene.git;h=00e57f8 ] LUCENE-9705: Create Lucene90SegmentInfoFormat (#30) The existing Lucene86SegmentInfoFormat is moved to backwards-codecs. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 9h 50m > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299868#comment-17299868 ] Greg Miller commented on LUCENE-9705: - Thanks [~jpountz] for the link (and [~julietibs] for creating the README)! Makes complete sense to me. It sounds like my reference to the "utility" classes is already well thought out (and documented in the README) as the "helper classes" that must be copied. It's too bad there's not a way to make these helper classes common between the backwards compatible codecs and the current codecs while still maintaining package privacy. I've always thought it would be nice to have "inherited package private" access modifier in java that would allow package-visible classes to be visible in all sub-packages as well. If something like that existed, you could image a structure like the following where utility classes could exist at a top level and not need to be duplicated: o.a.l.codecs.ForUtil.java (still package-private) o.a.l.codecs.lucene90.Lucene90PostingsReader.java o.a.l.codecs.backwards_codecs.lucene84.Lucene84PostingsReader.java But, given current choices, I agree that duplication is the right trade-off to maintain package privacy. It seems I've wandered a little far off into the weeds now, so I'll leave it there. Thanks for entertaining my meandering rant :) > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 9.5h > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299342#comment-17299342 ] Adrien Grand commented on LUCENE-9705: -- bq. I'm not sure I would propose anything different (package-privacy is nice) +1 to make sure that utility classes for these file formats are pkg-private, this doesn't seem to be always the case today and I'm not seeing a reason why they couldn't be made pkg-private? bq. I'm not very familiar with all the different codec classes here or the process FYI the process is now documented at https://github.com/apache/lucene/blob/main/lucene/backward-codecs/README.md thanks to [~jtibshirani]. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 9.5h > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299167#comment-17299167 ] Greg Miller commented on LUCENE-9705: - I'm not very familiar with all the different codec classes here or the process, but I think there are some cases at least where classes get duplicated even though there's no change right? Sort of to Mike S.'s point (if I'm understanding at least). This is maybe limited to package-private utility stuff? So maybe it's not worth splitting hairs over, but I've at least seen duplication (with no change) in the PForUtil/ForUtil/ForDeltaUtil classes. I'm not sure I would propose anything different (package-privacy is nice), but it is a little bit of a rough edge with the current solution. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 9.5h > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17297875#comment-17297875 ] ASF subversion and git services commented on LUCENE-9705: - Commit 144ef2a0c054b54ee533f5618f36651931825f7d in lucene-solr's branch refs/heads/master from Ignacio Vera [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=144ef2a ] LUCENE-9705: Create Lucene90StoredFieldsFormat (#2444) > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 9.5h > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17289828#comment-17289828 ] ASF subversion and git services commented on LUCENE-9705: - Commit f8be421ae1f4d9d021b71c7ec4e060413b951363 in lucene-solr's branch refs/heads/master from Ignacio Vera [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f8be421 ] LUCENE-9705: Create Lucene90TermVectorsFormat (#2334) > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 7.5h > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[
https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17288862#comment-17288862
]
David Smiley commented on LUCENE-9705:
--
Solr tests started failing:
{noformat}
2> 422680 INFO
(TEST-TestCodecSupport.testDynamicFieldsDocValuesFormats-seed#[419213DD9EC7422B])
[ ] o.a.s.SolrTestCaseJ4 ###Ending testDynamicFieldsDocValuesFormats
> org.junit.ComparisonFailure: expected: but was:
> at
__randomizedtesting.SeedInfo.seed([419213DD9EC7422B:A1414D1B8859F47F]:0)
> at org.junit.Assert.assertEquals(Assert.java:117)
> at org.junit.Assert.assertEquals(Assert.java:146)
> at
org.apache.solr.core.TestCodecSupport.testDynamicFieldsDocValuesFormats(TestCodecSupport.java:89)
{noformat}
Reported here: SOLR-15181
> Move all codec formats to the o.a.l.codecs.Lucene90 package
> ---
>
> Key: LUCENE-9705
> URL: https://issues.apache.org/jira/browse/LUCENE-9705
> Project: Lucene - Core
> Issue Type: Wish
>Reporter: Ignacio Vera
>Priority: Major
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> Current formats are distributed in different packages, prefixed with the
> Lucene version they were created. With the upcoming release of Lucene 9.0, it
> would be nice to move all those formats to just the o.a.l.codecs.Lucene90
> package (and of course moving the current ones to the backwards-codecs).
> This issue would actually facilitate moving the directory API to little
> endian (LUCENE-9047) as the only codecs that would need to handle backwards
> compatibility will be the codecs in backwards codecs.
> In addition, it can help formalising the use of internal versions vs format
> versioning ( LUCENE-9616)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17288594#comment-17288594 ] ASF subversion and git services commented on LUCENE-9705: - Commit 4d7b2aebfe01e2a8d0407c179238bb5e092b8d2a in lucene-solr's branch refs/heads/master from Julie Tibshirani [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4d7b2ae ] LUCENE-9705: Create Lucene90DocValuesFormat and Lucene90NormsFormat (#2392) For now these are just copies of Lucene80DocValuesFormat and Lucene80NormsFormat. The existing formats were moved to backwards-codecs. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 6h 50m > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17288565#comment-17288565 ] ASF subversion and git services commented on LUCENE-9705: - Commit f43fe7642e9f1eb48b7438e2b132d9efc641edb6 in lucene-solr's branch refs/heads/master from Julie Tibshirani [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f43fe76 ] LUCENE-9705: Create Lucene90PostingsFormat (#2310) For now this is just a copy of Lucene90PostingsFormat. The existing Lucene84PostingsFormat was moved to backwards-codecs, along with its utility classes. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 6h 40m > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17286161#comment-17286161 ] Julie Tibshirani commented on LUCENE-9705: -- [~ivera] I opened PRs for the following formats: Lucene50LiveDocsFormat, Lucene80DocValuesFormat, Lucene80NormsFormat, Lucene84PostingsFormat. I don't plan to work on more, but will keep a lookout for code reviews. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 6h 10m > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17283195#comment-17283195 ] ASF subversion and git services commented on LUCENE-9705: - Commit 096f054d562978a768d346f66b50332c686919a0 in lucene-solr's branch refs/heads/master from Julie Tibshirani [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=096f054 ] LUCENE-9705: Reset internal version in Lucene90FieldInfosFormat. (#2339) Since this is a fresh format, we can remove older version logic and reset the internal version to 0. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 4h 50m > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281617#comment-17281617 ] ASF subversion and git services commented on LUCENE-9705: - Commit eafeb6643408e7e978f2fcb8d456b5eb3ca9c187 in lucene-solr's branch refs/heads/master from Ignacio Vera [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=eafeb66 ] LUCENE-9705: Move Lucene50CompoundFormat to Lucene90CompoundFormat (#2268) > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 3h 40m > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279059#comment-17279059 ] ASF subversion and git services commented on LUCENE-9705: - Commit f0a2f1fe0398a96c6d12959de7069c588395b8af in lucene-solr's branch refs/heads/master from Julie Tibshirani [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f0a2f1f ] LUCENE-9705: Create Lucene90LiveDocsFormat (#2274) For now this is just a copy of Lucene50LiveDocsFormat. The existing Lucene50LiveDocsFormat was moved to backwards-codecs. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 1h 50m > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17277050#comment-17277050 ] Ignacio Vera commented on LUCENE-9705: -- Thanks Julie, I think you capture the spirit of this change. In addition once we have new formats, we can try to simplify things, for example getting rid of PackedInts (legacy) in all current codecs in favour of DirectReader and DirectWriter. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[
https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276599#comment-17276599
]
Julie Tibshirani commented on LUCENE-9705:
--
{quote}It's especially clear here where we must copy a lot of classes with no
change at all, merely to clearly and consistently document the index version
change.
{quote}
I’ll try to add some context since I suspect there might be misunderstanding.
In general when there is a new major version, we *do not* plan to create all
new index format classes. We only copy a class and move it to backwards-codecs
when there is a change to that specific format, for example {{PointsFormat}}.
This proposal applies only to the 9.0 release, and its main purpose is to
support the work in LUCENE-9047 to move all formats to little endian. My
understanding is that moving to little endian impacts all the formats and will
be much cleaner if we used these fresh {{Lucene90*Format}}.
{quote}I wonder if we (eventually) should consider shifting to a versioning
system that doesn't require new classes. Is this somehow a feature of the
service discovery API that we use?
{quote}
We indeed load codecs (with their formats) through a service discovery API. If
a user wants to read indices from a previous major version, they can depend on
backwards-codecs so Lucene loads the correct older codec. As of LUCENE-9669, we
allow reading indices back to version N-2.
I personally really like the current "copy-on-write" system for formats.
There’s code duplication, but it has advantages over combining different
version logic in the same file:
* It’s really clear how each version behaves. Having a direct copy like
\{{Lucene70Codec} is almost as if we were pulling in the codec jars from Lucene
7.0.
* It decreases risk of introducing bugs or accidental changes. If you’re
making an enhancement to a new format, there’s little chance of changing the
logic for an old format (since it lives in a separate class). This is
especially important since older formats are not tested as thoroughly.
I started to appreciate it after experiencing the alternative in Elasticsearch,
where we’re constantly bumping into if/ else version checks when making changes.
> Move all codec formats to the o.a.l.codecs.Lucene90 package
> ---
>
> Key: LUCENE-9705
> URL: https://issues.apache.org/jira/browse/LUCENE-9705
> Project: Lucene - Core
> Issue Type: Wish
>Reporter: Ignacio Vera
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Current formats are distributed in different packages, prefixed with the
> Lucene version they were created. With the upcoming release of Lucene 9.0, it
> would be nice to move all those formats to just the o.a.l.codecs.Lucene90
> package (and of course moving the current ones to the backwards-codecs).
> This issue would actually facilitate moving the directory API to little
> endian (LUCENE-9047) as the only codecs that would need to handle backwards
> compatibility will be the codecs in backwards codecs.
> In addition, it can help formalising the use of internal versions vs format
> versioning ( LUCENE-9616)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276575#comment-17276575 ] Michael Sokolov commented on LUCENE-9705: - Just throwing this out there; I have no real proposal, just a feeling, but it seems very heavyweight that we create a new package and new java classes every time we change our index format. It's especially clear here where we must copy a lot of classes with no change at all, merely to clearly and consistently document the index version change. I noticed that we also have to copy (and slightly change) the package-level javadocs when we do this, and this has been done pretty inconsistently over time. I wonder if we (eventually) should consider shifting to a versioning system that doesn't require new classes. Is this somehow a feature of the service discovery API that we use? > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[
https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276214#comment-17276214
]
Ignacio Vera commented on LUCENE-9705:
--
{quote}for Lucene 9.1 we come up with some new improvement that changes one of
the index formats does that mean that *all* of the formats have to be moved
again?
{quote}
No, doing this change does not change the way we are adding new formats /
modifying existing formats.
{quote}I thought it was nice having old formats easily distinguishable
{quote}
That way of distinguish formats might be misleading. There are currently two
methods to add changes to an index 1) by adding a new format and moving the old
one to backwards codec like we are doing here 2) increasing the internal
version of the index. So it might happen that a new format is not
distinguishable from the previous one except by reading the actual index.
> Move all codec formats to the o.a.l.codecs.Lucene90 package
> ---
>
> Key: LUCENE-9705
> URL: https://issues.apache.org/jira/browse/LUCENE-9705
> Project: Lucene - Core
> Issue Type: Wish
>Reporter: Ignacio Vera
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Current formats are distributed in different packages, prefixed with the
> Lucene version they were created. With the upcoming release of Lucene 9.0, it
> would be nice to move all those formats to just the o.a.l.codecs.Lucene90
> package (and of course moving the current ones to the backwards-codecs).
> This issue would actually facilitate moving the directory API to little
> endian (LUCENE-9047) as the only codecs that would need to handle backwards
> compatibility will be the codecs in backwards codecs.
> In addition, it can help formalising the use of internal versions vs format
> versioning ( LUCENE-9616)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275415#comment-17275415 ] Mike Drob commented on LUCENE-9705: --- Right, I get that there are new codecs all the time. What I'm trying to understand is that if we do this now, and then for Lucene 9.1 we come up with some new improvement that changes one of the index formats does that mean that *all* of the formats have to be moved again? I thought it was nice having old formats easily distinguishable, so that a developer can look and see more easily what's new and what's not. At the end of the day, I trust you to do the right thing, please don't take this as me trying to hold up the improvements. I want to make sure that if we're addressing technical debt, we're actually addressing it and not replacing it with other (potentially prettier?) technical debt. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274514#comment-17274514 ] Ignacio Vera commented on LUCENE-9705: -- There should be no implication for the users. We are constantly introducing new codecs, normally due to a [change in one the underlaying index formats|https://github.com/apache/lucene-solr/pull/973] or a [new index format|https://github.com/apache/lucene-solr/pull/1930]. In Lucene 9.0 we are introducing already a new codec because we are introducing the new Vectors format. We are only suggesting to create a new version for all existing formats in this new codec so we can 1) hopefully eliminate technical debt 2) it will help with LUCENE-9047. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274460#comment-17274460 ] Mike Drob commented on LUCENE-9705: --- I’m confused about the compatibility implications here. Would codecs have to be repackaged every release going forward? That sounds like potentially a huge upgrade pain for users (or maybe not, if nobody specifies codecs directly) > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274241#comment-17274241 ] Ignacio Vera commented on LUCENE-9705: -- Just to keep track here are the formats that need to be moved to Lucene90 package: * Lucene50CompoundFormat * Lucene50LiveDocsFormat * Lucene50TermVectorsFormat * Lucene80DocValuesFormat * Lucene80NormsFormat * Lucene84PostingsFormat * Lucene86PointsFormat * Lucene87StoredFieldsFormat > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[ https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273995#comment-17273995 ] Adrien Grand commented on LUCENE-9705: -- +1 to split all file formats that are used by Lucene90Codec into one that lives in lucene/backward-codecs unchanged, and another one that lives in the oal.codecs.lucene90 package of lucene/core and only preserves the latest version. > Move all codec formats to the o.a.l.codecs.Lucene90 package > --- > > Key: LUCENE-9705 > URL: https://issues.apache.org/jira/browse/LUCENE-9705 > Project: Lucene - Core > Issue Type: Wish >Reporter: Ignacio Vera >Priority: Major > > Current formats are distributed in different packages, prefixed with the > Lucene version they were created. With the upcoming release of Lucene 9.0, it > would be nice to move all those formats to just the o.a.l.codecs.Lucene90 > package (and of course moving the current ones to the backwards-codecs). > This issue would actually facilitate moving the directory API to little > endian (LUCENE-9047) as the only codecs that would need to handle backwards > compatibility will be the codecs in backwards codecs. > In addition, it can help formalising the use of internal versions vs format > versioning ( LUCENE-9616) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package
[
https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273903#comment-17273903
]
Julie Tibshirani commented on LUCENE-9705:
--
This proposal makes sense to me. I'd be happy to help with the effort if you'd
like to split up work.
bq. In addition, it can help formalising the use of internal versions vs format
versioning ( LUCENE-9616)
Do we plan to separate out the logic more clearly, and avoid sharing big chunks
of logic across format versions (as we do now for example with
{{CompressingStoredFieldsReader}})? If so we could take the opportunity to
'start fresh' and reset all internal format versions. Going forward, we could
have a clear policy that for any significant change, we should create a new
format and copy the old one to backwards-codecs instead of using an internal
version. (Perhaps we'll even find that internal versions are not useful in some
places and choose to remove them?)
> Move all codec formats to the o.a.l.codecs.Lucene90 package
> ---
>
> Key: LUCENE-9705
> URL: https://issues.apache.org/jira/browse/LUCENE-9705
> Project: Lucene - Core
> Issue Type: Wish
>Reporter: Ignacio Vera
>Priority: Major
>
> Current formats are distributed in different packages, prefixed with the
> Lucene version they were created. With the upcoming release of Lucene 9.0, it
> would be nice to move all those formats to just the o.a.l.codecs.Lucene90
> package (and of course moving the current ones to the backwards-codecs).
> This issue would actually facilitate moving the directory API to little
> endian (LUCENE-9047) as the only codecs that would need to handle backwards
> compatibility will be the codecs in backwards codecs.
> In addition, it can help formalising the use of internal versions vs format
> versioning ( LUCENE-9616)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
