[jira] [Created] (PYLUCENE-31) JCC Parallel/Multiprocess Compilation + Caching
Lee Skillen created PYLUCENE-31: --- Summary: JCC Parallel/Multiprocess Compilation + Caching Key: PYLUCENE-31 URL: https://issues.apache.org/jira/browse/PYLUCENE-31 Project: PyLucene Issue Type: Improvement Environment: Linux 3.11.0-19-generic #33-Ubuntu SMP x86_64 GNU/Linux Reporter: Lee Skillen Priority: Minor JCC utilises distutils.Extension() in order to build JCC itself and the packages that it generates for Java wrapping - Unfortunately distutils performs its build sequentially and doesn't take advantage of any additional free cores for parallel building. As discussed on the list this is likely a design decision due to potential issues that may arise when building projects with awkward, cyclic or recursive dependencies. These issues shouldn't appear within JCC-based projects because of the generative nature of the build; i.e. all dependencies are resolved and generated prior to building, and the build process itself is about compilation and construction of the wrapper alone, of which the wrapper files are contained to a sequence of flattened compilation units. Enabling this requires monkey patching of distutils, which was also discussed on the list as being a potential source of issues, although we feel that the risk is likely lower than the current setuptools patching utilised. This would be optional functionality that is also only enabled if the monkey-patching succeeds. Distutils itself is also part of the standard library and might be less susceptible to change than setuptools, and the area of code monkey patched almost hasn't changed since 2002 (see: http://hg.python.org/cpython/file/tip/Lib/distutils/ccompiler.py). In addition to the distutils changes this patch also includes changes to the wrapper class generation to make it more cache friendly, with the target being that no changes in the wrapped code equals no changes in the wrapper code. So any changes that minimally change the wrapped code mean that with a tool such as ccache the rebuild time would be significantly reduced (almost to a nth, where n is the number of files and only one has changed). Obviously the maintainers would have to assess this risk and decide whether they would like to accept the patch or not. Code has only been tested on Linux with Python 2.7.5 but should gracefully fail and prevent parallelisation if one of the requirements hasn't been met (not on linux, no multiprocessing support, or monkey patching somehow fails). The change to caching should still benefit everyone regardless. Please note that an additional dependency on orderedset has been added to achieve the more deterministic ordering - This may not be desirable (i.e. another package might be desired, such as ordered-set, or the code might be inlined into the package instead), as per maintainer comments. --- [following repeated from mailing list] --- Performance Statistics :- The following are some quick and dirty statistics for building the jcc pylucene itself (incl. java lucene which accounts for about 30-ish seconds upfront) - The JCC files are split using --files 8, and each build is preceded with a make clean: Serial (unpatched): real5m1.502s user5m22.887s sys 0m7.749s Parallel (patched, 4 physical cores, 8 hyperthreads, 8 parallel jobs): real1m37.382s user7m16.658s sys 0m8.697s Furthermore, some additional changes were made to the wrapped file generation to make the generated code more ccache friendly (additional deterministic sorting for methods and some usage of an ordered set). With these in place and the CC and CCACHE_COMPILERCHECK environment variables set to ccache gcc and content respectively, and ensuring ccache is installed, subsequent compilation time is reduced again as follows: Parallel (patched, 4 physical cores, 8 hyperthreads, 8 parallel jobs, ccache enabled): real0m43.051s user1m10.392s sys 0m4.547s This was a run in which nothing changed between runs, so a realistic run in which changes occur it'll be a figure between 0m43.051s and 1m37.382s, depending on how drastic the change was. If many changes are expected and you want to keep it more cache friendly then using a higher --files would probably work (to an extent), or ideally use --files separate, although it doesn't currently work for me (need to investigate). We're mostly utilising the PyLucene build as a test bed since it is repeatable for others, rather than just showing numbers for own application compilations; we also use it to run the unit test suite after changes to JCC itself to ensure it still works as intended for PyLucene. For illustrative purposes though our application takes 1m53s to compile with JCC from scratch serially, 0m31s in parallel (8 jobs), 0m14s in parallel with ccache enabled and minimal changes, and 0m8s with
[jira] [Updated] (PYLUCENE-31) JCC Parallel/Multiprocess Compilation + Caching
[ https://issues.apache.org/jira/browse/PYLUCENE-31?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lee Skillen updated PYLUCENE-31: Attachment: feature-parallel-build.patch JCC Parallel/Multiprocess Compilation + Caching --- Key: PYLUCENE-31 URL: https://issues.apache.org/jira/browse/PYLUCENE-31 Project: PyLucene Issue Type: Improvement Environment: Linux 3.11.0-19-generic #33-Ubuntu SMP x86_64 GNU/Linux Reporter: Lee Skillen Priority: Minor Labels: build, cache, ccache, distutils, jcc, parallel Attachments: feature-parallel-build.patch JCC utilises distutils.Extension() in order to build JCC itself and the packages that it generates for Java wrapping - Unfortunately distutils performs its build sequentially and doesn't take advantage of any additional free cores for parallel building. As discussed on the list this is likely a design decision due to potential issues that may arise when building projects with awkward, cyclic or recursive dependencies. These issues shouldn't appear within JCC-based projects because of the generative nature of the build; i.e. all dependencies are resolved and generated prior to building, and the build process itself is about compilation and construction of the wrapper alone, of which the wrapper files are contained to a sequence of flattened compilation units. Enabling this requires monkey patching of distutils, which was also discussed on the list as being a potential source of issues, although we feel that the risk is likely lower than the current setuptools patching utilised. This would be optional functionality that is also only enabled if the monkey-patching succeeds. Distutils itself is also part of the standard library and might be less susceptible to change than setuptools, and the area of code monkey patched almost hasn't changed since 2002 (see: http://hg.python.org/cpython/file/tip/Lib/distutils/ccompiler.py). In addition to the distutils changes this patch also includes changes to the wrapper class generation to make it more cache friendly, with the target being that no changes in the wrapped code equals no changes in the wrapper code. So any changes that minimally change the wrapped code mean that with a tool such as ccache the rebuild time would be significantly reduced (almost to a nth, where n is the number of files and only one has changed). Obviously the maintainers would have to assess this risk and decide whether they would like to accept the patch or not. Code has only been tested on Linux with Python 2.7.5 but should gracefully fail and prevent parallelisation if one of the requirements hasn't been met (not on linux, no multiprocessing support, or monkey patching somehow fails). The change to caching should still benefit everyone regardless. Please note that an additional dependency on orderedset has been added to achieve the more deterministic ordering - This may not be desirable (i.e. another package might be desired, such as ordered-set, or the code might be inlined into the package instead), as per maintainer comments. --- [following repeated from mailing list] --- Performance Statistics :- The following are some quick and dirty statistics for building the jcc pylucene itself (incl. java lucene which accounts for about 30-ish seconds upfront) - The JCC files are split using --files 8, and each build is preceded with a make clean: Serial (unpatched): real5m1.502s user5m22.887s sys 0m7.749s Parallel (patched, 4 physical cores, 8 hyperthreads, 8 parallel jobs): real1m37.382s user7m16.658s sys 0m8.697s Furthermore, some additional changes were made to the wrapped file generation to make the generated code more ccache friendly (additional deterministic sorting for methods and some usage of an ordered set). With these in place and the CC and CCACHE_COMPILERCHECK environment variables set to ccache gcc and content respectively, and ensuring ccache is installed, subsequent compilation time is reduced again as follows: Parallel (patched, 4 physical cores, 8 hyperthreads, 8 parallel jobs, ccache enabled): real0m43.051s user1m10.392s sys 0m4.547s This was a run in which nothing changed between runs, so a realistic run in which changes occur it'll be a figure between 0m43.051s and 1m37.382s, depending on how drastic the change was. If many changes are expected and you want to keep it more cache friendly then using a higher --files would probably work (to an extent), or ideally use --files separate, although it doesn't currently work for me (need to investigate). We're mostly utilising the PyLucene build as a test bed since it is repeatable for others, rather than just showing
[jira] [Updated] (PYLUCENE-31) JCC Parallel/Multiprocess Compilation + Caching
[ https://issues.apache.org/jira/browse/PYLUCENE-31?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lee Skillen updated PYLUCENE-31: Attachment: (was: feature-parallel-build.patch) JCC Parallel/Multiprocess Compilation + Caching --- Key: PYLUCENE-31 URL: https://issues.apache.org/jira/browse/PYLUCENE-31 Project: PyLucene Issue Type: Improvement Environment: Linux 3.11.0-19-generic #33-Ubuntu SMP x86_64 GNU/Linux Reporter: Lee Skillen Priority: Minor Labels: build, cache, ccache, distutils, jcc, parallel JCC utilises distutils.Extension() in order to build JCC itself and the packages that it generates for Java wrapping - Unfortunately distutils performs its build sequentially and doesn't take advantage of any additional free cores for parallel building. As discussed on the list this is likely a design decision due to potential issues that may arise when building projects with awkward, cyclic or recursive dependencies. These issues shouldn't appear within JCC-based projects because of the generative nature of the build; i.e. all dependencies are resolved and generated prior to building, and the build process itself is about compilation and construction of the wrapper alone, of which the wrapper files are contained to a sequence of flattened compilation units. Enabling this requires monkey patching of distutils, which was also discussed on the list as being a potential source of issues, although we feel that the risk is likely lower than the current setuptools patching utilised. This would be optional functionality that is also only enabled if the monkey-patching succeeds. Distutils itself is also part of the standard library and might be less susceptible to change than setuptools, and the area of code monkey patched almost hasn't changed since 2002 (see: http://hg.python.org/cpython/file/tip/Lib/distutils/ccompiler.py). In addition to the distutils changes this patch also includes changes to the wrapper class generation to make it more cache friendly, with the target being that no changes in the wrapped code equals no changes in the wrapper code. So any changes that minimally change the wrapped code mean that with a tool such as ccache the rebuild time would be significantly reduced (almost to a nth, where n is the number of files and only one has changed). Obviously the maintainers would have to assess this risk and decide whether they would like to accept the patch or not. Code has only been tested on Linux with Python 2.7.5 but should gracefully fail and prevent parallelisation if one of the requirements hasn't been met (not on linux, no multiprocessing support, or monkey patching somehow fails). The change to caching should still benefit everyone regardless. Please note that an additional dependency on orderedset has been added to achieve the more deterministic ordering - This may not be desirable (i.e. another package might be desired, such as ordered-set, or the code might be inlined into the package instead), as per maintainer comments. --- [following repeated from mailing list] --- Performance Statistics :- The following are some quick and dirty statistics for building the jcc pylucene itself (incl. java lucene which accounts for about 30-ish seconds upfront) - The JCC files are split using --files 8, and each build is preceded with a make clean: Serial (unpatched): real5m1.502s user5m22.887s sys 0m7.749s Parallel (patched, 4 physical cores, 8 hyperthreads, 8 parallel jobs): real1m37.382s user7m16.658s sys 0m8.697s Furthermore, some additional changes were made to the wrapped file generation to make the generated code more ccache friendly (additional deterministic sorting for methods and some usage of an ordered set). With these in place and the CC and CCACHE_COMPILERCHECK environment variables set to ccache gcc and content respectively, and ensuring ccache is installed, subsequent compilation time is reduced again as follows: Parallel (patched, 4 physical cores, 8 hyperthreads, 8 parallel jobs, ccache enabled): real0m43.051s user1m10.392s sys 0m4.547s This was a run in which nothing changed between runs, so a realistic run in which changes occur it'll be a figure between 0m43.051s and 1m37.382s, depending on how drastic the change was. If many changes are expected and you want to keep it more cache friendly then using a higher --files would probably work (to an extent), or ideally use --files separate, although it doesn't currently work for me (need to investigate). We're mostly utilising the PyLucene build as a test bed since it is repeatable for others, rather than just showing numbers for own application compilations; we
[jira] [Updated] (PYLUCENE-31) JCC Parallel/Multiprocess Compilation + Caching
[ https://issues.apache.org/jira/browse/PYLUCENE-31?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lee Skillen updated PYLUCENE-31: Attachment: feature-parallel-build.patch Corrected out-of-date patch. JCC Parallel/Multiprocess Compilation + Caching --- Key: PYLUCENE-31 URL: https://issues.apache.org/jira/browse/PYLUCENE-31 Project: PyLucene Issue Type: Improvement Environment: Linux 3.11.0-19-generic #33-Ubuntu SMP x86_64 GNU/Linux Reporter: Lee Skillen Priority: Minor Labels: build, cache, ccache, distutils, jcc, parallel Attachments: feature-parallel-build.patch JCC utilises distutils.Extension() in order to build JCC itself and the packages that it generates for Java wrapping - Unfortunately distutils performs its build sequentially and doesn't take advantage of any additional free cores for parallel building. As discussed on the list this is likely a design decision due to potential issues that may arise when building projects with awkward, cyclic or recursive dependencies. These issues shouldn't appear within JCC-based projects because of the generative nature of the build; i.e. all dependencies are resolved and generated prior to building, and the build process itself is about compilation and construction of the wrapper alone, of which the wrapper files are contained to a sequence of flattened compilation units. Enabling this requires monkey patching of distutils, which was also discussed on the list as being a potential source of issues, although we feel that the risk is likely lower than the current setuptools patching utilised. This would be optional functionality that is also only enabled if the monkey-patching succeeds. Distutils itself is also part of the standard library and might be less susceptible to change than setuptools, and the area of code monkey patched almost hasn't changed since 2002 (see: http://hg.python.org/cpython/file/tip/Lib/distutils/ccompiler.py). In addition to the distutils changes this patch also includes changes to the wrapper class generation to make it more cache friendly, with the target being that no changes in the wrapped code equals no changes in the wrapper code. So any changes that minimally change the wrapped code mean that with a tool such as ccache the rebuild time would be significantly reduced (almost to a nth, where n is the number of files and only one has changed). Obviously the maintainers would have to assess this risk and decide whether they would like to accept the patch or not. Code has only been tested on Linux with Python 2.7.5 but should gracefully fail and prevent parallelisation if one of the requirements hasn't been met (not on linux, no multiprocessing support, or monkey patching somehow fails). The change to caching should still benefit everyone regardless. Please note that an additional dependency on orderedset has been added to achieve the more deterministic ordering - This may not be desirable (i.e. another package might be desired, such as ordered-set, or the code might be inlined into the package instead), as per maintainer comments. --- [following repeated from mailing list] --- Performance Statistics :- The following are some quick and dirty statistics for building the jcc pylucene itself (incl. java lucene which accounts for about 30-ish seconds upfront) - The JCC files are split using --files 8, and each build is preceded with a make clean: Serial (unpatched): real5m1.502s user5m22.887s sys 0m7.749s Parallel (patched, 4 physical cores, 8 hyperthreads, 8 parallel jobs): real1m37.382s user7m16.658s sys 0m8.697s Furthermore, some additional changes were made to the wrapped file generation to make the generated code more ccache friendly (additional deterministic sorting for methods and some usage of an ordered set). With these in place and the CC and CCACHE_COMPILERCHECK environment variables set to ccache gcc and content respectively, and ensuring ccache is installed, subsequent compilation time is reduced again as follows: Parallel (patched, 4 physical cores, 8 hyperthreads, 8 parallel jobs, ccache enabled): real0m43.051s user1m10.392s sys 0m4.547s This was a run in which nothing changed between runs, so a realistic run in which changes occur it'll be a figure between 0m43.051s and 1m37.382s, depending on how drastic the change was. If many changes are expected and you want to keep it more cache friendly then using a higher --files would probably work (to an extent), or ideally use --files separate, although it doesn't currently work for me (need to investigate). We're mostly utilising the PyLucene build as a test bed since it is repeatable for others,
[jira] [Assigned] (SOLR-5968) BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields are requested
[ https://issues.apache.org/jira/browse/SOLR-5968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar reassigned SOLR-5968: --- Assignee: Shalin Shekhar Mangar BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields are requested Key: SOLR-5968 URL: https://issues.apache.org/jira/browse/SOLR-5968 Project: Solr Issue Type: Bug Reporter: Gregg Donovan Assignee: Shalin Shekhar Mangar Priority: Minor Attachments: SOLR-5968-NOCOMMIT.diff, SOLR-5968.diff If fl contains only pseudo fields, BinaryResponseWriter will load unnecessary stored fields before discarding them. The call to SolrIndexSearcher#doc(int i, SetString fields) can be skipped for this case. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5801) Resurrect org.apache.lucene.facet.util.OrdinalMappingAtomicReader
[ https://issues.apache.org/jira/browse/LUCENE-5801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068276#comment-14068276 ] ASF subversion and git services commented on LUCENE-5801: - Commit 1612197 from [~shaie] in branch 'dev/trunk' [ https://svn.apache.org/r1612197 ] LUCENE-5801: add default indexFieldName to facetFields Resurrect org.apache.lucene.facet.util.OrdinalMappingAtomicReader - Key: LUCENE-5801 URL: https://issues.apache.org/jira/browse/LUCENE-5801 Project: Lucene - Core Issue Type: Bug Affects Versions: 4.7 Reporter: Nicola Buso Assignee: Shai Erera Fix For: 5.0, 4.10 Attachments: LUCENE-5801.patch, LUCENE-5801.patch, LUCENE-5801.patch, LUCENE-5801.patch, LUCENE-5801_1.patch, LUCENE-5801_2.patch from lucene 4.6.1 the class: org.apache.lucene.facet.util.OrdinalMappingAtomicReader was removed; resurrect it because used merging indexes related to merged taxonomies. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5801) Resurrect org.apache.lucene.facet.util.OrdinalMappingAtomicReader
[ https://issues.apache.org/jira/browse/LUCENE-5801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068279#comment-14068279 ] Shai Erera commented on LUCENE-5801: Committed the fix. Resurrect org.apache.lucene.facet.util.OrdinalMappingAtomicReader - Key: LUCENE-5801 URL: https://issues.apache.org/jira/browse/LUCENE-5801 Project: Lucene - Core Issue Type: Bug Affects Versions: 4.7 Reporter: Nicola Buso Assignee: Shai Erera Fix For: 5.0, 4.10 Attachments: LUCENE-5801.patch, LUCENE-5801.patch, LUCENE-5801.patch, LUCENE-5801.patch, LUCENE-5801_1.patch, LUCENE-5801_2.patch from lucene 4.6.1 the class: org.apache.lucene.facet.util.OrdinalMappingAtomicReader was removed; resurrect it because used merging indexes related to merged taxonomies. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5801) Resurrect org.apache.lucene.facet.util.OrdinalMappingAtomicReader
[ https://issues.apache.org/jira/browse/LUCENE-5801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068277#comment-14068277 ] ASF subversion and git services commented on LUCENE-5801: - Commit 1612198 from [~shaie] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1612198 ] LUCENE-5801: add default indexFieldName to facetFields Resurrect org.apache.lucene.facet.util.OrdinalMappingAtomicReader - Key: LUCENE-5801 URL: https://issues.apache.org/jira/browse/LUCENE-5801 Project: Lucene - Core Issue Type: Bug Affects Versions: 4.7 Reporter: Nicola Buso Assignee: Shai Erera Fix For: 5.0, 4.10 Attachments: LUCENE-5801.patch, LUCENE-5801.patch, LUCENE-5801.patch, LUCENE-5801.patch, LUCENE-5801_1.patch, LUCENE-5801_2.patch from lucene 4.6.1 the class: org.apache.lucene.facet.util.OrdinalMappingAtomicReader was removed; resurrect it because used merging indexes related to merged taxonomies. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5968) BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields are requested
[ https://issues.apache.org/jira/browse/SOLR-5968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068284#comment-14068284 ] ASF subversion and git services commented on SOLR-5968: --- Commit 1612200 from sha...@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1612200 ] SOLR-5968: BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields are requested BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields are requested Key: SOLR-5968 URL: https://issues.apache.org/jira/browse/SOLR-5968 Project: Solr Issue Type: Bug Reporter: Gregg Donovan Assignee: Shalin Shekhar Mangar Priority: Minor Attachments: SOLR-5968-NOCOMMIT.diff, SOLR-5968.diff If fl contains only pseudo fields, BinaryResponseWriter will load unnecessary stored fields before discarding them. The call to SolrIndexSearcher#doc(int i, SetString fields) can be skipped for this case. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6227) ChaosMonkeySafeLeaderTest failures on jenkins
[
https://issues.apache.org/jira/browse/SOLR-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068287#comment-14068287
]
Shalin Shekhar Mangar commented on SOLR-6227:
-
I haven't seen the failure mentioned in the issue description but my jenkins
found the following failure yesterday:
{code}
java.lang.AssertionError: expected:0 but was:1
at
__randomizedtesting.SeedInfo.seed([2D7931A1F137DAA5:AC9FBFB98668BA99]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at
org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:141)
at
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
{code}
ChaosMonkeySafeLeaderTest failures on jenkins
-
Key: SOLR-6227
URL: https://issues.apache.org/jira/browse/SOLR-6227
Project: Solr
Issue Type: Bug
Components: SolrCloud, Tests
Reporter: Shalin Shekhar Mangar
Fix For: 4.10
This is happening very frequently.
{code}
1 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 143 from
https://127.0.0.1:36610/xvv/collection1lastClient and got 142 from
https://127.0.0.1:33168/xvv/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 143 from
https://127.0.0.1:36610/xvv/collection1lastClient and got 142 from
https://127.0.0.1:33168/xvv/collection1
at
__randomizedtesting.SeedInfo.seed([3C1FB6EAFE71:BDF938F2AA829E4D]:0)
at org.junit.Assert.fail(Assert.java:93)
at
org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1139)
at
org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1118)
at
org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:150)
at
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5968) BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields are requested
[ https://issues.apache.org/jira/browse/SOLR-5968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068316#comment-14068316 ] ASF subversion and git services commented on SOLR-5968: --- Commit 1612210 from sha...@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1612210 ] SOLR-5968: BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields are requested BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields are requested Key: SOLR-5968 URL: https://issues.apache.org/jira/browse/SOLR-5968 Project: Solr Issue Type: Bug Reporter: Gregg Donovan Assignee: Shalin Shekhar Mangar Priority: Minor Attachments: SOLR-5968-NOCOMMIT.diff, SOLR-5968.diff If fl contains only pseudo fields, BinaryResponseWriter will load unnecessary stored fields before discarding them. The call to SolrIndexSearcher#doc(int i, SetString fields) can be skipped for this case. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-5968) BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields are requested
[ https://issues.apache.org/jira/browse/SOLR-5968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar resolved SOLR-5968. - Resolution: Fixed Fix Version/s: 4.10 5.0 Thanks Gregg. I've committed your patch without the test. I can't quite figure out how to test it better either and this optimization is really useful. BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields are requested Key: SOLR-5968 URL: https://issues.apache.org/jira/browse/SOLR-5968 Project: Solr Issue Type: Bug Reporter: Gregg Donovan Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 5.0, 4.10 Attachments: SOLR-5968-NOCOMMIT.diff, SOLR-5968.diff If fl contains only pseudo fields, BinaryResponseWriter will load unnecessary stored fields before discarding them. The call to SolrIndexSearcher#doc(int i, SetString fields) can be skipped for this case. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068330#comment-14068330 ] Yonik Seeley commented on SOLR-6260: Reasonable people can disagree. One should not have to face hurtful personal attacks when they do. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5473) Split clusterstate.json per collection and watch states selectively
[ https://issues.apache.org/jira/browse/SOLR-5473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068335#comment-14068335 ] Noble Paul commented on SOLR-5473: -- [~markrmil...@gmail.com] Can you look at the latest patch and let me know your comments Split clusterstate.json per collection and watch states selectively Key: SOLR-5473 URL: https://issues.apache.org/jira/browse/SOLR-5473 Project: Solr Issue Type: Sub-task Components: SolrCloud Reporter: Noble Paul Assignee: Noble Paul Fix For: 5.0 Attachments: SOLR-5473-74 .patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74_POC.patch, SOLR-5473-configname-fix.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473_undo.patch, ec2-23-20-119-52_solr.log, ec2-50-16-38-73_solr.log As defined in the parent issue, store the states of each collection under /collections/collectionname/state.json node -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5958) Document (and fix) numShards and router selection parameter in SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068340#comment-14068340 ] Noble Paul commented on SOLR-5958: -- bq.Though the bootstrap does not treat numShards as a required param, the Collection API does and throw an error if we don't specify numShards. The collection API is not used to create the collection, but the STATE command does that. The STATE command would fail if numShards is not present, but the nodes will always send the value 1 if nothing else is configured Document (and fix) numShards and router selection parameter in SolrCloud Key: SOLR-5958 URL: https://issues.apache.org/jira/browse/SOLR-5958 Project: Solr Issue Type: Task Components: SolrCloud Reporter: Anshum Gupta Assignee: Anshum Gupta Priority: Minor Right now numShards works in rather mysterious ways (unless you know how it works). We should clearly document the following: * If we start SolrCloud with bootstrapping, without mentioning numShards parameter, it defaults to 1 and also defaults the router to 'implicit'. * Mentioning numShards param, defaults the router to compositeId. * Though the bootstrap does not treat numShards as a required param, the Collection API does and throw an error if we don't specify numShards. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-5958) Document (and fix) numShards and router selection parameter in SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068340#comment-14068340 ] Noble Paul edited comment on SOLR-5958 at 7/21/14 9:25 AM: --- bq.Though the bootstrap does not treat numShards as a required param, the Collection API does and throw an error if we don't specify numShards. The collection API is not used to create the collection, but the STATE command does that. The STATE command would fail if numShards is not present, but the nodes will always send the value 1 if nothing else is configured But the collection API has to use some numShards for a non 'implicit router' collection . Hence the router is implicit Probably we need our documentation to improve was (Author: noble.paul): bq.Though the bootstrap does not treat numShards as a required param, the Collection API does and throw an error if we don't specify numShards. The collection API is not used to create the collection, but the STATE command does that. The STATE command would fail if numShards is not present, but the nodes will always send the value 1 if nothing else is configured Document (and fix) numShards and router selection parameter in SolrCloud Key: SOLR-5958 URL: https://issues.apache.org/jira/browse/SOLR-5958 Project: Solr Issue Type: Task Components: SolrCloud Reporter: Anshum Gupta Assignee: Anshum Gupta Priority: Minor Right now numShards works in rather mysterious ways (unless you know how it works). We should clearly document the following: * If we start SolrCloud with bootstrapping, without mentioning numShards parameter, it defaults to 1 and also defaults the router to 'implicit'. * Mentioning numShards param, defaults the router to compositeId. * Though the bootstrap does not treat numShards as a required param, the Collection API does and throw an error if we don't specify numShards. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5819) Add block tree postings format that supports term ords
[ https://issues.apache.org/jira/browse/LUCENE-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068341#comment-14068341 ] ASF subversion and git services commented on LUCENE-5819: - Commit 1612213 from [~mikemccand] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1612213 ] LUCENE-5819: add terms dict and postings format that implement term ordinals Add block tree postings format that supports term ords -- Key: LUCENE-5819 URL: https://issues.apache.org/jira/browse/LUCENE-5819 Project: Lucene - Core Issue Type: Improvement Components: modules/other Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.0, 4.10 Attachments: LUCENE-5819.patch, LUCENE-5819.patch BlockTree is our default terms dictionary today, but it doesn't support term ords, which is an optional API in the postings format to retrieve the ordinal for the currently seek'd term, and also later seek by that ordinal e.g. to lookup the term. This can possibly be useful for e.g. faceting, and maybe at some point we can share the postings terms dict with the one used by sorted/set DV for cases when app wants to invert and facet on a given field. The older (3.x) block terms dict can easily support ords, and we have a Lucene41OrdsPF in test-framework, but it's not as fast / compact as block-tree, and doesn't (can't easily) implement an optimized intersect, but it could be for fields we'd want to facet on, these tradeoffs don't matter. It's nice to have options... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-5819) Add block tree postings format that supports term ords
[ https://issues.apache.org/jira/browse/LUCENE-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-5819. Resolution: Fixed Add block tree postings format that supports term ords -- Key: LUCENE-5819 URL: https://issues.apache.org/jira/browse/LUCENE-5819 Project: Lucene - Core Issue Type: Improvement Components: modules/other Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.0, 4.10 Attachments: LUCENE-5819.patch, LUCENE-5819.patch BlockTree is our default terms dictionary today, but it doesn't support term ords, which is an optional API in the postings format to retrieve the ordinal for the currently seek'd term, and also later seek by that ordinal e.g. to lookup the term. This can possibly be useful for e.g. faceting, and maybe at some point we can share the postings terms dict with the one used by sorted/set DV for cases when app wants to invert and facet on a given field. The older (3.x) block terms dict can easily support ords, and we have a Lucene41OrdsPF in test-framework, but it's not as fast / compact as block-tree, and doesn't (can't easily) implement an optimized intersect, but it could be for fields we'd want to facet on, these tradeoffs don't matter. It's nice to have options... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5819) Add block tree postings format that supports term ords
[ https://issues.apache.org/jira/browse/LUCENE-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068355#comment-14068355 ] ASF subversion and git services commented on LUCENE-5819: - Commit 1612217 from [~mikemccand] in branch 'dev/trunk' [ https://svn.apache.org/r1612217 ] LUCENE-5819: fix ord bug; add test case; remove dead code Add block tree postings format that supports term ords -- Key: LUCENE-5819 URL: https://issues.apache.org/jira/browse/LUCENE-5819 Project: Lucene - Core Issue Type: Improvement Components: modules/other Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.0, 4.10 Attachments: LUCENE-5819.patch, LUCENE-5819.patch BlockTree is our default terms dictionary today, but it doesn't support term ords, which is an optional API in the postings format to retrieve the ordinal for the currently seek'd term, and also later seek by that ordinal e.g. to lookup the term. This can possibly be useful for e.g. faceting, and maybe at some point we can share the postings terms dict with the one used by sorted/set DV for cases when app wants to invert and facet on a given field. The older (3.x) block terms dict can easily support ords, and we have a Lucene41OrdsPF in test-framework, but it's not as fast / compact as block-tree, and doesn't (can't easily) implement an optimized intersect, but it could be for fields we'd want to facet on, these tradeoffs don't matter. It's nice to have options... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_11) - Build # 10862 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10862/ Java: 32bit/jdk1.8.0_11 -client -XX:+UseConcMarkSweepGC 1 tests failed. REGRESSION: org.apache.lucene.uninverting.TestFieldCacheSort.testEmptyStringVsNullStringSort Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([852BB84974DA2250:E4C88A9E0CBB24E0]:0) at org.apache.lucene.index.AssertingAtomicReader.getSortedDocValues(AssertingAtomicReader.java:638) at org.apache.lucene.index.MultiDocValues.getSortedValues(MultiDocValues.java:312) at org.apache.lucene.index.SlowCompositeReaderWrapper.getSortedDocValues(SlowCompositeReaderWrapper.java:128) at org.apache.lucene.index.FilterAtomicReader.getSortedDocValues(FilterAtomicReader.java:414) at org.apache.lucene.index.FieldFilterAtomicReader.getSortedDocValues(FieldFilterAtomicReader.java:132) at org.apache.lucene.index.ParallelAtomicReader.getSortedDocValues(ParallelAtomicReader.java:288) at org.apache.lucene.index.FilterAtomicReader.getSortedDocValues(FilterAtomicReader.java:414) at org.apache.lucene.index.FieldFilterAtomicReader.getSortedDocValues(FieldFilterAtomicReader.java:132) at org.apache.lucene.index.ParallelAtomicReader.getSortedDocValues(ParallelAtomicReader.java:288) at org.apache.lucene.index.FilterAtomicReader.getSortedDocValues(FilterAtomicReader.java:414) at org.apache.lucene.index.AssertingAtomicReader.getSortedDocValues(AssertingAtomicReader.java:635) at org.apache.lucene.index.ParallelAtomicReader.getSortedDocValues(ParallelAtomicReader.java:288) at org.apache.lucene.index.DocValues.getSorted(DocValues.java:273) at org.apache.lucene.search.FieldComparator$TermOrdValComparator.getSortedDocValues(FieldComparator.java:821) at org.apache.lucene.search.FieldComparator$TermOrdValComparator.setNextReader(FieldComparator.java:826) at org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.doSetNextReader(TopFieldCollector.java:97) at org.apache.lucene.search.SimpleCollector.getLeafCollector(SimpleCollector.java:33) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:605) at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:94) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:573) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:525) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:502) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:318) at org.apache.lucene.uninverting.TestFieldCacheSort.testEmptyStringVsNullStringSort(TestFieldCacheSort.java:1029) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at
[jira] [Commented] (SOLR-3619) Rename 'example' dir to 'server' and pull examples into an 'examples' directory
[ https://issues.apache.org/jira/browse/SOLR-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068403#comment-14068403 ] Grant Ingersoll commented on SOLR-3619: --- bq. Yeah, it feels like one should still be able to start the server and then index a document (as they can do now) without any other mandatory steps. +1 Agree, single core dies, collection1 dies. Everything should just work out of the box! The first 5 minute experience should be all about the user and their data and very little to do about solr configs, schemas, etc. Same for pretty much the whole first day. By the end of the first week, a new user should have a thorough understanding of what they need to do to get to production. Easy to start, easy to finish. Rename 'example' dir to 'server' and pull examples into an 'examples' directory --- Key: SOLR-3619 URL: https://issues.apache.org/jira/browse/SOLR-3619 Project: Solr Issue Type: Improvement Reporter: Mark Miller Fix For: 4.9, 5.0 Attachments: SOLR-3619.patch, server-name-layout.png -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5835) Add sortMissingLast support to TermValComparator
[ https://issues.apache.org/jira/browse/LUCENE-5835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068404#comment-14068404 ] ASF subversion and git services commented on LUCENE-5835: - Commit 1612245 from [~jpountz] in branch 'dev/trunk' [ https://svn.apache.org/r1612245 ] LUCENE-5835: TermValComparator can sort missing values last. Add sortMissingLast support to TermValComparator Key: LUCENE-5835 URL: https://issues.apache.org/jira/browse/LUCENE-5835 Project: Lucene - Core Issue Type: Improvement Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Fix For: 5.0, 4.10 Attachments: LUCENE-5835.patch It would be nice to allow to configure the behavior on missing values for this comparator, similarly to what TermOrdValComparator does. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5835) Add sortMissingLast support to TermValComparator
[ https://issues.apache.org/jira/browse/LUCENE-5835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068414#comment-14068414 ] ASF subversion and git services commented on LUCENE-5835: - Commit 1612249 from [~jpountz] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1612249 ] LUCENE-5835: TermValComparator can sort missing values last. Add sortMissingLast support to TermValComparator Key: LUCENE-5835 URL: https://issues.apache.org/jira/browse/LUCENE-5835 Project: Lucene - Core Issue Type: Improvement Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Fix For: 5.0, 4.10 Attachments: LUCENE-5835.patch It would be nice to allow to configure the behavior on missing values for this comparator, similarly to what TermOrdValComparator does. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_11) - Build # 10862 - Failure!
I'm looking into it. On Mon, Jul 21, 2014 at 12:46 PM, Policeman Jenkins Server jenk...@thetaphi.de wrote: Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10862/ Java: 32bit/jdk1.8.0_11 -client -XX:+UseConcMarkSweepGC 1 tests failed. REGRESSION: org.apache.lucene.uninverting.TestFieldCacheSort.testEmptyStringVsNullStringSort Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([852BB84974DA2250:E4C88A9E0CBB24E0]:0) at org.apache.lucene.index.AssertingAtomicReader.getSortedDocValues(AssertingAtomicReader.java:638) at org.apache.lucene.index.MultiDocValues.getSortedValues(MultiDocValues.java:312) at org.apache.lucene.index.SlowCompositeReaderWrapper.getSortedDocValues(SlowCompositeReaderWrapper.java:128) at org.apache.lucene.index.FilterAtomicReader.getSortedDocValues(FilterAtomicReader.java:414) at org.apache.lucene.index.FieldFilterAtomicReader.getSortedDocValues(FieldFilterAtomicReader.java:132) at org.apache.lucene.index.ParallelAtomicReader.getSortedDocValues(ParallelAtomicReader.java:288) at org.apache.lucene.index.FilterAtomicReader.getSortedDocValues(FilterAtomicReader.java:414) at org.apache.lucene.index.FieldFilterAtomicReader.getSortedDocValues(FieldFilterAtomicReader.java:132) at org.apache.lucene.index.ParallelAtomicReader.getSortedDocValues(ParallelAtomicReader.java:288) at org.apache.lucene.index.FilterAtomicReader.getSortedDocValues(FilterAtomicReader.java:414) at org.apache.lucene.index.AssertingAtomicReader.getSortedDocValues(AssertingAtomicReader.java:635) at org.apache.lucene.index.ParallelAtomicReader.getSortedDocValues(ParallelAtomicReader.java:288) at org.apache.lucene.index.DocValues.getSorted(DocValues.java:273) at org.apache.lucene.search.FieldComparator$TermOrdValComparator.getSortedDocValues(FieldComparator.java:821) at org.apache.lucene.search.FieldComparator$TermOrdValComparator.setNextReader(FieldComparator.java:826) at org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.doSetNextReader(TopFieldCollector.java:97) at org.apache.lucene.search.SimpleCollector.getLeafCollector(SimpleCollector.java:33) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:605) at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:94) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:573) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:525) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:502) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:318) at org.apache.lucene.uninverting.TestFieldCacheSort.testEmptyStringVsNullStringSort(TestFieldCacheSort.java:1029) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_11) - Build # 10862 - Failure!
This is due to https://issues.apache.org/jira/browse/LUCENE-5788. On Mon, Jul 21, 2014 at 1:16 PM, Adrien Grand jpou...@gmail.com wrote: I'm looking into it. On Mon, Jul 21, 2014 at 12:46 PM, Policeman Jenkins Server jenk...@thetaphi.de wrote: Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10862/ Java: 32bit/jdk1.8.0_11 -client -XX:+UseConcMarkSweepGC 1 tests failed. REGRESSION: org.apache.lucene.uninverting.TestFieldCacheSort.testEmptyStringVsNullStringSort Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([852BB84974DA2250:E4C88A9E0CBB24E0]:0) at org.apache.lucene.index.AssertingAtomicReader.getSortedDocValues(AssertingAtomicReader.java:638) at org.apache.lucene.index.MultiDocValues.getSortedValues(MultiDocValues.java:312) at org.apache.lucene.index.SlowCompositeReaderWrapper.getSortedDocValues(SlowCompositeReaderWrapper.java:128) at org.apache.lucene.index.FilterAtomicReader.getSortedDocValues(FilterAtomicReader.java:414) at org.apache.lucene.index.FieldFilterAtomicReader.getSortedDocValues(FieldFilterAtomicReader.java:132) at org.apache.lucene.index.ParallelAtomicReader.getSortedDocValues(ParallelAtomicReader.java:288) at org.apache.lucene.index.FilterAtomicReader.getSortedDocValues(FilterAtomicReader.java:414) at org.apache.lucene.index.FieldFilterAtomicReader.getSortedDocValues(FieldFilterAtomicReader.java:132) at org.apache.lucene.index.ParallelAtomicReader.getSortedDocValues(ParallelAtomicReader.java:288) at org.apache.lucene.index.FilterAtomicReader.getSortedDocValues(FilterAtomicReader.java:414) at org.apache.lucene.index.AssertingAtomicReader.getSortedDocValues(AssertingAtomicReader.java:635) at org.apache.lucene.index.ParallelAtomicReader.getSortedDocValues(ParallelAtomicReader.java:288) at org.apache.lucene.index.DocValues.getSorted(DocValues.java:273) at org.apache.lucene.search.FieldComparator$TermOrdValComparator.getSortedDocValues(FieldComparator.java:821) at org.apache.lucene.search.FieldComparator$TermOrdValComparator.setNextReader(FieldComparator.java:826) at org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.doSetNextReader(TopFieldCollector.java:97) at org.apache.lucene.search.SimpleCollector.getLeafCollector(SimpleCollector.java:33) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:605) at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:94) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:573) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:525) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:502) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:318) at org.apache.lucene.uninverting.TestFieldCacheSort.testEmptyStringVsNullStringSort(TestFieldCacheSort.java:1029) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
[jira] [Resolved] (LUCENE-5835) Add sortMissingLast support to TermValComparator
[ https://issues.apache.org/jira/browse/LUCENE-5835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrien Grand resolved LUCENE-5835. -- Resolution: Fixed Add sortMissingLast support to TermValComparator Key: LUCENE-5835 URL: https://issues.apache.org/jira/browse/LUCENE-5835 Project: Lucene - Core Issue Type: Improvement Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Fix For: 5.0, 4.10 Attachments: LUCENE-5835.patch It would be nice to allow to configure the behavior on missing values for this comparator, similarly to what TermOrdValComparator does. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5834) Make empty doc values impls singletons
[
https://issues.apache.org/jira/browse/LUCENE-5834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068424#comment-14068424
]
ASF subversion and git services commented on LUCENE-5834:
-
Commit 1612251 from [~jpountz] in branch 'dev/trunk'
[ https://svn.apache.org/r1612251 ]
LUCENE-5834: Empty sorted set and numeric doc values are now singletons.
Make empty doc values impls singletons
--
Key: LUCENE-5834
URL: https://issues.apache.org/jira/browse/LUCENE-5834
Project: Lucene - Core
Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Trivial
Fix For: 5.0, 4.10
Attachments: LUCENE-5834.patch
Making these empty instances singletons would allow to use
{{unwrapSingleton}} to check if they are single-valued.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-5834) Make empty doc values impls singletons
[
https://issues.apache.org/jira/browse/LUCENE-5834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand resolved LUCENE-5834.
--
Resolution: Fixed
Make empty doc values impls singletons
--
Key: LUCENE-5834
URL: https://issues.apache.org/jira/browse/LUCENE-5834
Project: Lucene - Core
Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Trivial
Fix For: 5.0, 4.10
Attachments: LUCENE-5834.patch
Making these empty instances singletons would allow to use
{{unwrapSingleton}} to check if they are single-valued.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5834) Make empty doc values impls singletons
[
https://issues.apache.org/jira/browse/LUCENE-5834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068425#comment-14068425
]
ASF subversion and git services commented on LUCENE-5834:
-
Commit 1612252 from [~jpountz] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1612252 ]
LUCENE-5834: Empty sorted set and numeric doc values are now singletons.
Make empty doc values impls singletons
--
Key: LUCENE-5834
URL: https://issues.apache.org/jira/browse/LUCENE-5834
Project: Lucene - Core
Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Trivial
Fix For: 5.0, 4.10
Attachments: LUCENE-5834.patch
Making these empty instances singletons would allow to use
{{unwrapSingleton}} to check if they are single-valued.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6262) Make the name attribute optional for components in solrconfig.xml
Noble Paul created SOLR-6262: Summary: Make the name attribute optional for components in solrconfig.xml Key: SOLR-6262 URL: https://issues.apache.org/jira/browse/SOLR-6262 Project: Solr Issue Type: Improvement Reporter: Noble Paul Assignee: Noble Paul It is not a good idea to let people decide the names of our standard components such as update, replication, /get etc. These names can be hard coded in the Java file itself. and let us remove the names from solrconfig.xml. However it should be possible to override the name by specifying the 'name' attribute -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6262) Make the name attribute optional for components in solrconfig.xml
[ https://issues.apache.org/jira/browse/SOLR-6262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068434#comment-14068434 ] Yonik Seeley commented on SOLR-6262: +1 Make the name attribute optional for components in solrconfig.xml --- Key: SOLR-6262 URL: https://issues.apache.org/jira/browse/SOLR-6262 Project: Solr Issue Type: Improvement Reporter: Noble Paul Assignee: Noble Paul It is not a good idea to let people decide the names of our standard components such as update, replication, /get etc. These names can be hard coded in the Java file itself. and let us remove the names from solrconfig.xml. However it should be possible to override the name by specifying the 'name' attribute -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6262) Make the name attribute optional for components in solrconfig.xml
[ https://issues.apache.org/jira/browse/SOLR-6262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068439#comment-14068439 ] Shalin Shekhar Mangar commented on SOLR-6262: - +1 I'd argue the same for the primitive types (int/float/long/double/date/boolean/string) in our schema.xml but that can be another issue. Make the name attribute optional for components in solrconfig.xml --- Key: SOLR-6262 URL: https://issues.apache.org/jira/browse/SOLR-6262 Project: Solr Issue Type: Improvement Reporter: Noble Paul Assignee: Noble Paul It is not a good idea to let people decide the names of our standard components such as update, replication, /get etc. These names can be hard coded in the Java file itself. and let us remove the names from solrconfig.xml. However it should be possible to override the name by specifying the 'name' attribute -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5838) hunspell buggy with over 64k affixes
[ https://issues.apache.org/jira/browse/LUCENE-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated LUCENE-5838: Attachment: LUCENE-5838.patch patch with test that generates its own file, so it doesnt need a 1MB test data file. hunspell buggy with over 64k affixes Key: LUCENE-5838 URL: https://issues.apache.org/jira/browse/LUCENE-5838 Project: Lucene - Core Issue Type: Bug Reporter: Robert Muir Attachments: LUCENE-5838.patch, LUCENE-5838.patch currently we build TreeMapString,ListCharacter in ram, to sort before adding to the FST (which encodes the list as IntsRef). char overflows here if there are more than 64k affixes (e.g. basque). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6227) ChaosMonkeySafeLeaderTest failures on jenkins
[ https://issues.apache.org/jira/browse/SOLR-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068486#comment-14068486 ] Deepak Kumar commented on SOLR-6227: Hello all, While invoking http://localhost:9090/solr/admin/collections?action=CREATEname=corenumShards=1replicationFactor=1collection.configName=coreconf I am getting below exception, this has been happening consistently for 4.7.2 and 4.7.1 version, please help me understand if its the very same thing: -- solr.log -- [ERROR] [2014-07-21 18:06:20,960] [Overseer-92140072928280576-localhost:9090_solr-n_03] [cloud.OverseerCollectionProcessor] - [Collection createcollection of createcollection failed:org.apache.solr.common.SolrException at org.apache.solr.cloud.OverseerCollectionProcessor.createCollection(OverseerCollectionProcessor.java:1687) at org.apache.solr.cloud.OverseerCollectionProcessor.processMessage(OverseerCollectionProcessor.java:387) at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:200) at java.lang.Thread.run(Thread.java:662) Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/usersearches at org.apache.zookeeper.KeeperException.create(KeeperException.java:119) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:429) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73) at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:426) at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:383) at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:370) at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:357) at org.apache.solr.cloud.OverseerCollectionProcessor.createConfNode(OverseerCollectionProcessor.java:1711) at org.apache.solr.cloud.OverseerCollectionProcessor.createCollection(OverseerCollectionProcessor.java:1624) ... 3 more ] [ERROR] [2014-07-21 16:39:55,906] [http-9090-1] [servlet.SolrDispatchFilter] - [null:org.apache.solr.common.SolrException at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:248) at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:233) at org.apache.solr.handler.admin.CollectionsHandler.handleCreateAction(CollectionsHandler.java:368) at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:141) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:720) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:265) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:205) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:662) ] -- SOLR http response -- ?xml version=1.0 encoding=UTF-8? response lst name=responseHeaderint name=status500/intint name=QTime210/int/lststr name=Operation createcollection caused exception:org.apache.solr.common.SolrException:org.apache.solr.common.SolrException/strlst name=exceptionnull name=msg/int name=rspCode500/int/lstlst name=errorstr name=traceorg.apache.solr.common.SolrException at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:248) at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:233) at
[jira] [Commented] (SOLR-5473) Split clusterstate.json per collection and watch states selectively
[ https://issues.apache.org/jira/browse/SOLR-5473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068504#comment-14068504 ] Alan Woodward commented on SOLR-5473: - I think part of the reason this is so unwieldy is that ClusterState itself is monolithic - you call ZkStateReader.getClusterState() and it goes and gets the state of the entire cluster, and then you typically only need information for a single collection. So ClusterState needs to know about all the different state versions, which bloats it up, and then leaves you with API warts like ZkController being responsible for removing watches. What I think should really happen here is that we should add an intermediate layer, CollectionState. This has three implementations, one that is a singleton watching the master clusterstate.json, one that is a separate object with watchers for each 'external' collection, one that just directly fetches data from ZK whenever it's queried. When ZkStateReader starts up (and can we maybe move createClusterStateWatchersAndUpdate() into the constructor?) it works out which CollectionState type it needs for each collection in the cluster. Users of the API just call ZkStateReader.getCollection() and they get the right kind of CollectionState object, no need to have external knowledge of what the state version of the collection is. Having stuck my oar in here, I'm now going offline for a couple of weeks :-) Maybe this API change should be a separate issue, but I think it should be nailed down before this one is committed. Split clusterstate.json per collection and watch states selectively Key: SOLR-5473 URL: https://issues.apache.org/jira/browse/SOLR-5473 Project: Solr Issue Type: Sub-task Components: SolrCloud Reporter: Noble Paul Assignee: Noble Paul Fix For: 5.0 Attachments: SOLR-5473-74 .patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74_POC.patch, SOLR-5473-configname-fix.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473_undo.patch, ec2-23-20-119-52_solr.log, ec2-50-16-38-73_solr.log As defined in the parent issue, store the states of each collection under /collections/collectionname/state.json node -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6248) MoreLikeThis Query Parser
[ https://issues.apache.org/jira/browse/SOLR-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068501#comment-14068501 ] Steve Molloy commented on SOLR-6248: Would that approach also support sending in text that isn't in the index? This is the main reason we're using the MLT handler, which we need to be distributed (thus SOLR-5480). but if we can have a single approach for both, I agree that not maintaining 2 configurations (and 2 handlers in the code) would be much better. Let me know if I can help out. MoreLikeThis Query Parser - Key: SOLR-6248 URL: https://issues.apache.org/jira/browse/SOLR-6248 Project: Solr Issue Type: New Feature Reporter: Anshum Gupta MLT Component doesn't let people highlight/paginate and the handler comes with an cost of maintaining another piece in the config. Also, any changes to the default (number of results to be fetched etc.) /select handler need to be copied/synced with this handler too. Having an MLT QParser would let users get back docs based on a query for them to paginate, highlight etc. It would also give them the flexibility to use this anywhere i.e. q,fq,bq etc. A bit of history about MLT (thanks to Hoss) MLT Handler pre-dates the existence of QParsers and was meant to take an arbitrary query as input, find docs that match that query, club them together to find interesting terms, and then use those terms as if they were my main query to generate a main result set. This result would then be used as the set to facet, highlight etc. The flow: Query - DocList(m) - Bag (terms) - Query - DocList\(y) The MLT component on the other hand solved a very different purpose of augmenting the main result set. It is used to get similar docs for each of the doc in the main result set. DocSet\(n) - n * Bag (terms) - n * (Query) - n * DocList(m) The new approach: All of this can be done better and cleaner (and makes more sense too) using an MLT QParser. An important thing to handle here is the case where the user doesn't have TermVectors, in which case, it does what happens right now i.e. parsing stored fields. Also, in case the user doesn't have a field (to be used for MLT) indexed, the field would need to be a TextField with an index analyzer defined. This analyzer will then be used to extract terms for MLT. In case of SolrCloud mode, '/get-termvectors' can be used after looking at the schema (if TermVectors are enabled for the field). If not, a /get call can be used to fetch the field and parse it. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5839) Regex bug in AnalyzingQueryParser
Tim Allison created LUCENE-5839: --- Summary: Regex bug in AnalyzingQueryParser Key: LUCENE-5839 URL: https://issues.apache.org/jira/browse/LUCENE-5839 Project: Lucene - Core Issue Type: Bug Components: core/queryparser Affects Versions: 4.9 Reporter: Tim Allison Priority: Trivial Dennis Walter recently pointed out this bug on dev@lucene.apache.org: // gobble escaped chars or find a wildcard character private final Pattern wildcardPattern = Pattern.compile((\\.)|([?*]+)); The first group will match a literal dot (.), while its intention seems to be to match a backslash and a single character. So the expression should instead be (.)|([?*]+) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request: LUCENE-5839: Fix regex in AnalyzingQuery...
GitHub user tballison opened a pull request: https://github.com/apache/lucene-solr/pull/67 LUCENE-5839: Fix regex in AnalyzingQueryParser LUCENE-5839: Fix regex in AnalyzingQueryParser You can merge this pull request into a Git repository by running: $ git pull https://github.com/tballison/lucene-solr trunk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/lucene-solr/pull/67.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #67 commit 1eac4382dd1ee7a4319096499335d7f7f28f526a Author: tballison talli...@mitre.org Date: 2014-07-21T13:22:38Z Fix regex in AnalyzingQueryParser --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5839) Regex bug in AnalyzingQueryParser
[ https://issues.apache.org/jira/browse/LUCENE-5839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068511#comment-14068511 ] ASF GitHub Bot commented on LUCENE-5839: GitHub user tballison opened a pull request: https://github.com/apache/lucene-solr/pull/67 LUCENE-5839: Fix regex in AnalyzingQueryParser LUCENE-5839: Fix regex in AnalyzingQueryParser You can merge this pull request into a Git repository by running: $ git pull https://github.com/tballison/lucene-solr trunk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/lucene-solr/pull/67.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #67 commit 1eac4382dd1ee7a4319096499335d7f7f28f526a Author: tballison talli...@mitre.org Date: 2014-07-21T13:22:38Z Fix regex in AnalyzingQueryParser Regex bug in AnalyzingQueryParser - Key: LUCENE-5839 URL: https://issues.apache.org/jira/browse/LUCENE-5839 Project: Lucene - Core Issue Type: Bug Components: core/queryparser Affects Versions: 4.9 Reporter: Tim Allison Priority: Trivial Dennis Walter recently pointed out this bug on dev@lucene.apache.org: // gobble escaped chars or find a wildcard character private final Pattern wildcardPattern = Pattern.compile((\\.)|([?*]+)); The first group will match a literal dot (.), while its intention seems to be to match a backslash and a single character. So the expression should instead be (.)|([?*]+) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5839) Regex bug in AnalyzingQueryParser
[ https://issues.apache.org/jira/browse/LUCENE-5839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068520#comment-14068520 ] Tim Allison commented on LUCENE-5839: - Created edge case to test this. LUCENE-5504 is a cleaner way to go to deal with this. While looking at the code again, I noticed that AnalyzingQueryParser is using the superclass' getXQuery. This will have the effect of lowercasing whatever happens after analysis. If someone's analyzer is upcasing, AnalyzingQueryParser will fail. I can submit another patch to fix this, but I think something along the lines of LUCENE-5504 is a better way to go. Regex bug in AnalyzingQueryParser - Key: LUCENE-5839 URL: https://issues.apache.org/jira/browse/LUCENE-5839 Project: Lucene - Core Issue Type: Bug Components: core/queryparser Affects Versions: 4.9 Reporter: Tim Allison Priority: Trivial Dennis Walter recently pointed out this bug on dev@lucene.apache.org: // gobble escaped chars or find a wildcard character private final Pattern wildcardPattern = Pattern.compile((\\.)|([?*]+)); The first group will match a literal dot (.), while its intention seems to be to match a backslash and a single character. So the expression should instead be (.)|([?*]+) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Bug in AnalyzingQueryParser Pattern
Thank you, Dennis: https://issues.apache.org/jira/i#browse/LUCENE-5839 From: Dennis Walter [mailto:dennis.wal...@gmail.com] Sent: Sunday, July 20, 2014 2:52 PM To: dev@lucene.apache.org Subject: Bug in AnalyzingQueryParser Pattern Hi there, While reading the source code of AnalyzingQueryParser to understand what it does, I think I found a bug in the regular expression used to detect wildcards. It is defined as // gobble escaped chars or find a wildcard character private final Pattern wildcardPattern = Pattern.compile((\\.)|([?*]+))file:///\\.)|([%3f*]+)%22); The first group will match a literal dot (.), while its intention seems to be to match a backslash and a single character. So the expression should instead be (.)|([?*]+)file:///\\\.)|([%3f*]+) Best Regards Dennis
[GitHub] lucene-solr pull request: Lucene5205
Github user tballison closed the pull request at: https://github.com/apache/lucene-solr/pull/64 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5836) BytesRef.copyBytes and copyChars don't oversize
[ https://issues.apache.org/jira/browse/LUCENE-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrien Grand updated LUCENE-5836: - Attachment: LUCENE-5836.patch And what if we move the StringBuilder-like methods to a different class? (see eg. attached patch which doesn't compile but should give an idea of the change) Would that work for you? BytesRef.copyBytes and copyChars don't oversize --- Key: LUCENE-5836 URL: https://issues.apache.org/jira/browse/LUCENE-5836 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand Attachments: LUCENE-5836.patch When copying data from another BytesRef/CharSequence, these methods don't oversize. This is not an issue if this method is used only once per BytesRef instance but I just reviewed the usage of these methods and they are very frequently used in loops to do things like: - keep track of the top values in comparators - keep track of the previous terms in various loops over a terms enum (lucene49 DV consumer, BlockTreeTermsWriter) - etc. Although unlikely, it might be possible to hit a worst-case and to resize the underlying byte[] on every call to copyBytes? Should we oversize the underlying array in these methods? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5836) BytesRef.copyBytes and copyChars don't oversize
[ https://issues.apache.org/jira/browse/LUCENE-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068538#comment-14068538 ] Robert Muir commented on LUCENE-5836: - that would be a fantastic improvement IMO. BytesRef.copyBytes and copyChars don't oversize --- Key: LUCENE-5836 URL: https://issues.apache.org/jira/browse/LUCENE-5836 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand Attachments: LUCENE-5836.patch When copying data from another BytesRef/CharSequence, these methods don't oversize. This is not an issue if this method is used only once per BytesRef instance but I just reviewed the usage of these methods and they are very frequently used in loops to do things like: - keep track of the top values in comparators - keep track of the previous terms in various loops over a terms enum (lucene49 DV consumer, BlockTreeTermsWriter) - etc. Although unlikely, it might be possible to hit a worst-case and to resize the underlying byte[] on every call to copyBytes? Should we oversize the underlying array in these methods? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5836) BytesRef.copyBytes and copyChars don't oversize
[ https://issues.apache.org/jira/browse/LUCENE-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068539#comment-14068539 ] Adrien Grand commented on LUCENE-5836: -- OK, I'll give it a try. BytesRef.copyBytes and copyChars don't oversize --- Key: LUCENE-5836 URL: https://issues.apache.org/jira/browse/LUCENE-5836 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand Attachments: LUCENE-5836.patch When copying data from another BytesRef/CharSequence, these methods don't oversize. This is not an issue if this method is used only once per BytesRef instance but I just reviewed the usage of these methods and they are very frequently used in loops to do things like: - keep track of the top values in comparators - keep track of the previous terms in various loops over a terms enum (lucene49 DV consumer, BlockTreeTermsWriter) - etc. Although unlikely, it might be possible to hit a worst-case and to resize the underlying byte[] on every call to copyBytes? Should we oversize the underlying array in these methods? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5473) Split clusterstate.json per collection and watch states selectively
[ https://issues.apache.org/jira/browse/SOLR-5473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068548#comment-14068548 ] Noble Paul commented on SOLR-5473: -- [~romseygeek] Is this comment made after seeing the latst patch or is it based on the old one? I have implemented it in more or less the way you suggested Split clusterstate.json per collection and watch states selectively Key: SOLR-5473 URL: https://issues.apache.org/jira/browse/SOLR-5473 Project: Solr Issue Type: Sub-task Components: SolrCloud Reporter: Noble Paul Assignee: Noble Paul Fix For: 5.0 Attachments: SOLR-5473-74 .patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74.patch, SOLR-5473-74_POC.patch, SOLR-5473-configname-fix.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473.patch, SOLR-5473_undo.patch, ec2-23-20-119-52_solr.log, ec2-50-16-38-73_solr.log As defined in the parent issue, store the states of each collection under /collections/collectionname/state.json node -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: SolrCloud on HDFS empty tlog hence doesn't replay after Solr process crash and restart
Any thought about this issue: Solr on HDFS generate empty tlog when add
documents without commit.
Thanks,
Tom
On Fri, Jul 18, 2014 at 12:21 PM, Tom Chen tomchen1...@gmail.com wrote:
Hi,
This seems a bug for Solr running on HDFS.
Reproduce steps:
1) Setup Solr to run on HDFS like this:
java -Dsolr.directoryFactory=HdfsDirectoryFactory
-Dsolr.lock.type=hdfs
-Dsolr.hdfs.home=hdfs://host:port/path
For the purpose of this testing, turn off the default auto commit in
solrconfig.xml, i.e. comment out autoCommit like this:
!--
autoCommit
maxTime${solr.autoCommit.maxTime:15000}/maxTime
openSearcherfalse/openSearcher
/autoCommit
--
2) Add a document without commit:
curl http://localhost:8983/solr/collection1/update?commit=false; -H
Content-type:text/xml; charset=utf-8 --data-binary @solr.xml
3) Solr generate empty tlog file (0 file size, the last one ends with 6):
[hadoop@hdtest042 exampledocs]$ hadoop fs -ls
/path/collection1/core_node1/data/tlog
Found 5 items
-rw-r--r-- 1 hadoop hadoop667 2014-07-18 08:47
/path/collection1/core_node1/data/tlog/tlog.001
-rw-r--r-- 1 hadoop hadoop 67 2014-07-18 08:47
/path/collection1/core_node1/data/tlog/tlog.003
-rw-r--r-- 1 hadoop hadoop667 2014-07-18 08:47
/path/collection1/core_node1/data/tlog/tlog.004
-rw-r--r-- 1 hadoop hadoop 0 2014-07-18 09:02
/path/collection1/core_node1/data/tlog/tlog.005
-rw-r--r-- 1 hadoop hadoop 0 2014-07-18 09:02
/path/collection1/core_node1/data/tlog/tlog.006
4) Simulate Solr crash by killing the process with -9 option.
5) restart the Solr process. Observation is that uncommitted document are
not replayed, files in tlog directory are cleaned up. Hence uncommitted
document(s) is lost.
Am I missing anything or this is a bug?
BTW, additional observations:
a) If in step 4) Solr is stopped gracefully (i.e. without -9 option),
non-empty tlog file is geneated and after re-starting Solr, uncommitted
document is replayed as expected.
b) If Solr doesn't run on HDFS (i.e. on local file system), this issue is
not observed either.
Thanks,
Tom
[jira] [Commented] (SOLR-3619) Rename 'example' dir to 'server' and pull examples into an 'examples' directory
[ https://issues.apache.org/jira/browse/SOLR-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068591#comment-14068591 ] Mark Miller commented on SOLR-3619: --- bq. Maybe it would be good to repeat that exercise and drop the discussion down to the lower level. No, please no! Let's just talk about a server directory. Let's not screw this issue by trying to do more than that. Been there. That can come later. bq. I am trying to understand what will be the new first command for a new user instead of java -jar start.jar. No! :) This issue doesn't change that. If we let this issue get bogged down with all that shit, it's dead just like last time and just like in other issues. bq. Rename 'example' dir to 'server' and pull examples into an 'examples' directory This doesn't require crazy new layouts! This doesn't require removing collection1! This doesn't require a new way to start Solr! Let's focus on this first. Rename 'example' dir to 'server' and pull examples into an 'examples' directory --- Key: SOLR-3619 URL: https://issues.apache.org/jira/browse/SOLR-3619 Project: Solr Issue Type: Improvement Reporter: Mark Miller Fix For: 4.9, 5.0 Attachments: SOLR-3619.patch, server-name-layout.png -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5205) [PATCH] SpanQueryParser with recursion, analysis and syntax very similar to classic QueryParser
[ https://issues.apache.org/jira/browse/LUCENE-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068596#comment-14068596 ] Tim Allison commented on LUCENE-5205: - Agreed on way ahead. Looks like I forgot to commit the few manual changes: BasicAutomata-Automata. I closed the broken pull request. Let me know if you have luck now. Sorry about that! [PATCH] SpanQueryParser with recursion, analysis and syntax very similar to classic QueryParser --- Key: LUCENE-5205 URL: https://issues.apache.org/jira/browse/LUCENE-5205 Project: Lucene - Core Issue Type: Improvement Components: core/queryparser Reporter: Tim Allison Labels: patch Fix For: 4.9 Attachments: LUCENE-5205-cleanup-tests.patch, LUCENE-5205-date-pkg-prvt.patch, LUCENE-5205.patch.gz, LUCENE-5205.patch.gz, LUCENE-5205_dateTestReInitPkgPrvt.patch, LUCENE-5205_improve_stop_word_handling.patch, LUCENE-5205_smallTestMods.patch, LUCENE_5205.patch, SpanQueryParser_v1.patch.gz, patch.txt This parser extends QueryParserBase and includes functionality from: * Classic QueryParser: most of its syntax * SurroundQueryParser: recursive parsing for near and not clauses. * ComplexPhraseQueryParser: can handle near queries that include multiterms (wildcard, fuzzy, regex, prefix), * AnalyzingQueryParser: has an option to analyze multiterms. At a high level, there's a first pass BooleanQuery/field parser and then a span query parser handles all terminal nodes and phrases. Same as classic syntax: * term: test * fuzzy: roam~0.8, roam~2 * wildcard: te?t, test*, t*st * regex: /\[mb\]oat/ * phrase: jakarta apache * phrase with slop: jakarta apache~3 * default or clause: jakarta apache * grouping or clause: (jakarta apache) * boolean and +/-: (lucene OR apache) NOT jakarta; +lucene +apache -jakarta * multiple fields: title:lucene author:hatcher Main additions in SpanQueryParser syntax vs. classic syntax: * Can require in order for phrases with slop with the \~ operator: jakarta apache\~3 * Can specify not near: fever bieber!\~3,10 :: find fever but not if bieber appears within 3 words before or 10 words after it. * Fully recursive phrasal queries with \[ and \]; as in: \[\[jakarta apache\]~3 lucene\]\~4 :: find jakarta within 3 words of apache, and that hit has to be within four words before lucene * Can also use \[\] for single level phrasal queries instead of as in: \[jakarta apache\] * Can use or grouping clauses in phrasal queries: apache (lucene solr)\~3 :: find apache and then either lucene or solr within three words. * Can use multiterms in phrasal queries: jakarta\~1 ap*che\~2 * Did I mention full recursion: \[\[jakarta\~1 ap*che\]\~2 (solr~ /l\[ou\]\+\[cs\]\[en\]\+/)]\~10 :: Find something like jakarta within two words of ap*che and that hit has to be within ten words of something like solr or that lucene regex. * Can require at least x number of hits at boolean level: apache AND (lucene solr tika)~2 * Can use negative only query: -jakarta :: Find all docs that don't contain jakarta * Can use an edit distance 2 for fuzzy query via SlowFuzzyQuery (beware of potential performance issues!). Trivial additions: * Can specify prefix length in fuzzy queries: jakarta~1,2 (edit distance =1, prefix =2) * Can specifiy Optimal String Alignment (OSA) vs Levenshtein for distance =2: (jakarta~1 (OSA) vs jakarta~1(Levenshtein) This parser can be very useful for concordance tasks (see also LUCENE-5317 and LUCENE-5318) and for analytical search. Until LUCENE-2878 is closed, this might have a use for fans of SpanQuery. Most of the documentation is in the javadoc for SpanQueryParser. Any and all feedback is welcome. Thank you. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068645#comment-14068645 ] Mark Miller commented on SOLR-6260: --- There is nothing person in anything I wrote. Feel free to pull out what you think is and I'll correct it. Reasonable people can disagree, and the same culture that had Lucene locked up for years in the mid 2000's can lock up Solr here. We need a couple things: * To transition to a SolrCloud first world where everything is sensible and not because of history baggage. * To encourage and develop new contributors. Pushing back on this simple issue and issues like it is part of an overall culture that hurts both those things. When I joined, I wanted to fix crap like this too. I was slowly beaten down over time until I just accepted that a lot of our code was crap. I stopped even trying to fix it. I won't be part of beating that out of a new generation of committers. This needs to be fixed. The benefits are bigger than you understand IMO. And the benefits of not doing it, are super minor. Solr needs to worry about the future, not the past. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068652#comment-14068652 ] Mark Miller commented on SOLR-6260: --- If we cannot come to consensus on this issue, I think we should put it to a vote. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5836) BytesRef.copyBytes and copyChars don't oversize
[ https://issues.apache.org/jira/browse/LUCENE-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068664#comment-14068664 ] Michael McCandless commented on LUCENE-5836: +1 for BytesRefBuilder! BytesRef.copyBytes and copyChars don't oversize --- Key: LUCENE-5836 URL: https://issues.apache.org/jira/browse/LUCENE-5836 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand Attachments: LUCENE-5836.patch When copying data from another BytesRef/CharSequence, these methods don't oversize. This is not an issue if this method is used only once per BytesRef instance but I just reviewed the usage of these methods and they are very frequently used in loops to do things like: - keep track of the top values in comparators - keep track of the previous terms in various loops over a terms enum (lucene49 DV consumer, BlockTreeTermsWriter) - etc. Although unlikely, it might be possible to hit a worst-case and to resize the underlying byte[] on every call to copyBytes? Should we oversize the underlying array in these methods? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068669#comment-14068669 ] Noble Paul commented on SOLR-6260: -- We should not hang on to an old name because it was there for some time +1 to make a name change Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_05) - Build # 10820 - Failure!
Ah ... this appears to be a slight variantion on LUCENE-5788.
: Date: Tue, 15 Jul 2014 09:01:42 -0700 (MST)
: From: Chris Hostetter hossman_luc...@fucit.org
: To: Lucene Dev dev@lucene.apache.org
: Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_05) - Build #
: 10820 - Failure!
:
:
: FYI: Seed reproduces for me on my 64bit linux java7...
:
: ant test -Dtestcase=TestFieldCacheSort
: -Dtests.method=testEmptyStringVsNullStringSort
: -Dtests.seed=A82FCB68C76B741B -Dtests.multiplier=3 -Dtests.slow=true
: -Dtests.locale=ru -Dtests.timezone=Africa/Lusaka
: -Dtests.file.encoding=UTF-8
:
:
: The failing asssert is assert fi != null
:
:
: public SortedDocValues getSortedDocValues(String field) throws IOException {
: SortedDocValues dv = super.getSortedDocValues(field);
: FieldInfo fi = getFieldInfos().fieldInfo(field);
: if (dv != null) {
: assert fi != null;
: assert fi.getDocValuesType() == FieldInfo.DocValuesType.SORTED;
: return new AssertingSortedDocValues(dv, maxDoc());
: } else {
: assert fi == null || fi.getDocValuesType() !=
FieldInfo.DocValuesType.SORTED;
: return null;
: }
: }
:
:
:
:
:
: : Date: Tue, 15 Jul 2014 05:36:03 + (UTC)
: : From: Policeman Jenkins Server jenk...@thetaphi.de
: : Reply-To: dev@lucene.apache.org
: : To: dev@lucene.apache.org
: : Subject: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_05) - Build #
10820
: : - Failure!
: :
: : Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10820/
: : Java: 32bit/jdk1.8.0_05 -client -XX:+UseParallelGC
: :
: : 1 tests failed.
: : REGRESSION:
org.apache.lucene.uninverting.TestFieldCacheSort.testEmptyStringVsNullStringSort
: :
: : Error Message:
: :
: :
: : Stack Trace:
: : java.lang.AssertionError
: : at
__randomizedtesting.SeedInfo.seed([A82FCB68C76B741B:C9CCF9BFBF0A72AB]:0)
: : at
org.apache.lucene.index.AssertingAtomicReader.getSortedDocValues(AssertingAtomicReader.java:638)
: : at
org.apache.lucene.index.FilterAtomicReader.getSortedDocValues(FilterAtomicReader.java:414)
: : at
org.apache.lucene.index.AssertingAtomicReader.getSortedDocValues(AssertingAtomicReader.java:635)
: : at org.apache.lucene.index.DocValues.getSorted(DocValues.java:273)
: : at
org.apache.lucene.search.FieldComparator$TermOrdValComparator.getSortedDocValues(FieldComparator.java:821)
: : at
org.apache.lucene.search.FieldComparator$TermOrdValComparator.setNextReader(FieldComparator.java:826)
: : at
org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.doSetNextReader(TopFieldCollector.java:97)
: : at
org.apache.lucene.search.SimpleCollector.getLeafCollector(SimpleCollector.java:33)
: : at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:605)
: : at
org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:94)
: : at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:573)
: : at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:525)
: : at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:502)
: : at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:318)
: : at
org.apache.lucene.uninverting.TestFieldCacheSort.testEmptyStringVsNullStringSort(TestFieldCacheSort.java:1029)
: : at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: : at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
: : at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: : at java.lang.reflect.Method.invoke(Method.java:483)
: : at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
: : at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
: : at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
: : at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
: : at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
: : at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
: : at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
: : at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
: : at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
: : at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
: : at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: : at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
: :
[jira] [Commented] (SOLR-3345) BaseDistributedSearchTestCase should always ignore QTime
[ https://issues.apache.org/jira/browse/SOLR-3345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068697#comment-14068697 ] Mark Miller commented on SOLR-3345: --- Could you put up an SVN patch Vamsee? I'm having a tough time getting this to apply to an SVN checkout properly. BaseDistributedSearchTestCase should always ignore QTime Key: SOLR-3345 URL: https://issues.apache.org/jira/browse/SOLR-3345 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0-ALPHA Reporter: Benson Margulies Assignee: Mark Miller Attachments: SOLR-3345.patch The existing subclasses of BaseDistributedSearchTestCase all skip QTime. I can't see any way in which those numbers will ever match. Why not make this the default, or only, behavior? (This is really a question, in that I will provide a patch if no one tells me that it is a bad idea.) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3619) Rename 'example' dir to 'server' and pull examples into an 'examples' directory
[
https://issues.apache.org/jira/browse/SOLR-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068711#comment-14068711
]
Mark Miller commented on SOLR-3619:
---
{quote}I do have lots of comments and ideas on general UX/experience
improvement but it can wait for a personal project, another JIRA or an evening
huddle at the conference.{quote}
Start filing JIRAs and linking them as related to this :) We have a lot to do.
The problem is, every issue can be contentious depending on who gets involved.
I think to start, we will have to go very step at a time. Like here: let's see
what we have to do to get a sever folder while making as few other changes as
possible. Even if it can only go into 5x. We can figure out how to do that with
minimal disruption I think. And that gives a toe hold to start pushing from.
In the meantime, let's discuss all these other ideas in related issues. We
should discuss them, it's just dangerous to keep weighing this issue down with
that discussion - that type of thing can put an issue on ice.
Some JIRA's are already out there, like adding start scripts, and I think
removing collection1, etc.
Rename 'example' dir to 'server' and pull examples into an 'examples'
directory
---
Key: SOLR-3619
URL: https://issues.apache.org/jira/browse/SOLR-3619
Project: Solr
Issue Type: Improvement
Reporter: Mark Miller
Fix For: 4.9, 5.0
Attachments: SOLR-3619.patch, server-name-layout.png
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: SolrCloud on HDFS empty tlog hence doesn't replay after Solr process crash and restart
It’s on my list to investigate. -- Mark Miller about.me/markrmiller On July 21, 2014 at 10:26:09 AM, Tom Chen (tomchen1...@gmail.com) wrote: Any thought about this issue: Solr on HDFS generate empty tlog when add documents without commit. Thanks, Tom On Fri, Jul 18, 2014 at 12:21 PM, Tom Chen wrote: Hi, This seems a bug for Solr running on HDFS. Reproduce steps: 1) Setup Solr to run on HDFS like this: java -Dsolr.directoryFactory=HdfsDirectoryFactory -Dsolr.lock.type=hdfs -Dsolr.hdfs.home=hdfs://host:port/path For the purpose of this testing, turn off the default auto commit in solrconfig.xml, i.e. comment out autoCommit like this: 2) Add a document without commit: curl http://localhost:8983/solr/collection1/update?commit=false; -H Content-type:text/xml; charset=utf-8 --data-binary @solr.xml 3) Solr generate empty tlog file (0 file size, the last one ends with 6): [hadoop@hdtest042 exampledocs]$ hadoop fs -ls /path/collection1/core_node1/data/tlog Found 5 items -rw-r--r-- 1 hadoop hadoop 667 2014-07-18 08:47 /path/collection1/core_node1/data/tlog/tlog.001 -rw-r--r-- 1 hadoop hadoop 67 2014-07-18 08:47 /path/collection1/core_node1/data/tlog/tlog.003 -rw-r--r-- 1 hadoop hadoop 667 2014-07-18 08:47 /path/collection1/core_node1/data/tlog/tlog.004 -rw-r--r-- 1 hadoop hadoop 0 2014-07-18 09:02 /path/collection1/core_node1/data/tlog/tlog.005 -rw-r--r-- 1 hadoop hadoop 0 2014-07-18 09:02 /path/collection1/core_node1/data/tlog/tlog.006 4) Simulate Solr crash by killing the process with -9 option. 5) restart the Solr process. Observation is that uncommitted document are not replayed, files in tlog directory are cleaned up. Hence uncommitted document(s) is lost. Am I missing anything or this is a bug? BTW, additional observations: a) If in step 4) Solr is stopped gracefully (i.e. without -9 option), non-empty tlog file is geneated and after re-starting Solr, uncommitted document is replayed as expected. b) If Solr doesn't run on HDFS (i.e. on local file system), this issue is not observed either. Thanks, Tom - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5596) OverseerTest.testOverseerFailure - leader node already exists.
[ https://issues.apache.org/jira/browse/SOLR-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068747#comment-14068747 ] Mark Miller commented on SOLR-5596: --- Yeah, I think this is the same result as when I tried to remove the forceSync - still happens: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4201/ OverseerTest.testOverseerFailure - leader node already exists. -- Key: SOLR-5596 URL: https://issues.apache.org/jira/browse/SOLR-5596 Project: Solr Issue Type: Bug Reporter: Mark Miller Assignee: Shalin Shekhar Mangar Fix For: 4.9, 5.0 Seeing this a bunch on jenkins - previous leader ephemeral node is still around for some reason. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6252) Simplify UnInvertedField#getUnInvertedField synchronization module
[ https://issues.apache.org/jira/browse/SOLR-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068762#comment-14068762 ] Mark Miller commented on SOLR-6252: --- Yeah, sounds reasonable to me. Can you take one more pass [~vamsee]? Simplify UnInvertedField#getUnInvertedField synchronization module -- Key: SOLR-6252 URL: https://issues.apache.org/jira/browse/SOLR-6252 Project: Solr Issue Type: Improvement Components: search Affects Versions: 5.0 Reporter: Vamsee Yarlagadda Assignee: Mark Miller Priority: Minor Attachments: SOLR-6252.patch, SOLR-6252v2.patch Looks like UnInvertedField#getUnInvertedField has implemented a bit additional synchronization module rather than what is required, and thereby increasing the complexity. https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/request/UnInvertedField.java#L667 As pointed out in the above link, as the synchronization is performed on the cache variable(which itself will protect the threads from obtaining access to the cache), we can safely remove all the placeholder flags. As long as cache.get() is in synchronized block, we can simply populate the cache with new entries and other threads will be able to see the changes. This change has been introduced in https://issues.apache.org/jira/browse/SOLR-2548 (Multithreaded faceting) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6252) Simplify UnInvertedField#getUnInvertedField synchronization module
[ https://issues.apache.org/jira/browse/SOLR-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068795#comment-14068795 ] Gregory Chanan commented on SOLR-6252: -- Also, the synchronized lock is held when checking the placeholder, when it really only needs to be held for the get/put. Simplify UnInvertedField#getUnInvertedField synchronization module -- Key: SOLR-6252 URL: https://issues.apache.org/jira/browse/SOLR-6252 Project: Solr Issue Type: Improvement Components: search Affects Versions: 5.0 Reporter: Vamsee Yarlagadda Assignee: Mark Miller Priority: Minor Attachments: SOLR-6252.patch, SOLR-6252v2.patch Looks like UnInvertedField#getUnInvertedField has implemented a bit additional synchronization module rather than what is required, and thereby increasing the complexity. https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/request/UnInvertedField.java#L667 As pointed out in the above link, as the synchronization is performed on the cache variable(which itself will protect the threads from obtaining access to the cache), we can safely remove all the placeholder flags. As long as cache.get() is in synchronized block, we can simply populate the cache with new entries and other threads will be able to see the changes. This change has been introduced in https://issues.apache.org/jira/browse/SOLR-2548 (Multithreaded faceting) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5838) hunspell buggy with over 64k affixes
[ https://issues.apache.org/jira/browse/LUCENE-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068799#comment-14068799 ] ASF subversion and git services commented on LUCENE-5838: - Commit 1612349 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1612349 ] LUCENE-5838: fix hunspell when .aff file has over 64k affixes hunspell buggy with over 64k affixes Key: LUCENE-5838 URL: https://issues.apache.org/jira/browse/LUCENE-5838 Project: Lucene - Core Issue Type: Bug Reporter: Robert Muir Attachments: LUCENE-5838.patch, LUCENE-5838.patch currently we build TreeMapString,ListCharacter in ram, to sort before adding to the FST (which encodes the list as IntsRef). char overflows here if there are more than 64k affixes (e.g. basque). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6252) Simplify UnInvertedField#getUnInvertedField synchronization module
[ https://issues.apache.org/jira/browse/SOLR-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068795#comment-14068795 ] Gregory Chanan edited comment on SOLR-6252 at 7/21/14 5:13 PM: --- Also, the synchronized lock is held when checking the placeholder, when it really only needs to be held for the get/put. (it's subtle to get that correct though, because the non-placeholder needs to be notified correctly, so probably not worth doing. was (Author: gchanan): Also, the synchronized lock is held when checking the placeholder, when it really only needs to be held for the get/put. Simplify UnInvertedField#getUnInvertedField synchronization module -- Key: SOLR-6252 URL: https://issues.apache.org/jira/browse/SOLR-6252 Project: Solr Issue Type: Improvement Components: search Affects Versions: 5.0 Reporter: Vamsee Yarlagadda Assignee: Mark Miller Priority: Minor Attachments: SOLR-6252.patch, SOLR-6252v2.patch Looks like UnInvertedField#getUnInvertedField has implemented a bit additional synchronization module rather than what is required, and thereby increasing the complexity. https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/request/UnInvertedField.java#L667 As pointed out in the above link, as the synchronization is performed on the cache variable(which itself will protect the threads from obtaining access to the cache), we can safely remove all the placeholder flags. As long as cache.get() is in synchronized block, we can simply populate the cache with new entries and other threads will be able to see the changes. This change has been introduced in https://issues.apache.org/jira/browse/SOLR-2548 (Multithreaded faceting) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6263) Add handler name accessibility
Erik Hatcher created SOLR-6263: -- Summary: Add handler name accessibility Key: SOLR-6263 URL: https://issues.apache.org/jira/browse/SOLR-6263 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 4.9 Reporter: Erik Hatcher Fix For: 5.0, 4.10 Currently the DIH handler name is buried and private/package accessible. It's useful to allow plugins access to it though. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5838) hunspell buggy with over 64k affixes
[ https://issues.apache.org/jira/browse/LUCENE-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068824#comment-14068824 ] ASF subversion and git services commented on LUCENE-5838: - Commit 1612351 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1612351 ] LUCENE-5838: fix hunspell when .aff file has over 64k affixes hunspell buggy with over 64k affixes Key: LUCENE-5838 URL: https://issues.apache.org/jira/browse/LUCENE-5838 Project: Lucene - Core Issue Type: Bug Reporter: Robert Muir Fix For: 5.0, 4.10 Attachments: LUCENE-5838.patch, LUCENE-5838.patch currently we build TreeMapString,ListCharacter in ram, to sort before adding to the FST (which encodes the list as IntsRef). char overflows here if there are more than 64k affixes (e.g. basque). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-5838) hunspell buggy with over 64k affixes
[ https://issues.apache.org/jira/browse/LUCENE-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-5838. - Resolution: Fixed Fix Version/s: 4.10 5.0 hunspell buggy with over 64k affixes Key: LUCENE-5838 URL: https://issues.apache.org/jira/browse/LUCENE-5838 Project: Lucene - Core Issue Type: Bug Reporter: Robert Muir Fix For: 5.0, 4.10 Attachments: LUCENE-5838.patch, LUCENE-5838.patch currently we build TreeMapString,ListCharacter in ram, to sort before adding to the FST (which encodes the list as IntsRef). char overflows here if there are more than 64k affixes (e.g. basque). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6263) Add handler name accessibility
[
https://issues.apache.org/jira/browse/SOLR-6263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068828#comment-14068828
]
ASF subversion and git services commented on SOLR-6263:
---
Commit 1612353 from [~ehatcher] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1612353 ]
SOLR-6263: Add DIH handler name to variable resolver as ${dih.handlerName}
Add handler name accessibility
--
Key: SOLR-6263
URL: https://issues.apache.org/jira/browse/SOLR-6263
Project: Solr
Issue Type: Improvement
Components: contrib - DataImportHandler
Affects Versions: 4.9
Reporter: Erik Hatcher
Fix For: 5.0, 4.10
Currently the DIH handler name is buried and private/package accessible.
It's useful to allow plugins access to it though.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6263) Add handler name accessibility
[
https://issues.apache.org/jira/browse/SOLR-6263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068839#comment-14068839
]
ASF subversion and git services commented on SOLR-6263:
---
Commit 1612354 from [~ehatcher] in branch 'dev/trunk'
[ https://svn.apache.org/r1612354 ]
SOLR-6263: Add DIH handler name to variable resolver as ${dih.handlerName}
Add handler name accessibility
--
Key: SOLR-6263
URL: https://issues.apache.org/jira/browse/SOLR-6263
Project: Solr
Issue Type: Improvement
Components: contrib - DataImportHandler
Affects Versions: 4.9
Reporter: Erik Hatcher
Fix For: 5.0, 4.10
Currently the DIH handler name is buried and private/package accessible.
It's useful to allow plugins access to it though.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6263) Add handler name accessibility
[ https://issues.apache.org/jira/browse/SOLR-6263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Hatcher resolved SOLR-6263. Resolution: Fixed Add handler name accessibility -- Key: SOLR-6263 URL: https://issues.apache.org/jira/browse/SOLR-6263 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 4.9 Reporter: Erik Hatcher Fix For: 5.0, 4.10 Currently the DIH handler name is buried and private/package accessible. It's useful to allow plugins access to it though. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069029#comment-14069029 ] Yonik Seeley commented on SOLR-6260: -1 The benefits to this simple class rename are blown way out of proportion by fallaciously linking it to real improvements. Further, if you're worried about attracting new committers, fostering a more civil (and less aggressive and combative) environment would go a long way. Although there are multiple minor downsides (countering the minor upside), I'll remove my -1 if we can at least implement back compat with existing solrconfigs. It should be trivial. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069051#comment-14069051 ] Shalin Shekhar Mangar commented on SOLR-6260: - -1 to breaking compatibility in 4.x (but I'm quite sure we weren't debating that, were we?) bq. Further, if you're worried about attracting new committers, fostering a more civil (and less aggressive and combative) environment would go a long way. Big +1 and thank you for putting it so well. This has become a big pain point. We're all working towards the same goal of making Solr better, we can surely be more courteous to each other. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069062#comment-14069062 ] Tomás Fernández Löbbe commented on SOLR-6260: - bq. I'll remove my -1 if we can at least implement back compat with existing solrconfigs. Sure, I can do this bq. -1 to breaking compatibility in 4.x (but I'm quite sure we weren't debating that, were we?) Yes, I was thinking on doing this change in trunk only. If merged to 4.x it should have been in a backward compatible way. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5968) BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields are requested
[ https://issues.apache.org/jira/browse/SOLR-5968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069063#comment-14069063 ] Gregg Donovan commented on SOLR-5968: - Thanks, Shalin! BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields are requested Key: SOLR-5968 URL: https://issues.apache.org/jira/browse/SOLR-5968 Project: Solr Issue Type: Bug Reporter: Gregg Donovan Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 5.0, 4.10 Attachments: SOLR-5968-NOCOMMIT.diff, SOLR-5968.diff If fl contains only pseudo fields, BinaryResponseWriter will load unnecessary stored fields before discarding them. The call to SolrIndexSearcher#doc(int i, SetString fields) can be skipped for this case. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069064#comment-14069064 ] Mark Miller commented on SOLR-6260: --- bq. I'll remove my -1 if we can at least implement back compat with existing solrconfigs. It should be trivial. Of course we would handle back compat where it makes sense. That's an absurd condition. In any case, if you want to -1 over a vote that shows Solr developers want this change, be my guest. We will see how long that works out for you. bq. Further, if you're worried about attracting new committers, I think the current culture of push back is worse than combating those that foster the pushback. I plan on continuing to push very hard for change in our obstructionist culture - one that I've played my part in at times too. It's necessary. Being aggressive is necessary. I can find a variety of issues that Tomas has filed that have a quick wet blanket thrown on them. We have done everything we can to push him away rather than pull him in from where I am standing. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller reassigned SOLR-6260: - Assignee: Mark Miller Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Assignee: Mark Miller Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069076#comment-14069076 ] Yonik Seeley commented on SOLR-6260: bq. Being aggressive is necessary. I disagree. It wasn't necessary in the early days of Lucene/Solr. Although, I'll partially agree - the culture has changed such that one often does now need to be aggressive else those more aggressive will run right over you. It's sad. It makes me enjoy participating here much less. But it's only currently necessary because the culture has changed to allow it. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Assignee: Mark Miller Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069078#comment-14069078 ] Yonik Seeley commented on SOLR-6260: bq. Yes, I was thinking on doing this change in trunk only. If merged to 4.x it should have been in a backward compatible way. I meant implementing back compat in trunk too - making it easy for users to upgrade is important (as long as it's not a burden of course.) Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Assignee: Mark Miller Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069079#comment-14069079 ] Mark Miller commented on SOLR-6260: --- It's currently necessary because the old culture was obstructionist. Same as the old Lucene culture. It's a great culture for maintenance mode. Lucene changed that because of a couple aggressive people. The results speak for themselves. I've watched people spin in circles for years trying another approach. We have made very little progress on many simple, common sense issues. I'm ready to make some progress or go home. To let contributors help us. To get their patches in. To help them work around problems rather than just put up easy to lay down road blocks. We have a lot to do. Our current pace, the pace of the past 5+ years, it won't cut it. It's no good. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Assignee: Mark Miller Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069110#comment-14069110 ] Mark Miller commented on SOLR-6260: --- bq. We're all working towards the same goal of making Solr better ... more courteous to each other. We all have a variety of different goals and motivations actually. Let's not pretend everyone is simply working towards making Solr better. That's a lot of people and changing motivations to account for. Where are people not being courteous to each other in this thread? The absence of lively debate tends to mean one thing: people don't care. Now Yonik seems to have taken my broader message personally - the message of , We tend to push back on everything and not help contributors, especially promising contributors, get things done. We favor obstructionism because it's low effort. But it's not meant for him personally. Everyone has played their part in this. I'm not quite sure where I called anyone out for any personal behavior. I called out what I think is happening to Tomas - the wet blanket responses he has received. I think the behavior of the project must change. That these issues must get in. And if you think that happens easily, guess again. I can find a lot of issues that are killed with a comment or two. It takes something like this to get these issues over the hump - to make an issue out of this. To cause discussion around this. To move forward. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Assignee: Mark Miller Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069167#comment-14069167 ] Yonik Seeley commented on SOLR-6260: bq. It takes something like this to get these issues over the hump - to make an issue out of this. See, I was looking at this issue just based on it's technical merits. That should always be OK. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Assignee: Mark Miller Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069183#comment-14069183 ] Mark Miller commented on SOLR-6260: --- That was okay. But so is my coming to try and make the issue happen anyway. When someone with enough merit comes in and only mentions, I'm pretty much against this, it has a strong chilling effect! If you had come in with, I have these concerns, but if we do this or this and/or figure out this, we could probably make this happen, I wouldn't have felt so much that I have to try and make sure this issue still happens. I've just seen too many of these issues die summarily with light and easy pushback. For some crap JIRAs, perhaps that is sensible. Given the history of Tomas and his recent contributions and the quality of work I have seen and my own personal history and interest in this change...here we are! However, I'm not saying you shouldn't have done whatever you want to do. I'm responding to it. I'm trying to start a tone going forward that will let us dump single core and non cloud mode and all the baggage that makes the system not as beautiful as it needs to be. Developers want to work on beautiful, not ugly. That's our largest issue. Not any internal community communication. Now when I look at the pace of this happening, we will be done in 5-10 years. Perhaps. It's not going to cut it. We will be marginalized. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Assignee: Mark Miller Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069198#comment-14069198 ] Jack Krupansky commented on SOLR-6260: -- I noticed that the SolrCore code does in fact default the update handler class to DIH2/SIH if the class attribute is not specified, so maybe the upgrade instructions can simply be for users to remove the updateHandler class attribute, rather than for them to have to learn yet another internal name. And I would reiterate my proposal to remove the class attribute from the example solrconfig.xml files, for both 5.0 and 4.x. Either way, the patch should include changes to the Upgrading section of CHANGES.txt. Do those three things and then I'm an easy +1! Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Assignee: Mark Miller Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069202#comment-14069202 ] Yonik Seeley commented on SOLR-6260: bq. I noticed that the SolrCore code does in fact default the update handler class to DIH2/SIH if the class attribute is not specified Then we should probably remove it in the example solrconfigs. That will also lessen the pain next time someone comes up with a better name ;-) Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Assignee: Mark Miller Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069198#comment-14069198 ] Jack Krupansky edited comment on SOLR-6260 at 7/21/14 8:20 PM: --- I noticed that the SolrCore code does in fact default the update handler class to DUH2/SUH if the class attribute is not specified, so maybe the upgrade instructions can simply be for users to remove the updateHandler class attribute, rather than for them to have to learn yet another internal name. And I would reiterate my proposal to remove the class attribute from the example solrconfig.xml files, for both 5.0 and 4.x. Either way, the patch should include changes to the Upgrading section of CHANGES.txt. Do those three things and then I'm an easy +1! was (Author: jkrupan): I noticed that the SolrCore code does in fact default the update handler class to DIH2/SIH if the class attribute is not specified, so maybe the upgrade instructions can simply be for users to remove the updateHandler class attribute, rather than for them to have to learn yet another internal name. And I would reiterate my proposal to remove the class attribute from the example solrconfig.xml files, for both 5.0 and 4.x. Either way, the patch should include changes to the Upgrading section of CHANGES.txt. Do those three things and then I'm an easy +1! Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Assignee: Mark Miller Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5986) Don't allow runaway queries from harming Solr cluster health or search performance
[ https://issues.apache.org/jira/browse/SOLR-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069225#comment-14069225 ] Jim Walker commented on SOLR-5986: -- Steve, good point regarding automation. That should come first. I just talked to Anshum who has this covered; he will know what needs to happen here best. Cheers Don't allow runaway queries from harming Solr cluster health or search performance -- Key: SOLR-5986 URL: https://issues.apache.org/jira/browse/SOLR-5986 Project: Solr Issue Type: Improvement Components: search Reporter: Steve Davids Priority: Critical Fix For: 4.9 The intent of this ticket is to have all distributed search requests stop wasting CPU cycles on requests that have already timed out or are so complicated that they won't be able to execute. We have come across a case where a nasty wildcard query within a proximity clause was causing the cluster to enumerate terms for hours even though the query timeout was set to minutes. This caused a noticeable slowdown within the system which made us restart the replicas that happened to service that one request, the worst case scenario are users with a relatively low zk timeout value will have nodes start dropping from the cluster due to long GC pauses. [~amccurry] Built a mechanism into Apache Blur to help with the issue in BLUR-142 (see commit comment for code, though look at the latest code on the trunk for newer bug fixes). Solr should be able to either prevent these problematic queries from running by some heuristic (possibly estimated size of heap usage) or be able to execute a thread interrupt on all query threads once the time threshold is met. This issue mirrors what others have discussed on the mailing list: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200903.mbox/%3c856ac15f0903272054q2dbdbd19kea3c5ba9e105b...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6260) Rename DirectUpdateHandler2
[ https://issues.apache.org/jira/browse/SOLR-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069233#comment-14069233 ] Mark Miller commented on SOLR-6260: --- bq. That will also lessen the pain next time someone comes up with a better name That's an important part of this issue too. We should clearly document that this is an expert and internal class. We need to own this, and right now it's in a grey area. What better time to clear up that it's internal and not user supported and rename it all in one. Take it out of config, document at as expert/internal. Unless someone has a burning desire to call it NewUpdateHandler3 in a couple years, I think a new name will stick for some time. Anytime you name a class Foo2, I think you can expect someone is going to want to fix it later. Call it Foo or Bar and you will get less name change demand. Rename DirectUpdateHandler2 --- Key: SOLR-6260 URL: https://issues.apache.org/jira/browse/SOLR-6260 Project: Solr Issue Type: Improvement Affects Versions: 5.0 Reporter: Tomás Fernández Löbbe Assignee: Mark Miller Priority: Minor Attachments: SOLR-6260.patch, SOLR-6260.patch DirectUpdateHandler was removed, I think in Solr 4. DirectUpdateHandler2 should be renamed, at least remove that 2. I don't know really what direct means here. Maybe it could be renamed to DefaultUpdateHandler, or UpdateHandlerDefaultImpl, or other good suggestions -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5746) solr.xml parsing of str vs int vs bool is brittle; fails silently; expects odd type for shareSchema
[
https://issues.apache.org/jira/browse/SOLR-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069345#comment-14069345
]
Hoss Man commented on SOLR-5746:
bq. My only review comment would be to prefer using hamcrest assertThat()
functions in tests, rather than assertTrue(), just because you get nicer error
messages when they fail. ...
the assertTrue's actually do have messages on them, and they are actual value
asserts that just happen to be true -- but i'll change them to assertEquals so
it's more obvious what's happening if/when they fail.
going to commit as soon as my updated to trunk started running all tests
passes again.
solr.xml parsing of str vs int vs bool is brittle; fails silently;
expects odd type for shareSchema
--
Key: SOLR-5746
URL: https://issues.apache.org/jira/browse/SOLR-5746
Project: Solr
Issue Type: Bug
Affects Versions: 4.3, 4.4, 4.5, 4.6
Reporter: Hoss Man
Attachments: SOLR-5746.patch, SOLR-5746.patch, SOLR-5746.patch,
SOLR-5746.patch, SOLR-5746.patch
A comment in the ref guide got me looking at ConfigSolrXml.java and noticing
that the parsing of solr.xml options here is very brittle and confusing. In
particular:
* if a boolean option foo is expected along the lines of {{bool
name=footrue/bool}} it will silently ignore {{str
name=footrue/str}}
* likewise for an int option {{int name=bar32/int}} vs {{str
name=bar32/str}}
... this is inconsistent with the way solrconfig.xml is parsed. In
solrconfig.xml, the xml nodes are parsed into a NamedList, and the above
options will work in either form, but an invalid value such as {{bool
name=fooNOT A BOOLEAN/bool}} will generate an error earlier (when
parsing config) then {{str name=fooNOT A BOOLEAN/str}} (attempt to
parse the string as a bool the first time the config value is needed)
In addition, i notice this really confusing line...
{code}
propMap.put(CfgProp.SOLR_SHARESCHEMA,
doSub(solr/str[@name='shareSchema']));
{code}
shareSchema is used internally as a boolean option, but as written the
parsing code will ignore it unless the user explicitly configures it as a
{{str/}}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-6261) Run checkIfIamLeader in a separate thread
[ https://issues.apache.org/jira/browse/SOLR-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller reassigned SOLR-6261: - Assignee: Mark Miller Run checkIfIamLeader in a separate thread - Key: SOLR-6261 URL: https://issues.apache.org/jira/browse/SOLR-6261 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.9 Reporter: Ramkumar Aiyengar Assignee: Mark Miller Priority: Minor Currently checking for leadership (due to the leader's ephemeral node going away) happens in ZK's event thread. If there are many cores and all of them are due leadership, then they would have to serially go through the two-way sync and leadership takeover. For tens of cores, this could mean 30-40s without leadership before the last in the list even gets to start the leadership process. If the leadership process happens in a separate thread, then the cores could all take over in parallel. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6261) Run checkIfIamLeader in a separate thread
[ https://issues.apache.org/jira/browse/SOLR-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069369#comment-14069369 ] Mark Miller commented on SOLR-6261: --- Hmm...I'm a little hesitant to fire up a new thread for every one rather than use the Update executor or something. Seems like a good step forward though. Run checkIfIamLeader in a separate thread - Key: SOLR-6261 URL: https://issues.apache.org/jira/browse/SOLR-6261 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.9 Reporter: Ramkumar Aiyengar Assignee: Mark Miller Priority: Minor Currently checking for leadership (due to the leader's ephemeral node going away) happens in ZK's event thread. If there are many cores and all of them are due leadership, then they would have to serially go through the two-way sync and leadership takeover. For tens of cores, this could mean 30-40s without leadership before the last in the list even gets to start the leadership process. If the leadership process happens in a separate thread, then the cores could all take over in parallel. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6252) Simplify UnInvertedField#getUnInvertedField synchronization module
[ https://issues.apache.org/jira/browse/SOLR-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vamsee Yarlagadda updated SOLR-6252: Attachment: SOLR-6252-v3.patch Simplify UnInvertedField#getUnInvertedField synchronization module -- Key: SOLR-6252 URL: https://issues.apache.org/jira/browse/SOLR-6252 Project: Solr Issue Type: Improvement Components: search Affects Versions: 5.0 Reporter: Vamsee Yarlagadda Assignee: Mark Miller Priority: Minor Attachments: SOLR-6252-v3.patch, SOLR-6252.patch, SOLR-6252v2.patch Looks like UnInvertedField#getUnInvertedField has implemented a bit additional synchronization module rather than what is required, and thereby increasing the complexity. https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/request/UnInvertedField.java#L667 As pointed out in the above link, as the synchronization is performed on the cache variable(which itself will protect the threads from obtaining access to the cache), we can safely remove all the placeholder flags. As long as cache.get() is in synchronized block, we can simply populate the cache with new entries and other threads will be able to see the changes. This change has been introduced in https://issues.apache.org/jira/browse/SOLR-2548 (Multithreaded faceting) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6252) Simplify UnInvertedField#getUnInvertedField synchronization module
[ https://issues.apache.org/jira/browse/SOLR-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069392#comment-14069392 ] Vamsee Yarlagadda commented on SOLR-6252: - [~markrmil...@gmail.com] Good point! Updated the patch to reflect the suggestions. Thanks [~gchanan]. Simplify UnInvertedField#getUnInvertedField synchronization module -- Key: SOLR-6252 URL: https://issues.apache.org/jira/browse/SOLR-6252 Project: Solr Issue Type: Improvement Components: search Affects Versions: 5.0 Reporter: Vamsee Yarlagadda Assignee: Mark Miller Priority: Minor Attachments: SOLR-6252-v3.patch, SOLR-6252.patch, SOLR-6252v2.patch Looks like UnInvertedField#getUnInvertedField has implemented a bit additional synchronization module rather than what is required, and thereby increasing the complexity. https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/request/UnInvertedField.java#L667 As pointed out in the above link, as the synchronization is performed on the cache variable(which itself will protect the threads from obtaining access to the cache), we can safely remove all the placeholder flags. As long as cache.get() is in synchronized block, we can simply populate the cache with new entries and other threads will be able to see the changes. This change has been introduced in https://issues.apache.org/jira/browse/SOLR-2548 (Multithreaded faceting) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5746) solr.xml parsing of str vs int vs bool is brittle; fails silently; expects odd type for shareSchema
[
https://issues.apache.org/jira/browse/SOLR-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069422#comment-14069422
]
ASF subversion and git services commented on SOLR-5746:
---
Commit 1612419 from hoss...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1612419 ]
SOLR-5746: Bugs in solr.xml parsing have been fixed to more correctly deal with
the various datatypes of options people can specify, additional error handling
of duplicated/unidentified options has also been added
solr.xml parsing of str vs int vs bool is brittle; fails silently;
expects odd type for shareSchema
--
Key: SOLR-5746
URL: https://issues.apache.org/jira/browse/SOLR-5746
Project: Solr
Issue Type: Bug
Affects Versions: 4.3, 4.4, 4.5, 4.6
Reporter: Hoss Man
Attachments: SOLR-5746.patch, SOLR-5746.patch, SOLR-5746.patch,
SOLR-5746.patch, SOLR-5746.patch
A comment in the ref guide got me looking at ConfigSolrXml.java and noticing
that the parsing of solr.xml options here is very brittle and confusing. In
particular:
* if a boolean option foo is expected along the lines of {{bool
name=footrue/bool}} it will silently ignore {{str
name=footrue/str}}
* likewise for an int option {{int name=bar32/int}} vs {{str
name=bar32/str}}
... this is inconsistent with the way solrconfig.xml is parsed. In
solrconfig.xml, the xml nodes are parsed into a NamedList, and the above
options will work in either form, but an invalid value such as {{bool
name=fooNOT A BOOLEAN/bool}} will generate an error earlier (when
parsing config) then {{str name=fooNOT A BOOLEAN/str}} (attempt to
parse the string as a bool the first time the config value is needed)
In addition, i notice this really confusing line...
{code}
propMap.put(CfgProp.SOLR_SHARESCHEMA,
doSub(solr/str[@name='shareSchema']));
{code}
shareSchema is used internally as a boolean option, but as written the
parsing code will ignore it unless the user explicitly configures it as a
{{str/}}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6261) Run checkIfIamLeader in a separate thread
[ https://issues.apache.org/jira/browse/SOLR-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069428#comment-14069428 ] Ramkumar Aiyengar commented on SOLR-6261: - Yeah, I thought of pooling this up as well initially, but then this is really a function of number of cores in the instance and a lot of threadpools are a function of the number of cores already? Can still look into changing it.. Run checkIfIamLeader in a separate thread - Key: SOLR-6261 URL: https://issues.apache.org/jira/browse/SOLR-6261 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.9 Reporter: Ramkumar Aiyengar Assignee: Mark Miller Priority: Minor Currently checking for leadership (due to the leader's ephemeral node going away) happens in ZK's event thread. If there are many cores and all of them are due leadership, then they would have to serially go through the two-way sync and leadership takeover. For tens of cores, this could mean 30-40s without leadership before the last in the list even gets to start the leadership process. If the leadership process happens in a separate thread, then the cores could all take over in parallel. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6261) Run checkIfIamLeader in a separate thread
[ https://issues.apache.org/jira/browse/SOLR-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069428#comment-14069428 ] Ramkumar Aiyengar edited comment on SOLR-6261 at 7/21/14 10:28 PM: --- Yeah, I thought of pooling this up as well initially, but then this is really a function of number of cores in the instance and a lot of threads used by Solr are really a function of the number of cores anyway? Can still look into changing it.. was (Author: andyetitmoves): Yeah, I thought of pooling this up as well initially, but then this is really a function of number of cores in the instance and a lot of threadpools are a function of the number of cores already? Can still look into changing it.. Run checkIfIamLeader in a separate thread - Key: SOLR-6261 URL: https://issues.apache.org/jira/browse/SOLR-6261 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.9 Reporter: Ramkumar Aiyengar Assignee: Mark Miller Priority: Minor Currently checking for leadership (due to the leader's ephemeral node going away) happens in ZK's event thread. If there are many cores and all of them are due leadership, then they would have to serially go through the two-way sync and leadership takeover. For tens of cores, this could mean 30-40s without leadership before the last in the list even gets to start the leadership process. If the leadership process happens in a separate thread, then the cores could all take over in parallel. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6261) Run checkIfIamLeader in a separate thread
[ https://issues.apache.org/jira/browse/SOLR-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069436#comment-14069436 ] Mark Miller commented on SOLR-6261: --- I dunno - I struggled with it when I first saw it and quickly got lazy about it. Something nicer about it, I think it's best to use pools to spin up threads, but I have a hard time worrying about it too much in this case. Run checkIfIamLeader in a separate thread - Key: SOLR-6261 URL: https://issues.apache.org/jira/browse/SOLR-6261 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.9 Reporter: Ramkumar Aiyengar Assignee: Mark Miller Priority: Minor Currently checking for leadership (due to the leader's ephemeral node going away) happens in ZK's event thread. If there are many cores and all of them are due leadership, then they would have to serially go through the two-way sync and leadership takeover. For tens of cores, this could mean 30-40s without leadership before the last in the list even gets to start the leadership process. If the leadership process happens in a separate thread, then the cores could all take over in parallel. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6261) Run checkIfIamLeader in a separate thread
[ https://issues.apache.org/jira/browse/SOLR-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069438#comment-14069438 ] Mark Miller commented on SOLR-6261: --- We should look across our process methods and make sure there are not other obvious spots we are holding things up. Run checkIfIamLeader in a separate thread - Key: SOLR-6261 URL: https://issues.apache.org/jira/browse/SOLR-6261 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.9 Reporter: Ramkumar Aiyengar Assignee: Mark Miller Priority: Minor Currently checking for leadership (due to the leader's ephemeral node going away) happens in ZK's event thread. If there are many cores and all of them are due leadership, then they would have to serially go through the two-way sync and leadership takeover. For tens of cores, this could mean 30-40s without leadership before the last in the list even gets to start the leadership process. If the leadership process happens in a separate thread, then the cores could all take over in parallel. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6252) Simplify UnInvertedField#getUnInvertedField synchronization module
[ https://issues.apache.org/jira/browse/SOLR-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069439#comment-14069439 ] ASF subversion and git services commented on SOLR-6252: --- Commit 1612422 from [~markrmil...@gmail.com] in branch 'dev/trunk' [ https://svn.apache.org/r1612422 ] SOLR-6252: A couple of small improvements to UnInvertedField class. Simplify UnInvertedField#getUnInvertedField synchronization module -- Key: SOLR-6252 URL: https://issues.apache.org/jira/browse/SOLR-6252 Project: Solr Issue Type: Improvement Components: search Affects Versions: 5.0 Reporter: Vamsee Yarlagadda Assignee: Mark Miller Priority: Minor Attachments: SOLR-6252-v3.patch, SOLR-6252.patch, SOLR-6252v2.patch Looks like UnInvertedField#getUnInvertedField has implemented a bit additional synchronization module rather than what is required, and thereby increasing the complexity. https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/request/UnInvertedField.java#L667 As pointed out in the above link, as the synchronization is performed on the cache variable(which itself will protect the threads from obtaining access to the cache), we can safely remove all the placeholder flags. As long as cache.get() is in synchronized block, we can simply populate the cache with new entries and other threads will be able to see the changes. This change has been introduced in https://issues.apache.org/jira/browse/SOLR-2548 (Multithreaded faceting) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6252) Simplify UnInvertedField#getUnInvertedField synchronization module
[ https://issues.apache.org/jira/browse/SOLR-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069446#comment-14069446 ] ASF subversion and git services commented on SOLR-6252: --- Commit 1612423 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1612423 ] SOLR-6252: A couple of small improvements to UnInvertedField class. Simplify UnInvertedField#getUnInvertedField synchronization module -- Key: SOLR-6252 URL: https://issues.apache.org/jira/browse/SOLR-6252 Project: Solr Issue Type: Improvement Components: search Affects Versions: 5.0 Reporter: Vamsee Yarlagadda Assignee: Mark Miller Priority: Minor Attachments: SOLR-6252-v3.patch, SOLR-6252.patch, SOLR-6252v2.patch Looks like UnInvertedField#getUnInvertedField has implemented a bit additional synchronization module rather than what is required, and thereby increasing the complexity. https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/request/UnInvertedField.java#L667 As pointed out in the above link, as the synchronization is performed on the cache variable(which itself will protect the threads from obtaining access to the cache), we can safely remove all the placeholder flags. As long as cache.get() is in synchronized block, we can simply populate the cache with new entries and other threads will be able to see the changes. This change has been introduced in https://issues.apache.org/jira/browse/SOLR-2548 (Multithreaded faceting) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5840) hunspell zero-affix handling doesn't work with continuation classes
Robert Muir created LUCENE-5840:
---
Summary: hunspell zero-affix handling doesn't work with
continuation classes
Key: LUCENE-5840
URL: https://issues.apache.org/jira/browse/LUCENE-5840
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: LUCENE-5840.patch
We fixed handling for this in LUCENE-5817, for the simple case:
{noformat}
SFX 322 eer 0 .
{noformat}
But its broken in the case the zero affix has continuation flags:
{noformat}
SFX 322 eer 0/100 .
{noformat}
Because we look for 0 before we parse those away...
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5840) hunspell zero-affix handling doesn't work with continuation classes
[
https://issues.apache.org/jira/browse/LUCENE-5840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-5840:
Attachment: LUCENE-5840.patch
Simple patch. just move the zero affix handling after the parsing.
hunspell zero-affix handling doesn't work with continuation classes
---
Key: LUCENE-5840
URL: https://issues.apache.org/jira/browse/LUCENE-5840
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: LUCENE-5840.patch
We fixed handling for this in LUCENE-5817, for the simple case:
{noformat}
SFX 322 eer 0 .
{noformat}
But its broken in the case the zero affix has continuation flags:
{noformat}
SFX 322 eer 0/100 .
{noformat}
Because we look for 0 before we parse those away...
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3345) BaseDistributedSearchTestCase should always ignore QTime
[ https://issues.apache.org/jira/browse/SOLR-3345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vamsee Yarlagadda updated SOLR-3345: Attachment: SOLR-3345-SVN.patch BaseDistributedSearchTestCase should always ignore QTime Key: SOLR-3345 URL: https://issues.apache.org/jira/browse/SOLR-3345 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0-ALPHA Reporter: Benson Margulies Assignee: Mark Miller Attachments: SOLR-3345-SVN.patch, SOLR-3345.patch The existing subclasses of BaseDistributedSearchTestCase all skip QTime. I can't see any way in which those numbers will ever match. Why not make this the default, or only, behavior? (This is really a question, in that I will provide a patch if no one tells me that it is a bad idea.) -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
