[jira] [Created] (SOLR-13842) Remove wt=json from Implicit API definition's defaults

2019-10-13 Thread Munendra S N (Jira)
Munendra S N created SOLR-13842:
---

 Summary: Remove wt=json from Implicit API definition's defaults
 Key: SOLR-13842
 URL: https://issues.apache.org/jira/browse/SOLR-13842
 Project: Solr
  Issue Type: Task
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Munendra S N


>From solr 7, {{json}} is the default response writer. So, {{wt=json}} can be 
>removed from implicit API definitions



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13827) Fail on Unknown operation in Request Parameters API

2019-10-13 Thread Munendra S N (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950715#comment-16950715
 ] 

Munendra S N commented on SOLR-13827:
-

I have raised the PR. 

> Fail on Unknown operation in Request Parameters API
> ---
>
> Key: SOLR-13827
> URL: https://issues.apache.org/jira/browse/SOLR-13827
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: config-api
>Reporter: Munendra S N
>Assignee: Munendra S N
>Priority: Minor
> Attachments: SOLR-13827.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Request Parameters API supports set, update and delete operations. For any 
> other operation, The API should fail and return error.
> Currently, for unknown operation API returns 200 status
> The config/overlay API fails on unknown operations



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] munendrasn opened a new pull request #949: SOLR-13827: fail on unknown operation in request params API

2019-10-13 Thread GitBox
munendrasn opened a new pull request #949: SOLR-13827: fail on unknown 
operation in request params API
URL: https://github.com/apache/lucene-solr/pull/949
 
 
   
   
   
   # Description
   
   Please provide a short description of the changes you're making with this 
pull request.
   
   # Solution
   
   Please provide a short description of the approach taken to implement your 
solution.
   
   # Tests
   
   Please describe the tests you've developed or run to confirm this patch 
implements the feature or solves the problem.
   
   # Checklist
   
   Please review the following and check all that apply:
   
   - [ ] I have reviewed the guidelines for [How to 
Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms 
to the standards described there to the best of my ability.
   - [ ] I have created a Jira issue and added the issue ID to my pull request 
title.
   - [ ] I am authorized to contribute this code to the ASF and have removed 
any code I do not have a license to distribute.
   - [ ] I have given Solr maintainers 
[access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork)
 to contribute to my PR branch. (optional but recommended)
   - [ ] I have developed this patch against the `master` branch.
   - [ ] I have run `ant precommit` and the appropriate test suite.
   - [ ] I have added tests for my changes.
   - [ ] I have added documentation for the [Ref 
Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) 
(for Solr changes only).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Assigned] (SOLR-13403) Terms component fails for DatePointField

2019-10-13 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N reassigned SOLR-13403:
---

Assignee: Munendra S N

> Terms component fails for DatePointField
> 
>
> Key: SOLR-13403
> URL: https://issues.apache.org/jira/browse/SOLR-13403
> Project: Solr
>  Issue Type: Bug
>  Components: SearchComponents - other
>Reporter: Munendra S N
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-13403.patch, SOLR-13403.patch, SOLR-13403.patch
>
>
> Getting terms for PointFields except DatePointField. For DatePointField, the 
> request fails NPE



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13403) Terms component fails for DatePointField

2019-10-13 Thread Munendra S N (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950710#comment-16950710
 ] 

Munendra S N commented on SOLR-13403:
-

 [^SOLR-13403.patch] 
Slightly refactored one, where object to string is handled similarly, how it is 
handled in other place in codebase

> Terms component fails for DatePointField
> 
>
> Key: SOLR-13403
> URL: https://issues.apache.org/jira/browse/SOLR-13403
> Project: Solr
>  Issue Type: Bug
>  Components: SearchComponents - other
>Reporter: Munendra S N
>Priority: Major
> Attachments: SOLR-13403.patch, SOLR-13403.patch, SOLR-13403.patch
>
>
> Getting terms for PointFields except DatePointField. For DatePointField, the 
> request fails NPE



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-13403) Terms component fails for DatePointField

2019-10-13 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-13403:

Attachment: SOLR-13403.patch

> Terms component fails for DatePointField
> 
>
> Key: SOLR-13403
> URL: https://issues.apache.org/jira/browse/SOLR-13403
> Project: Solr
>  Issue Type: Bug
>  Components: SearchComponents - other
>Reporter: Munendra S N
>Priority: Major
> Attachments: SOLR-13403.patch, SOLR-13403.patch, SOLR-13403.patch
>
>
> Getting terms for PointFields except DatePointField. For DatePointField, the 
> request fails NPE



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13824) JSON Request API ignores prematurely closing curly brace.

2019-10-13 Thread Munendra S N (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950709#comment-16950709
 ] 

Munendra S N commented on SOLR-13824:
-

Hey [~mkhl]
Few other observations
* In some cases, Utils.fromJSON is used which calls {{getVal()}} directly.
* 
[CommandOperation|https://github.com/apache/lucene-solr/blob/b6ea7d60b7005d29400161eae291e51697a11b05/solr/solrj/src/java/org/apache/solr/common/util/CommandOperation.java#L268]
 also calls {{getVal()}} directly. I was able to reproduce the issue with here 
too

So, I was thinking probably a fix might need changes in {{getVal()}}
Something like
{code:java}
public Object getVal(boolean validateEOF) {
Object val = _getVal();
// validate for EOF
return val;
}
public Object getVal() {
return getVal(true);
}
{code}
Current, {{getVal}} becomes {{_getVal}}
So, by default we validate EOF and commandOperation can disable this and handle 
separately

Current patch LGTM as it fixes in JSON request API. It would be great if we 
could fix all such issues irrespective of the API.
If we are going ahead with the current patch then few suggestions
* can we please {{expectThrows}} to validate exception thrown
* {{Throws \{@link IllegalArgumentException} otherwise}}, I think this should 
have been {{Throws \{@link ParseException} otherwise}}

Also, I was able to reproduce this issue for extra }. Is there any other 
scenario where we could enter same issue. If so, patch would still work but we 
might need to change the error message

> JSON Request API ignores prematurely closing curly brace. 
> --
>
> Key: SOLR-13824
> URL: https://issues.apache.org/jira/browse/SOLR-13824
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: JSON Request API
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-13824.patch
>
>
> {code:java}
> json={query:"content:foo", facet:{zz:{field:id}}}
> {code}
> this works fine, but if we mistype {{}}} instead of {{,}}
> {code:java}
> json={query:"content:foo"} facet:{zz:{field:id}}}
> {code}
> It's captured only partially, here's we have under debug
> {code:java}
>   "json":{"query":"content:foo"},
> {code}
> I suppose it should throw an error with 400 code.  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] noblepaul opened a new pull request #948: SOLR-13841: Add jackson databind annotations to SolrJ classpath

2019-10-13 Thread GitBox
noblepaul opened a new pull request #948: SOLR-13841: Add jackson databind 
annotations to SolrJ classpath
URL: https://github.com/apache/lucene-solr/pull/948
 
 
   
   
   
   # Description
   
   Please provide a short description of the changes you're making with this 
pull request.
   
   # Solution
   
   Please provide a short description of the approach taken to implement your 
solution.
   
   # Tests
   
   Please describe the tests you've developed or run to confirm this patch 
implements the feature or solves the problem.
   
   # Checklist
   
   Please review the following and check all that apply:
   
   - [ ] I have reviewed the guidelines for [How to 
Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms 
to the standards described there to the best of my ability.
   - [ ] I have created a Jira issue and added the issue ID to my pull request 
title.
   - [ ] I am authorized to contribute this code to the ASF and have removed 
any code I do not have a license to distribute.
   - [ ] I have given Solr maintainers 
[access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork)
 to contribute to my PR branch. (optional but recommended)
   - [ ] I have developed this patch against the `master` branch.
   - [ ] I have run `ant precommit` and the appropriate test suite.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13841) Add jackson databind annotations to SolrJ classpath

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950660#comment-16950660
 ] 

ASF subversion and git services commented on SOLR-13841:


Commit 89d86730cc0f28d121b63e7bf7d35566f0cfb1b6 in lucene-solr's branch 
refs/heads/jira/SOLR-13841 from Noble Paul
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=89d8673 ]

SOLR-13841: Add jackson databind annotations to SolrJ classpath


> Add jackson databind annotations to SolrJ classpath
> ---
>
> Key: SOLR-13841
> URL: https://issues.apache.org/jira/browse/SOLR-13841
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>
> We can start using annotations in SolrJ to minimize the amount of code we 
> write & improve readability. Jackson is a widely used library and everyone is 
> already familiar with it



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13105) A visual guide to Solr Math Expressions and Streaming Expressions

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950652#comment-16950652
 ] 

ASF subversion and git services commented on SOLR-13105:


Commit fcf5421494cbc4153d8300654eb85a9844e8d7e7 in lucene-solr's branch 
refs/heads/SOLR-13105-visual from Joel Bernstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=fcf5421 ]

SOLR-13105: Proof machine learning docs 1


> A visual guide to Solr Math Expressions and Streaming Expressions
> -
>
> Key: SOLR-13105
> URL: https://issues.apache.org/jira/browse/SOLR-13105
> Project: Solr
>  Issue Type: New Feature
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: Screen Shot 2019-01-14 at 10.56.32 AM.png, Screen Shot 
> 2019-02-21 at 2.14.43 PM.png, Screen Shot 2019-03-03 at 2.28.35 PM.png, 
> Screen Shot 2019-03-04 at 7.47.57 PM.png, Screen Shot 2019-03-13 at 10.47.47 
> AM.png, Screen Shot 2019-03-30 at 6.17.04 PM.png
>
>
> Visualization is now a fundamental element of Solr Streaming Expressions and 
> Math Expressions. This ticket will create a visual guide to Solr Math 
> Expressions and Solr Streaming Expressions that includes *Apache Zeppelin* 
> visualization examples.
> It will also cover using the JDBC expression to *analyze* and *visualize* 
> results from any JDBC compliant data source.
> Intro from the guide:
> {code:java}
> Streaming Expressions exposes the capabilities of Solr Cloud as composable 
> functions. These functions provide a system for searching, transforming, 
> analyzing and visualizing data stored in Solr Cloud collections.
> At a high level there are four main capabilities that will be explored in the 
> documentation:
> * Searching, sampling and aggregating results from Solr.
> * Transforming result sets after they are retrieved from Solr.
> * Analyzing and modeling result sets using probability and statistics and 
> machine learning libraries.
> * Visualizing result sets, aggregations and statistical models of the data.
> {code}
>  
> A few sample visualizations are attached to the ticket.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13105) A visual guide to Solr Math Expressions and Streaming Expressions

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950649#comment-16950649
 ] 

ASF subversion and git services commented on SOLR-13105:


Commit ca2e9c5f11a12f74ff97cae3b3d0c8dbff2ee4f9 in lucene-solr's branch 
refs/heads/SOLR-13105-visual from Joel Bernstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ca2e9c5 ]

SOLR-13105: Proof machine learning docs


> A visual guide to Solr Math Expressions and Streaming Expressions
> -
>
> Key: SOLR-13105
> URL: https://issues.apache.org/jira/browse/SOLR-13105
> Project: Solr
>  Issue Type: New Feature
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: Screen Shot 2019-01-14 at 10.56.32 AM.png, Screen Shot 
> 2019-02-21 at 2.14.43 PM.png, Screen Shot 2019-03-03 at 2.28.35 PM.png, 
> Screen Shot 2019-03-04 at 7.47.57 PM.png, Screen Shot 2019-03-13 at 10.47.47 
> AM.png, Screen Shot 2019-03-30 at 6.17.04 PM.png
>
>
> Visualization is now a fundamental element of Solr Streaming Expressions and 
> Math Expressions. This ticket will create a visual guide to Solr Math 
> Expressions and Solr Streaming Expressions that includes *Apache Zeppelin* 
> visualization examples.
> It will also cover using the JDBC expression to *analyze* and *visualize* 
> results from any JDBC compliant data source.
> Intro from the guide:
> {code:java}
> Streaming Expressions exposes the capabilities of Solr Cloud as composable 
> functions. These functions provide a system for searching, transforming, 
> analyzing and visualizing data stored in Solr Cloud collections.
> At a high level there are four main capabilities that will be explored in the 
> documentation:
> * Searching, sampling and aggregating results from Solr.
> * Transforming result sets after they are retrieved from Solr.
> * Analyzing and modeling result sets using probability and statistics and 
> machine learning libraries.
> * Visualizing result sets, aggregations and statistical models of the data.
> {code}
>  
> A few sample visualizations are attached to the ticket.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13823) ClassCastException when using group.query and return score

2019-10-13 Thread Lucene/Solr QA (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950641#comment-16950641
 ] 

Lucene/Solr QA commented on SOLR-13823:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  4m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  4m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  4m 22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} | 
{color:green}  4m 22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} | 
{color:green}  4m 22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m  1s{color} 
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}100m 29s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.api.collections.ShardSplitTest |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13823 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12982877/SOLR-13823.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP 
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / 1d43bda |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
| unit | 
https://builds.apache.org/job/PreCommit-SOLR-Build/574/artifact/out/patch-unit-solr_core.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-SOLR-Build/574/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output | 
https://builds.apache.org/job/PreCommit-SOLR-Build/574/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> ClassCastException when using group.query and return score
> --
>
> Key: SOLR-13823
> URL: https://issues.apache.org/jira/browse/SOLR-13823
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Affects Versions: 8.2, 8.1.1
>Reporter: Uwe Jäger
>Assignee: Munendra S N
>Priority: Major
>  Labels: Grouping
> Attachments: SOLR-13823.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When grouping the results of a query with group.query there is a 
> ClassCastException in org.apache.solr.search.Grouping.CommandQuery.finish 
> (line 890) since the collector is wrapped in a MultiCollector. 
> The wanted topCollector is available in the inner class so it can be used 
> directly and the cast is not necessary at all. After that change there are 
> still the scores missing in the result, so populating the scores is 
> necessary, too.
> I will create a PR showing the error and also containing a fix.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13105) A visual guide to Solr Math Expressions and Streaming Expressions

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950640#comment-16950640
 ] 

ASF subversion and git services commented on SOLR-13105:


Commit 094d47967c478e98080d01b0635c5675729b5c85 in lucene-solr's branch 
refs/heads/SOLR-13105-visual from Joel Bernstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=094d479 ]

SOLR-13105: Improve math start 1


> A visual guide to Solr Math Expressions and Streaming Expressions
> -
>
> Key: SOLR-13105
> URL: https://issues.apache.org/jira/browse/SOLR-13105
> Project: Solr
>  Issue Type: New Feature
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: Screen Shot 2019-01-14 at 10.56.32 AM.png, Screen Shot 
> 2019-02-21 at 2.14.43 PM.png, Screen Shot 2019-03-03 at 2.28.35 PM.png, 
> Screen Shot 2019-03-04 at 7.47.57 PM.png, Screen Shot 2019-03-13 at 10.47.47 
> AM.png, Screen Shot 2019-03-30 at 6.17.04 PM.png
>
>
> Visualization is now a fundamental element of Solr Streaming Expressions and 
> Math Expressions. This ticket will create a visual guide to Solr Math 
> Expressions and Solr Streaming Expressions that includes *Apache Zeppelin* 
> visualization examples.
> It will also cover using the JDBC expression to *analyze* and *visualize* 
> results from any JDBC compliant data source.
> Intro from the guide:
> {code:java}
> Streaming Expressions exposes the capabilities of Solr Cloud as composable 
> functions. These functions provide a system for searching, transforming, 
> analyzing and visualizing data stored in Solr Cloud collections.
> At a high level there are four main capabilities that will be explored in the 
> documentation:
> * Searching, sampling and aggregating results from Solr.
> * Transforming result sets after they are retrieved from Solr.
> * Analyzing and modeling result sets using probability and statistics and 
> machine learning libraries.
> * Visualizing result sets, aggregations and statistical models of the data.
> {code}
>  
> A few sample visualizations are attached to the ticket.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13105) A visual guide to Solr Math Expressions and Streaming Expressions

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950639#comment-16950639
 ] 

ASF subversion and git services commented on SOLR-13105:


Commit 3f864673c2c406548c7a8c3c8e16625731084a86 in lucene-solr's branch 
refs/heads/SOLR-13105-visual from Joel Bernstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3f86467 ]

SOLR-13105: Improve math start


> A visual guide to Solr Math Expressions and Streaming Expressions
> -
>
> Key: SOLR-13105
> URL: https://issues.apache.org/jira/browse/SOLR-13105
> Project: Solr
>  Issue Type: New Feature
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: Screen Shot 2019-01-14 at 10.56.32 AM.png, Screen Shot 
> 2019-02-21 at 2.14.43 PM.png, Screen Shot 2019-03-03 at 2.28.35 PM.png, 
> Screen Shot 2019-03-04 at 7.47.57 PM.png, Screen Shot 2019-03-13 at 10.47.47 
> AM.png, Screen Shot 2019-03-30 at 6.17.04 PM.png
>
>
> Visualization is now a fundamental element of Solr Streaming Expressions and 
> Math Expressions. This ticket will create a visual guide to Solr Math 
> Expressions and Solr Streaming Expressions that includes *Apache Zeppelin* 
> visualization examples.
> It will also cover using the JDBC expression to *analyze* and *visualize* 
> results from any JDBC compliant data source.
> Intro from the guide:
> {code:java}
> Streaming Expressions exposes the capabilities of Solr Cloud as composable 
> functions. These functions provide a system for searching, transforming, 
> analyzing and visualizing data stored in Solr Cloud collections.
> At a high level there are four main capabilities that will be explored in the 
> documentation:
> * Searching, sampling and aggregating results from Solr.
> * Transforming result sets after they are retrieved from Solr.
> * Analyzing and modeling result sets using probability and statistics and 
> machine learning libraries.
> * Visualizing result sets, aggregations and statistical models of the data.
> {code}
>  
> A few sample visualizations are attached to the ticket.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Assigned] (SOLR-13841) Add jackson databind annotations to SolrJ classpath

2019-10-13 Thread Noble Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul reassigned SOLR-13841:
-

Assignee: Noble Paul

> Add jackson databind annotations to SolrJ classpath
> ---
>
> Key: SOLR-13841
> URL: https://issues.apache.org/jira/browse/SOLR-13841
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>
> We can start using annotations in SolrJ to minimize the amount of code we 
> write & improve readability. Jackson is a widely used library and everyone is 
> already familiar with it



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Created] (SOLR-13841) Add jackson databind annotations to SolrJ classpath

2019-10-13 Thread Noble Paul (Jira)
Noble Paul created SOLR-13841:
-

 Summary: Add jackson databind annotations to SolrJ classpath
 Key: SOLR-13841
 URL: https://issues.apache.org/jira/browse/SOLR-13841
 Project: Solr
  Issue Type: Task
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Noble Paul


We can start using annotations in SolrJ to minimize the amount of code we write 
& improve readability. Jackson is a widely used library and everyone is already 
familiar with it



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13731) javabin must support a 1:1 mapping of the JSON update format

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950637#comment-16950637
 ] 

ASF subversion and git services commented on SOLR-13731:


Commit 41779e5f605ff42e174794827f500ab80597f80d in lucene-solr's branch 
refs/heads/branch_8x from Noble Paul
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=41779e5 ]

SOLR-13731: javabin must support a 1:1 mapping of the JSON update format 



> javabin  must support a 1:1 mapping of the JSON update format
> -
>
> Key: SOLR-13731
> URL: https://issues.apache.org/jira/browse/SOLR-13731
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Objects like SolrInputDocument is serialized in such a way that the size is 
> known in advance. All objects should ideally support streaming friendly types.
> This is backward compatible . basically javabin will continue to serialize 
> using the old format , but will accept more efficient formats as input



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Issue Comment Deleted] (SOLR-13731) javabin must support a 1:1 mapping of the JSON update format

2019-10-13 Thread Noble Paul (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-13731:
--
Comment: was deleted

(was: Commit 76dcbf78c94272d469e78153915f82bb70a8a4df in lucene-solr's branch 
refs/heads/jira/SOLR-13731 from Noble Paul
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=76dcbf7 ]

Merge branch 'master' into jira/SOLR-13731
)

> javabin  must support a 1:1 mapping of the JSON update format
> -
>
> Key: SOLR-13731
> URL: https://issues.apache.org/jira/browse/SOLR-13731
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Objects like SolrInputDocument is serialized in such a way that the size is 
> known in advance. All objects should ideally support streaming friendly types.
> This is backward compatible . basically javabin will continue to serialize 
> using the old format , but will accept more efficient formats as input



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13731) javabin must support a 1:1 mapping of the JSON update format

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950636#comment-16950636
 ] 

ASF subversion and git services commented on SOLR-13731:


Commit b6ea7d60b7005d29400161eae291e51697a11b05 in lucene-solr's branch 
refs/heads/master from Noble Paul
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b6ea7d6 ]

SOLR-13731: javabin must support a 1:1 mapping of the JSON update format 



> javabin  must support a 1:1 mapping of the JSON update format
> -
>
> Key: SOLR-13731
> URL: https://issues.apache.org/jira/browse/SOLR-13731
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Objects like SolrInputDocument is serialized in such a way that the size is 
> known in advance. All objects should ideally support streaming friendly types.
> This is backward compatible . basically javabin will continue to serialize 
> using the old format , but will accept more efficient formats as input



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] noblepaul merged pull request #859: SOLR-13731: javabin must support a 1:1 mapping of the JSON update format

2019-10-13 Thread GitBox
noblepaul merged pull request #859: SOLR-13731: javabin must support a 1:1 
mapping of the JSON update format
URL: https://github.com/apache/lucene-solr/pull/859
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13731) javabin must support a 1:1 mapping of the JSON update format

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950635#comment-16950635
 ] 

ASF subversion and git services commented on SOLR-13731:


Commit 76dcbf78c94272d469e78153915f82bb70a8a4df in lucene-solr's branch 
refs/heads/jira/SOLR-13731 from Noble Paul
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=76dcbf7 ]

Merge branch 'master' into jira/SOLR-13731


> javabin  must support a 1:1 mapping of the JSON update format
> -
>
> Key: SOLR-13731
> URL: https://issues.apache.org/jira/browse/SOLR-13731
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Objects like SolrInputDocument is serialized in such a way that the size is 
> known in advance. All objects should ideally support streaming friendly types.
> This is backward compatible . basically javabin will continue to serialize 
> using the old format , but will accept more efficient formats as input



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13787) An annotation based system to write v2 only APIs

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950627#comment-16950627
 ] 

ASF subversion and git services commented on SOLR-13787:


Commit 5b6561eadb522150c8ea2954d60077ac445ad1d7 in lucene-solr's branch 
refs/heads/jira/SOLR-13731 from Noble Paul
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5b6561e ]

SOLR-13787: Support for Payload as 3rd param


> An annotation based system to write v2 only APIs
> 
>
> Key: SOLR-13787
> URL: https://issues.apache.org/jira/browse/SOLR-13787
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Fix For: master (9.0), 8.3
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> example v2 API may look as follows
> {code:java}
> @V2EndPoint(method = POST, path = "/cluster/package", permission = 
> PermissionNameProvider.Name.ALL)
> public static class ApiTest {
>   @Command(name = "add")
>   public void add(SolrQueryRequest req, SolrQueryResponse rsp, AddVersion 
> addVersion) {
>   }
>   @Command(name = "delete")
>   public void del(SolrQueryRequest req, SolrQueryResponse rsp, List 
> names) {
>   }
> }
> public static class AddVersion {
>   @JsonProperty(value = "package", required = true)
>   public String pkg;
>   @JsonProperty(value = "version", required = true)
>   public String version;
>   @JsonProperty(value = "files", required = true)
>   public List files;
> }
> {code}
> This expects you to already hava a POJO annotated with jackson annotations
>  
> The annotations are:
>  
> {code:java}
> @Retention(RetentionPolicy.RUNTIME)
> @Target({ElementType.TYPE})
> public @interface EndPoint {
> /**The suoported http methods*/
>   SolrRequest.METHOD[] method();
> /**supported paths*/
>   String[] path();
>   PermissionNameProvider.Name permission();
> }
> {code}
> {code:java}
> @Retention(RetentionPolicy.RUNTIME)
> @Target(ElementType.METHOD)
> public @interface Command {
>/**if this is not a json command , leave it empty.
>* Keep in mind that you cannot have duplicates.
>* Only one method per name
>*
>*/
>   String name() default "";
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13815) Live split can lose data

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950630#comment-16950630
 ] 

ASF subversion and git services commented on SOLR-13815:


Commit a057b0d159f669d28565f48c3ee2bee76ab3d821 in lucene-solr's branch 
refs/heads/jira/SOLR-13731 from Yonik Seeley
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a057b0d ]

SOLR-13815: fix live split data loss due to cluster state change between 
checking current shard state and getting list of subShards (#920)

* SOLR-13815: add simple live split test to help debugging possible issue

* SOLR-13815: fix live split data loss due to cluster state change berween 
checking current shard state and getting list of subShards


> Live split can lose data
> 
>
> Key: SOLR-13815
> URL: https://issues.apache.org/jira/browse/SOLR-13815
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
>Priority: Major
> Fix For: 8.3
>
> Attachments: fail.191004_053129, fail.191004_093307
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> This issue is to investigate potential data loss during a "live" split (i.e. 
> split happens while updates are flowing)
> This was discovered during the shared storage work which was based on a 
> non-release branch_8x sometime before 8.3, hence the first steps are to try 
> and reproduce on the master branch without any shared storage changes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13787) An annotation based system to write v2 only APIs

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950626#comment-16950626
 ] 

ASF subversion and git services commented on SOLR-13787:


Commit 2d32f0b5a6567cdea72181ebaa4ddd2af9421093 in lucene-solr's branch 
refs/heads/jira/SOLR-13731 from Noble Paul
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=2d32f0b ]

SOLR-13787: Added support for PayLoad as 3rd param


> An annotation based system to write v2 only APIs
> 
>
> Key: SOLR-13787
> URL: https://issues.apache.org/jira/browse/SOLR-13787
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Fix For: master (9.0), 8.3
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> example v2 API may look as follows
> {code:java}
> @V2EndPoint(method = POST, path = "/cluster/package", permission = 
> PermissionNameProvider.Name.ALL)
> public static class ApiTest {
>   @Command(name = "add")
>   public void add(SolrQueryRequest req, SolrQueryResponse rsp, AddVersion 
> addVersion) {
>   }
>   @Command(name = "delete")
>   public void del(SolrQueryRequest req, SolrQueryResponse rsp, List 
> names) {
>   }
> }
> public static class AddVersion {
>   @JsonProperty(value = "package", required = true)
>   public String pkg;
>   @JsonProperty(value = "version", required = true)
>   public String version;
>   @JsonProperty(value = "files", required = true)
>   public List files;
> }
> {code}
> This expects you to already hava a POJO annotated with jackson annotations
>  
> The annotations are:
>  
> {code:java}
> @Retention(RetentionPolicy.RUNTIME)
> @Target({ElementType.TYPE})
> public @interface EndPoint {
> /**The suoported http methods*/
>   SolrRequest.METHOD[] method();
> /**supported paths*/
>   String[] path();
>   PermissionNameProvider.Name permission();
> }
> {code}
> {code:java}
> @Retention(RetentionPolicy.RUNTIME)
> @Target(ElementType.METHOD)
> public @interface Command {
>/**if this is not a json command , leave it empty.
>* Keep in mind that you cannot have duplicates.
>* Only one method per name
>*
>*/
>   String name() default "";
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13787) An annotation based system to write v2 only APIs

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950632#comment-16950632
 ] 

ASF subversion and git services commented on SOLR-13787:


Commit 84126ea0eae452ff3cebbd5eb2b7d94573eb841e in lucene-solr's branch 
refs/heads/jira/SOLR-13731 from Noble Paul
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=84126ea ]

SOLR-13787: Better error logging


> An annotation based system to write v2 only APIs
> 
>
> Key: SOLR-13787
> URL: https://issues.apache.org/jira/browse/SOLR-13787
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Fix For: master (9.0), 8.3
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> example v2 API may look as follows
> {code:java}
> @V2EndPoint(method = POST, path = "/cluster/package", permission = 
> PermissionNameProvider.Name.ALL)
> public static class ApiTest {
>   @Command(name = "add")
>   public void add(SolrQueryRequest req, SolrQueryResponse rsp, AddVersion 
> addVersion) {
>   }
>   @Command(name = "delete")
>   public void del(SolrQueryRequest req, SolrQueryResponse rsp, List 
> names) {
>   }
> }
> public static class AddVersion {
>   @JsonProperty(value = "package", required = true)
>   public String pkg;
>   @JsonProperty(value = "version", required = true)
>   public String version;
>   @JsonProperty(value = "files", required = true)
>   public List files;
> }
> {code}
> This expects you to already hava a POJO annotated with jackson annotations
>  
> The annotations are:
>  
> {code:java}
> @Retention(RetentionPolicy.RUNTIME)
> @Target({ElementType.TYPE})
> public @interface EndPoint {
> /**The suoported http methods*/
>   SolrRequest.METHOD[] method();
> /**supported paths*/
>   String[] path();
>   PermissionNameProvider.Name permission();
> }
> {code}
> {code:java}
> @Retention(RetentionPolicy.RUNTIME)
> @Target(ElementType.METHOD)
> public @interface Command {
>/**if this is not a json command , leave it empty.
>* Keep in mind that you cannot have duplicates.
>* Only one method per name
>*
>*/
>   String name() default "";
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13815) Live split can lose data

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950631#comment-16950631
 ] 

ASF subversion and git services commented on SOLR-13815:


Commit a057b0d159f669d28565f48c3ee2bee76ab3d821 in lucene-solr's branch 
refs/heads/jira/SOLR-13731 from Yonik Seeley
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a057b0d ]

SOLR-13815: fix live split data loss due to cluster state change between 
checking current shard state and getting list of subShards (#920)

* SOLR-13815: add simple live split test to help debugging possible issue

* SOLR-13815: fix live split data loss due to cluster state change berween 
checking current shard state and getting list of subShards


> Live split can lose data
> 
>
> Key: SOLR-13815
> URL: https://issues.apache.org/jira/browse/SOLR-13815
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
>Priority: Major
> Fix For: 8.3
>
> Attachments: fail.191004_053129, fail.191004_093307
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> This issue is to investigate potential data loss during a "live" split (i.e. 
> split happens while updates are flowing)
> This was discovered during the shared storage work which was based on a 
> non-release branch_8x sometime before 8.3, hence the first steps are to try 
> and reproduce on the master branch without any shared storage changes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13821) Package Store

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950633#comment-16950633
 ] 

ASF subversion and git services commented on SOLR-13821:


Commit 88f457ee2a517f8c9656209846c16d402edfef0e in lucene-solr's branch 
refs/heads/jira/SOLR-13731 from Noble Paul
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=88f457e ]

SOLR-13821: refactored the code to change the API to suit package loader


> Package Store
> -
>
> Key: SOLR-13821
> URL: https://issues.apache.org/jira/browse/SOLR-13821
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
>Assignee: Noble Paul
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> Package store is a storage managed by Solr that holds the package artifacts. 
> This is replicated across nodes.
> Design is here: 
> [https://docs.google.com/document/d/15b3m3i3NFDKbhkhX_BN0MgvPGZaBj34TKNF2-UNC3U8/edit?ts=5d86a8ad#]
> The package store is powered by an underlying filestore. This filestore is a 
> fully replicated p2p filesystem storage for artifacts.
> The APIs are as follows
> {code:java}
> # add a file
> POST  /api/cluster/files/path/to/file.jar
> #retrieve a file
> GET /api/cluster/files/path/to/file.jar
> #list files in the /path/to directory
> GET /api/cluster/files/path/to
> #GET meta info of the jar
> GET /api/cluster/files/path/to/file.jar?meta=true
> {code}
> This store keeps 2 files per file
>  # The actual file say {{myplugin.jar}}
>  # A metadata file {{.myplugin.jar.json}} in the same directory
> The contenbts of the metadata file is
> {code:json}
> {
> "sha512" : ""
> "sig": {
> "" :""
> }}
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13815) Live split can lose data

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950629#comment-16950629
 ] 

ASF subversion and git services commented on SOLR-13815:


Commit a057b0d159f669d28565f48c3ee2bee76ab3d821 in lucene-solr's branch 
refs/heads/jira/SOLR-13731 from Yonik Seeley
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a057b0d ]

SOLR-13815: fix live split data loss due to cluster state change between 
checking current shard state and getting list of subShards (#920)

* SOLR-13815: add simple live split test to help debugging possible issue

* SOLR-13815: fix live split data loss due to cluster state change berween 
checking current shard state and getting list of subShards


> Live split can lose data
> 
>
> Key: SOLR-13815
> URL: https://issues.apache.org/jira/browse/SOLR-13815
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
>Priority: Major
> Fix For: 8.3
>
> Attachments: fail.191004_053129, fail.191004_093307
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> This issue is to investigate potential data loss during a "live" split (i.e. 
> split happens while updates are flowing)
> This was discovered during the shared storage work which was based on a 
> non-release branch_8x sometime before 8.3, hence the first steps are to try 
> and reproduce on the master branch without any shared storage changes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13827) Fail on Unknown operation in Request Parameters API

2019-10-13 Thread Noble Paul (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950623#comment-16950623
 ] 

Noble Paul commented on SOLR-13827:
---

Can you raise a PR? . It make sit easy to review & feed back

bq. So, should we remove wt=json from defaults of the implicit API?

Yes

> Fail on Unknown operation in Request Parameters API
> ---
>
> Key: SOLR-13827
> URL: https://issues.apache.org/jira/browse/SOLR-13827
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: config-api
>Reporter: Munendra S N
>Assignee: Munendra S N
>Priority: Minor
> Attachments: SOLR-13827.patch
>
>
> Request Parameters API supports set, update and delete operations. For any 
> other operation, The API should fail and return error.
> Currently, for unknown operation API returns 200 status
> The config/overlay API fails on unknown operations



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Comment Edited] (LUCENE-9004) Approximate nearest vector search

2019-10-13 Thread Tomoko Uchida (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950620#comment-16950620
 ] 

Tomoko Uchida edited comment on LUCENE-9004 at 10/13/19 11:46 PM:
--

I also really look forward to get this feature into Lucene!

FWIW, it seems like that there are several versions of the HNSW paper and I 
think this is the latest revision (v4). Pseudo algorithm parts have been 
refined or evolved from the original version ([~sokolov] introduced here) 
though I've not yet closely checked the diffs and have no idea about they have 
significant impacts here.

[https://arxiv.org/abs/1603.09320]

(I will check out / play with the PoC branch and share my findings, if it would 
be useful.)


was (Author: tomoko uchida):
I also really look forward to get this feature into Lucene!

FWIW, it seems like that there are several versions of the HSNW paper and I 
think this is the latest revision (v4). Pseudo algorithm parts have been 
refined or evolved from the original version ([~sokolov] introduced here) 
though I've not yet closely checked the diffs and have no idea about they have 
significant impacts here.

https://arxiv.org/abs/1603.09320

(I will check out / play with the PoC branch and share my findings, if it would 
be useful.)

> Approximate nearest vector search
> -
>
> Key: LUCENE-9004
> URL: https://issues.apache.org/jira/browse/LUCENE-9004
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Michael Sokolov
>Priority: Major
>
> "Semantic" search based on machine-learned vector "embeddings" representing 
> terms, queries and documents is becoming a must-have feature for a modern 
> search engine. SOLR-12890 is exploring various approaches to this, including 
> providing vector-based scoring functions. This is a spinoff issue from that.
> The idea here is to explore approximate nearest-neighbor search. Researchers 
> have found an approach based on navigating a graph that partially encodes the 
> nearest neighbor relation at multiple scales can provide accuracy > 95% (as 
> compared to exact nearest neighbor calculations) at a reasonable cost. This 
> issue will explore implementing HNSW (hierarchical navigable small-world) 
> graphs for the purpose of approximate nearest vector search (often referred 
> to as KNN or k-nearest-neighbor search).
> At a high level the way this algorithm works is this. First assume you have a 
> graph that has a partial encoding of the nearest neighbor relation, with some 
> short and some long-distance links. If this graph is built in the right way 
> (has the hierarchical navigable small world property), then you can 
> efficiently traverse it to find nearest neighbors (approximately) in log N 
> time where N is the number of nodes in the graph. I believe this idea was 
> pioneered in  [1]. The great insight in that paper is that if you use the 
> graph search algorithm to find the K nearest neighbors of a new document 
> while indexing, and then link those neighbors (undirectedly, ie both ways) to 
> the new document, then the graph that emerges will have the desired 
> properties.
> The implementation I propose for Lucene is as follows. We need two new data 
> structures to encode the vectors and the graph. We can encode vectors using a 
> light wrapper around {{BinaryDocValues}} (we also want to encode the vector 
> dimension and have efficient conversion from bytes to floats). For the graph 
> we can use {{SortedNumericDocValues}} where the values we encode are the 
> docids of the related documents. Encoding the interdocument relations using 
> docids directly will make it relatively fast to traverse the graph since we 
> won't need to lookup through an id-field indirection. This choice limits us 
> to building a graph-per-segment since it would be impractical to maintain a 
> global graph for the whole index in the face of segment merges. However 
> graph-per-segment is a very natural at search time - we can traverse each 
> segments' graph independently and merge results as we do today for term-based 
> search.
> At index time, however, merging graphs is somewhat challenging. While 
> indexing we build a graph incrementally, performing searches to construct 
> links among neighbors. When merging segments we must construct a new graph 
> containing elements of all the merged segments. Ideally we would somehow 
> preserve the work done when building the initial graphs, but at least as a 
> start I'd propose we construct a new graph from scratch when merging. The 
> process is going to be  limited, at least initially, to graphs that can fit 
> in RAM since we require random access to the entire graph while constructing 
> it: In order to add links bidirectionally we must continually update existing 
> documents.
> I think we want 

[jira] [Commented] (LUCENE-9004) Approximate nearest vector search

2019-10-13 Thread Tomoko Uchida (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950620#comment-16950620
 ] 

Tomoko Uchida commented on LUCENE-9004:
---

I also really look forward to get this feature into Lucene!

FWIW, it seems like that there are several versions of the HSNW paper and I 
think this is the latest revision (v4). Pseudo algorithm parts have been 
refined or evolved from the original version ([~sokolov] introduced here) 
though I've not yet closely checked the diffs and have no idea about they have 
significant impacts here.

https://arxiv.org/abs/1603.09320

(I will check out / play with the PoC branch and share my findings, if it would 
be useful.)

> Approximate nearest vector search
> -
>
> Key: LUCENE-9004
> URL: https://issues.apache.org/jira/browse/LUCENE-9004
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Michael Sokolov
>Priority: Major
>
> "Semantic" search based on machine-learned vector "embeddings" representing 
> terms, queries and documents is becoming a must-have feature for a modern 
> search engine. SOLR-12890 is exploring various approaches to this, including 
> providing vector-based scoring functions. This is a spinoff issue from that.
> The idea here is to explore approximate nearest-neighbor search. Researchers 
> have found an approach based on navigating a graph that partially encodes the 
> nearest neighbor relation at multiple scales can provide accuracy > 95% (as 
> compared to exact nearest neighbor calculations) at a reasonable cost. This 
> issue will explore implementing HNSW (hierarchical navigable small-world) 
> graphs for the purpose of approximate nearest vector search (often referred 
> to as KNN or k-nearest-neighbor search).
> At a high level the way this algorithm works is this. First assume you have a 
> graph that has a partial encoding of the nearest neighbor relation, with some 
> short and some long-distance links. If this graph is built in the right way 
> (has the hierarchical navigable small world property), then you can 
> efficiently traverse it to find nearest neighbors (approximately) in log N 
> time where N is the number of nodes in the graph. I believe this idea was 
> pioneered in  [1]. The great insight in that paper is that if you use the 
> graph search algorithm to find the K nearest neighbors of a new document 
> while indexing, and then link those neighbors (undirectedly, ie both ways) to 
> the new document, then the graph that emerges will have the desired 
> properties.
> The implementation I propose for Lucene is as follows. We need two new data 
> structures to encode the vectors and the graph. We can encode vectors using a 
> light wrapper around {{BinaryDocValues}} (we also want to encode the vector 
> dimension and have efficient conversion from bytes to floats). For the graph 
> we can use {{SortedNumericDocValues}} where the values we encode are the 
> docids of the related documents. Encoding the interdocument relations using 
> docids directly will make it relatively fast to traverse the graph since we 
> won't need to lookup through an id-field indirection. This choice limits us 
> to building a graph-per-segment since it would be impractical to maintain a 
> global graph for the whole index in the face of segment merges. However 
> graph-per-segment is a very natural at search time - we can traverse each 
> segments' graph independently and merge results as we do today for term-based 
> search.
> At index time, however, merging graphs is somewhat challenging. While 
> indexing we build a graph incrementally, performing searches to construct 
> links among neighbors. When merging segments we must construct a new graph 
> containing elements of all the merged segments. Ideally we would somehow 
> preserve the work done when building the initial graphs, but at least as a 
> start I'd propose we construct a new graph from scratch when merging. The 
> process is going to be  limited, at least initially, to graphs that can fit 
> in RAM since we require random access to the entire graph while constructing 
> it: In order to add links bidirectionally we must continually update existing 
> documents.
> I think we want to express this API to users as a single joint 
> {{KnnGraphField}} abstraction that joins together the vectors and the graph 
> as a single joint field type. Mostly it just looks like a vector-valued 
> field, but has this graph attached to it.
> I'll push a branch with my POC and would love to hear comments. It has many 
> nocommits, basic design is not really set, there is no Query implementation 
> and no integration iwth IndexSearcher, but it does work by some measure using 
> a standalone test class. I've tested with uniform random vectors and on my 
> laptop indexed 10K documents in around 10 

[jira] [Resolved] (SOLR-13815) Live split can lose data

2019-10-13 Thread Yonik Seeley (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley resolved SOLR-13815.
-
Resolution: Fixed

> Live split can lose data
> 
>
> Key: SOLR-13815
> URL: https://issues.apache.org/jira/browse/SOLR-13815
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
>Priority: Major
> Fix For: 8.3
>
> Attachments: fail.191004_053129, fail.191004_093307
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> This issue is to investigate potential data loss during a "live" split (i.e. 
> split happens while updates are flowing)
> This was discovered during the shared storage work which was based on a 
> non-release branch_8x sometime before 8.3, hence the first steps are to try 
> and reproduce on the master branch without any shared storage changes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Assigned] (SOLR-13815) Live split can lose data

2019-10-13 Thread Yonik Seeley (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley reassigned SOLR-13815:
---

Assignee: Yonik Seeley

> Live split can lose data
> 
>
> Key: SOLR-13815
> URL: https://issues.apache.org/jira/browse/SOLR-13815
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
>Priority: Major
> Fix For: 8.3
>
> Attachments: fail.191004_053129, fail.191004_093307
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> This issue is to investigate potential data loss during a "live" split (i.e. 
> split happens while updates are flowing)
> This was discovered during the shared storage work which was based on a 
> non-release branch_8x sometime before 8.3, hence the first steps are to try 
> and reproduce on the master branch without any shared storage changes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13815) Live split can lose data

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950478#comment-16950478
 ] 

ASF subversion and git services commented on SOLR-13815:


Commit 10d7df77ca120c22181d6315be94db8185b5e8cc in lucene-solr's branch 
refs/heads/branch_8_3 from Yonik Seeley
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=10d7df7 ]

SOLR-13815: enhance live split test to fail more often


> Live split can lose data
> 
>
> Key: SOLR-13815
> URL: https://issues.apache.org/jira/browse/SOLR-13815
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Priority: Major
> Fix For: 8.3
>
> Attachments: fail.191004_053129, fail.191004_093307
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> This issue is to investigate potential data loss during a "live" split (i.e. 
> split happens while updates are flowing)
> This was discovered during the shared storage work which was based on a 
> non-release branch_8x sometime before 8.3, hence the first steps are to try 
> and reproduce on the master branch without any shared storage changes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13815) Live split can lose data

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950477#comment-16950477
 ] 

ASF subversion and git services commented on SOLR-13815:


Commit 845c3e9775f0352c56e284ed7017848ee75ed070 in lucene-solr's branch 
refs/heads/branch_8x from Yonik Seeley
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=845c3e9 ]

SOLR-13815: enhance live split test to fail more often


> Live split can lose data
> 
>
> Key: SOLR-13815
> URL: https://issues.apache.org/jira/browse/SOLR-13815
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Priority: Major
> Fix For: 8.3
>
> Attachments: fail.191004_053129, fail.191004_093307
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> This issue is to investigate potential data loss during a "live" split (i.e. 
> split happens while updates are flowing)
> This was discovered during the shared storage work which was based on a 
> non-release branch_8x sometime before 8.3, hence the first steps are to try 
> and reproduce on the master branch without any shared storage changes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13815) Live split can lose data

2019-10-13 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950475#comment-16950475
 ] 

ASF subversion and git services commented on SOLR-13815:


Commit 1d43bda2840668e507b808b3f12fc86f0a8f3662 in lucene-solr's branch 
refs/heads/master from Yonik Seeley
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=1d43bda ]

SOLR-13815: enhance live split test to fail more often


> Live split can lose data
> 
>
> Key: SOLR-13815
> URL: https://issues.apache.org/jira/browse/SOLR-13815
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Priority: Major
> Fix For: 8.3
>
> Attachments: fail.191004_053129, fail.191004_093307
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> This issue is to investigate potential data loss during a "live" split (i.e. 
> split happens while updates are flowing)
> This was discovered during the shared storage work which was based on a 
> non-release branch_8x sometime before 8.3, hence the first steps are to try 
> and reproduce on the master branch without any shared storage changes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] yonik merged pull request #947: SOLR-13815: enhance live split test to fail more often

2019-10-13 Thread GitBox
yonik merged pull request #947: SOLR-13815: enhance live split test to fail 
more often
URL: https://github.com/apache/lucene-solr/pull/947
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] yonik opened a new pull request #947: SOLR-13815: enhance live split test to fail more often

2019-10-13 Thread GitBox
yonik opened a new pull request #947: SOLR-13815: enhance live split test to 
fail more often
URL: https://github.com/apache/lucene-solr/pull/947
 
 
   enhancement to the test case to allow more indexing threads to increase the 
chance of test failure.  On my laptop, this caused the test to fail almost 50% 
of the time with 4 threads.  Also changed the failure code to speed up checking 
what docs were missing (avoid a query-per-doc)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13808) Query DSL should let to cache filter

2019-10-13 Thread Mikhail Khludnev (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950468#comment-16950468
 ] 

Mikhail Khludnev commented on SOLR-13808:
-

the proposal above {{filter:{.. cache:true..}}}  doesn't seem feasible because 
[cache:true is 
deafult|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/search/ExtendedQueryBase.java#L23].
 Possible approaches:
 # introduce new local param {{{!bool cache=color:red}} 
 # in 9.x make all filters cached by default, and let only disable it 
deliberately {{filter:{.. cache:false..}}}   

What' your preference? 

> Query DSL should let to cache filter
> 
>
> Key: SOLR-13808
> URL: https://issues.apache.org/jira/browse/SOLR-13808
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mikhail Khludnev
>Priority: Major
>
> Query DSL let to express Lucene BQ's filter
>  
> {code:java}
> { query: {bool: { filter: {term: {f:name,query:"foo bar"}}} }}{code}
> However, it might easily catch the need in caching it in filter cache. This 
> might rely on ExtensibleQuery and QParser: 
> {code:java}
> { query: {bool: { filter: {term: {f:name,query:"foo bar", cache:true}}} }}
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13741) possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest

2019-10-13 Thread Jira


[ 
https://issues.apache.org/jira/browse/SOLR-13741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950466#comment-16950466
 ] 

Jan Høydahl commented on SOLR-13741:


Removed the code changes from from patch again so it only contains test 
improvements.

> possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest
> --
>
> Key: SOLR-13741
> URL: https://issues.apache.org/jira/browse/SOLR-13741
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Chris M. Hostetter
>Assignee: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch, 
> SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch
>
>
> A while back i saw a weird non-reproducible failure from 
> AuditLoggerIntegrationTest.  When i started reading through that code, 2 
> things jumped out at me:
> # the way the 'delay' option works is brittle, and makes assumptions about 
> CPU scheduling that aren't neccessarily going to be true (and also suffers 
> from the problem that Thread.sleep isn't garunteed to sleep as long as you 
> ask it too)
> # the way the existing {{waitForAuditEventCallbacks(number)}} logic works by 
> checking the size of a (List) {{buffer}} of recieved events in a sleep/poll 
> loop, until it contains at least N items -- but the code that adds items to 
> that buffer in the async Callback thread async _before_ the code that updates 
> other state variables (like the global {{count}} and the patch specific 
> {{resourceCounts}}) meaning that a test waiting on 3 events could "see" 3 
> events added to the buffer, but calling {{assertEquals(3, 
> receiver.getTotalCount())}} could subsequently fail because that variable 
> hadn't been udpated yet.
> #2 was the source of the failures I was seeing, and while a quick fix for 
> that specific problem would be to update all other state _before_ adding the 
> event to the buffer, I set out to try and make more general improvements to 
> the test:
> * eliminate the dependency on sleep loops by {{await}}-ing on concurrent data 
> structures
> * harden the assertions made about the expected events recieved (updating 
> some test methods that currently just assert the number of events recieved)
> * add new assertions that _only_ the expected events are recieved.
> In the process of doing this, I've found several oddities/descrepencies 
> between things the test currently claims/asserts, and what *actually* happens 
> under more rigerous scrutiny/assertions.
> I'll attach a patch shortly that has my (in progress) updates and inlcudes 
> copious nocommits about things seem suspect.  the summary of these concerns 
> is:
> * SolrException status codes that do not match what the existing test says 
> they should (but doesn't assert)
> * extra AuditEvents occuring that the existing test does not expect
> * AuditEvents for incorrect credentials that do not at all match the expected 
> AuditEvent in the existing test -- which the current test seems to miss in 
> it's assertions because it's picking up some extra events from triggered by 
> previuos requests earlier in the test that just happen to also match the 
> asserctions.
> ...it's not clear to me if the test logic is correct and these are "code 
> bugs" or if the test is faulty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-13741) possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest

2019-10-13 Thread Jira


 [ 
https://issues.apache.org/jira/browse/SOLR-13741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-13741:
---
Attachment: SOLR-13741.patch

> possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest
> --
>
> Key: SOLR-13741
> URL: https://issues.apache.org/jira/browse/SOLR-13741
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Chris M. Hostetter
>Assignee: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch, 
> SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch
>
>
> A while back i saw a weird non-reproducible failure from 
> AuditLoggerIntegrationTest.  When i started reading through that code, 2 
> things jumped out at me:
> # the way the 'delay' option works is brittle, and makes assumptions about 
> CPU scheduling that aren't neccessarily going to be true (and also suffers 
> from the problem that Thread.sleep isn't garunteed to sleep as long as you 
> ask it too)
> # the way the existing {{waitForAuditEventCallbacks(number)}} logic works by 
> checking the size of a (List) {{buffer}} of recieved events in a sleep/poll 
> loop, until it contains at least N items -- but the code that adds items to 
> that buffer in the async Callback thread async _before_ the code that updates 
> other state variables (like the global {{count}} and the patch specific 
> {{resourceCounts}}) meaning that a test waiting on 3 events could "see" 3 
> events added to the buffer, but calling {{assertEquals(3, 
> receiver.getTotalCount())}} could subsequently fail because that variable 
> hadn't been udpated yet.
> #2 was the source of the failures I was seeing, and while a quick fix for 
> that specific problem would be to update all other state _before_ adding the 
> event to the buffer, I set out to try and make more general improvements to 
> the test:
> * eliminate the dependency on sleep loops by {{await}}-ing on concurrent data 
> structures
> * harden the assertions made about the expected events recieved (updating 
> some test methods that currently just assert the number of events recieved)
> * add new assertions that _only_ the expected events are recieved.
> In the process of doing this, I've found several oddities/descrepencies 
> between things the test currently claims/asserts, and what *actually* happens 
> under more rigerous scrutiny/assertions.
> I'll attach a patch shortly that has my (in progress) updates and inlcudes 
> copious nocommits about things seem suspect.  the summary of these concerns 
> is:
> * SolrException status codes that do not match what the existing test says 
> they should (but doesn't assert)
> * extra AuditEvents occuring that the existing test does not expect
> * AuditEvents for incorrect credentials that do not at all match the expected 
> AuditEvent in the existing test -- which the current test seems to miss in 
> it's assertions because it's picking up some extra events from triggered by 
> previuos requests earlier in the test that just happen to also match the 
> asserctions.
> ...it's not clear to me if the test logic is correct and these are "code 
> bugs" or if the test is faulty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-12490) Query DSL supports for further referring and exclusion in JSON facets

2019-10-13 Thread Mikhail Khludnev (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-12490:

Status: Patch Available  (was: Open)

> Query DSL supports for further referring and exclusion in JSON facets 
> --
>
> Key: SOLR-12490
> URL: https://issues.apache.org/jira/browse/SOLR-12490
> Project: Solr
>  Issue Type: Improvement
>  Components: Facet Module, faceting
>Reporter: Mikhail Khludnev
>Priority: Major
>  Labels: newdev
> Attachments: SOLR-12490.patch
>
>
> It's spin off from the 
> [discussion|https://issues.apache.org/jira/browse/SOLR-9685?focusedCommentId=16508720=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16508720].
>  
> h2. Problem
> # after SOLR-9685 we can tag separate clauses in hairish queries like 
> {{parent}}, {{bool}}
> # we can {{domain.excludeTags}}
> # we are looking for child faceting with exclusions, see SOLR-9510, SOLR-8998 
>
> # but we can refer only separate params in {{domain.filter}}, it's not 
> possible to refer separate clauses
> see the first comment



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Resolved] (SOLR-13764) Parse Interval Query from JSON API

2019-10-13 Thread Mikhail Khludnev (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev resolved SOLR-13764.
-
Resolution: Won't Fix

> Parse Interval Query from JSON API
> --
>
> Key: SOLR-13764
> URL: https://issues.apache.org/jira/browse/SOLR-13764
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Reporter: Mikhail Khludnev
>Priority: Minor
>
> h2. Context
> Lucene has Intervals query LUCENE-8196. Note: these are a kind of healthy 
> man's Spans/Phrases. Note: It's not about ranges nor facets.
> h2. Problem
> There's no way to search by IntervalQuery via JSON Query DSL.
> h2. Suggestion
>  * Create classic QParser \{{ {!interval df=text_content}a_json_param}}, ie 
> one can combine a few such refs in {{json.query.bool}}
>  * It accepts just a name of JSON params, nothing like this happens yet.
>  * This param carries plain json which is accessible via {{req.getJSON()}}
> please examine 
> https://cwiki.apache.org/confluence/display/SOLR/SOLR-13764+Discussion+-+Interval+Queries+in+JSON
>  for syntax proposal.
> h2. Challenges
>  * I have no idea about particular JSON DSL for these queries, Lucene API 
> seems like easy JSON-able. Proposals are welcome.
>  * Another awkward things is combining analysis and low level query API. eg 
> what if one request term for one word and analysis yield two tokens, and vice 
> versa requesting phrase might end up with single token stream.
>  * Putting json into Jira ticket description
> h2. Q: Why don't..
> .. put intervals DSL right into {{json.query}}, avoiding these odd param 
> refs? 
>  A: It requires heavy lifting for {{JsonQueryConverter}} which is streamlined 
> for handling old good http parametrized queires.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13823) ClassCastException when using group.query and return score

2019-10-13 Thread Erick Erickson (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950462#comment-16950462
 ] 

Erick Erickson commented on SOLR-13823:
---

Uwe:

It'll be up to whoever commits this to assign the fix version. Currently 8.3 is 
being released and limited JIRAs are being committed to that branch, so 
probably this will go into 8.4.



> ClassCastException when using group.query and return score
> --
>
> Key: SOLR-13823
> URL: https://issues.apache.org/jira/browse/SOLR-13823
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Affects Versions: 8.2, 8.1.1
>Reporter: Uwe Jäger
>Assignee: Munendra S N
>Priority: Major
>  Labels: Grouping
> Attachments: SOLR-13823.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When grouping the results of a query with group.query there is a 
> ClassCastException in org.apache.solr.search.Grouping.CommandQuery.finish 
> (line 890) since the collector is wrapped in a MultiCollector. 
> The wanted topCollector is available in the inner class so it can be used 
> directly and the cast is not necessary at all. After that change there are 
> still the scores missing in the result, so populating the scores is 
> necessary, too.
> I will create a PR showing the error and also containing a fix.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-13824) JSON Request API ignores prematurely closing curly brace.

2019-10-13 Thread Mikhail Khludnev (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-13824:

Status: Patch Available  (was: Open)

> JSON Request API ignores prematurely closing curly brace. 
> --
>
> Key: SOLR-13824
> URL: https://issues.apache.org/jira/browse/SOLR-13824
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: JSON Request API
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-13824.patch
>
>
> {code:java}
> json={query:"content:foo", facet:{zz:{field:id}}}
> {code}
> this works fine, but if we mistype {{}}} instead of {{,}}
> {code:java}
> json={query:"content:foo"} facet:{zz:{field:id}}}
> {code}
> It's captured only partially, here's we have under debug
> {code:java}
>   "json":{"query":"content:foo"},
> {code}
> I suppose it should throw an error with 400 code.  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-13824) JSON Request API ignores prematurely closing curly brace.

2019-10-13 Thread Mikhail Khludnev (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-13824:

Attachment: SOLR-13824.patch
Status: Open  (was: Open)

> JSON Request API ignores prematurely closing curly brace. 
> --
>
> Key: SOLR-13824
> URL: https://issues.apache.org/jira/browse/SOLR-13824
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: JSON Request API
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-13824.patch
>
>
> {code:java}
> json={query:"content:foo", facet:{zz:{field:id}}}
> {code}
> this works fine, but if we mistype {{}}} instead of {{,}}
> {code:java}
> json={query:"content:foo"} facet:{zz:{field:id}}}
> {code}
> It's captured only partially, here's we have under debug
> {code:java}
>   "json":{"query":"content:foo"},
> {code}
> I suppose it should throw an error with 400 code.  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Comment Edited] (SOLR-13741) possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest

2019-10-13 Thread Jira


[ 
https://issues.apache.org/jira/browse/SOLR-13741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950421#comment-16950421
 ] 

Jan Høydahl edited comment on SOLR-13741 at 10/13/19 8:00 PM:
--

-Spun off SOLR-13840 for the bugs when generating event based on 
HttpServletRequest-

I folded this into SOLR-13835 and put up a PR. This is a contained fix that 
could go in 8.3 right now.


was (Author: janhoy):
Spun off SOLR-13840 for the bugs when generating event based on 
HttpServletRequest

> possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest
> --
>
> Key: SOLR-13741
> URL: https://issues.apache.org/jira/browse/SOLR-13741
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Chris M. Hostetter
>Assignee: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch, 
> SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch
>
>
> A while back i saw a weird non-reproducible failure from 
> AuditLoggerIntegrationTest.  When i started reading through that code, 2 
> things jumped out at me:
> # the way the 'delay' option works is brittle, and makes assumptions about 
> CPU scheduling that aren't neccessarily going to be true (and also suffers 
> from the problem that Thread.sleep isn't garunteed to sleep as long as you 
> ask it too)
> # the way the existing {{waitForAuditEventCallbacks(number)}} logic works by 
> checking the size of a (List) {{buffer}} of recieved events in a sleep/poll 
> loop, until it contains at least N items -- but the code that adds items to 
> that buffer in the async Callback thread async _before_ the code that updates 
> other state variables (like the global {{count}} and the patch specific 
> {{resourceCounts}}) meaning that a test waiting on 3 events could "see" 3 
> events added to the buffer, but calling {{assertEquals(3, 
> receiver.getTotalCount())}} could subsequently fail because that variable 
> hadn't been udpated yet.
> #2 was the source of the failures I was seeing, and while a quick fix for 
> that specific problem would be to update all other state _before_ adding the 
> event to the buffer, I set out to try and make more general improvements to 
> the test:
> * eliminate the dependency on sleep loops by {{await}}-ing on concurrent data 
> structures
> * harden the assertions made about the expected events recieved (updating 
> some test methods that currently just assert the number of events recieved)
> * add new assertions that _only_ the expected events are recieved.
> In the process of doing this, I've found several oddities/descrepencies 
> between things the test currently claims/asserts, and what *actually* happens 
> under more rigerous scrutiny/assertions.
> I'll attach a patch shortly that has my (in progress) updates and inlcudes 
> copious nocommits about things seem suspect.  the summary of these concerns 
> is:
> * SolrException status codes that do not match what the existing test says 
> they should (but doesn't assert)
> * extra AuditEvents occuring that the existing test does not expect
> * AuditEvents for incorrect credentials that do not at all match the expected 
> AuditEvent in the existing test -- which the current test seems to miss in 
> it's assertions because it's picking up some extra events from triggered by 
> previuos requests earlier in the test that just happen to also match the 
> asserctions.
> ...it's not clear to me if the test logic is correct and these are "code 
> bugs" or if the test is faulty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13829) RecursiveEvaluator casts Continuous numbers to Discrete Numbers, causing mismatch

2019-10-13 Thread Joel Bernstein (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950460#comment-16950460
 ] 

Joel Bernstein commented on SOLR-13829:
---

Let's open a separate issue to decide on a way forward. I'll comment first here 
and then open the ticket.

The issue of how to handle precision is an interesting question. I'd like to 
suggest the following approach:

1) Each math function returns a consistent type (long, double, bigdecimal) 
based on its design. For example if a function does high precision floating 
point math it returns a bigdecimal. If it always returns a whole number then it 
should return a long. If it doesn't do high precision floating point math it 
should return a double. Vector math and matrix math traditionally return double 
arrays and matrices, and I think this convention should be followed.

It's important to note that 90% of the math functions are implemented by Apache 
Commons Math which may or may not be using high precision techniques. So 
precision is going to be lost in places that we don't control.

2) All functions should expect Numbers for parameters and convert to what it 
expects internally. 

3) Users can use the *precision* function to manage floating point error if 
they choose to. A typical approach is to use 5 decimal places of precision to 
eliminate the floating point errors. This can be done in a stream before adding 
numbers to an aggregation, following an aggregation or applied after any of the 
math functions that are not operating with high precision.

What I like about this approach is that it's consistent internally, easily 
explainable, consistent with how other numeric systems like excel operate and 
provides techniques for achieving a specific level of precision if the user 
wants it.

 

 

 

> RecursiveEvaluator casts Continuous numbers to Discrete Numbers, causing 
> mismatch
> -
>
> Key: SOLR-13829
> URL: https://issues.apache.org/jira/browse/SOLR-13829
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Trey Grainger
>Priority: Major
> Fix For: 8.3
>
> Attachments: SOLR-13829.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In trying to use the "sort" streaming evaluator on float field (pfloat), I am 
> getting casting errors back based upon which values are calculated based upon 
> underlying values in a field.
> Example:
> *Docs:* (paste each into "Documents" pane in Solr Admin UI as type:"json")
>  
> {code:java}
> {"id": "1", "name":"donut","vector_fs":[5.0,0.0,1.0,5.0,0.0,4.0,5.0,1.0]}
> {"id": "2", "name":"cheese 
> pizza","vector_fs":[5.0,0.0,4.0,4.0,0.0,1.0,5.0,2.0]}{code}
>  
> *Streaming Expression:*
>  
> {code:java}
> sort(select(search(food_collection, q="*:*", fl="id,vector_fs", sort="id 
> asc"), cosineSimilarity(vector_fs, array(5.0,0.0,1.0,5.0,0.0,4.0,5.0,1.0)) as 
> sim, id), by="sim desc"){code}
>  
> *Response:*
>  
> {code:java}
> { 
>   "result-set": {
> "docs": [
>   {
> "EXCEPTION": "class java.lang.Double cannot be cast to class 
> java.lang.Long (java.lang.Double and java.lang.Long are in module java.base 
> of loader 'bootstrap')",
> "EOF": true,
> "RESPONSE_TIME": 13
>   }
> ]
>   }
> }{code}
>  
>  
> This is because in org.apache.solr.client.solrj.io.eval.RecursiveEvaluator, 
> there is a line which examines a numeric (BigDecimal) value and - regardless 
> of the type of the field the value originated from - converts it to a Long if 
> it looks like a whole number. This is the code in question from that class:
> {code:java}
> protected Object normalizeOutputType(Object value) {
> if(null == value){
>   return null;
> } else if (value instanceof VectorFunction) {
>   return value;
> } else if(value instanceof BigDecimal){
>   BigDecimal bd = (BigDecimal)value;
>   if(bd.signum() == 0 || bd.scale() <= 0 || 
> bd.stripTrailingZeros().scale() <= 0){
> try{
>   return bd.longValueExact();
> }
> catch(ArithmeticException e){
>   // value was too big for a long, so use a double which can handle 
> scientific notation
> }
>   }
>   
>   return bd.doubleValue();
> }
> ... [other type conversions]
> {code}
> Because of the *return bd.longValueExact()*; line, the calculated value for 
> "sim" in doc 1 is "Float(1)", whereas the calculated value for "sim" for doc 
> 2 is "Double(0.88938313). These are coming back as incompatible data types, 
> even though the source data is all of the same type and should be comparable.
> Thus when the *sort* evaluator streaming expression (and probably others) 
> runs on these calculated values and the list should 

[jira] [Commented] (SOLR-13835) HttpSolrCall produces incorrect extra AuditEvent on AuthorizationResponse.PROMPT

2019-10-13 Thread Jira


[ 
https://issues.apache.org/jira/browse/SOLR-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950454#comment-16950454
 ] 

Jan Høydahl commented on SOLR-13835:


{quote}Jan: Maybe i'm missing something, but IIUC in the context of how simple 
those blocks were when the code was initially added, it was reasonable for the 
first block to fall through to the second
{quote}
The logic was flawed originally since the code would return a 403 *instead* of 
a 401 due to the fall-through.

> HttpSolrCall produces incorrect extra AuditEvent on 
> AuthorizationResponse.PROMPT
> 
>
> Key: SOLR-13835
> URL: https://issues.apache.org/jira/browse/SOLR-13835
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Authentication, Authorization
>Reporter: Chris M. Hostetter
>Assignee: Jan Høydahl
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> spinning this out of SOLR-13741...
> {quote}
> Wrt the REJECTED + UNAUTHORIZED events I see the same as you, and I believe 
> there is a code bug, not a test bug. In HttpSolrCall#471 in the 
> {{authorize()}} call, if authResponse == PROMPT, it will actually match both 
> blocks and emit two audit events: 
> [https://github.com/apache/lucene-solr/blob/26ede632e6259eb9d16861a3c0f782c9c8999762/solr/core/src/java/org/apache/solr/servlet/HttpSolrCall.java#L475:L493]
>  
> {code:java}
> if (authResponse.statusCode == AuthorizationResponse.PROMPT.statusCode) {...}
> if (!(authResponse.statusCode == HttpStatus.SC_ACCEPTED) && 
> !(authResponse.statusCode == HttpStatus.SC_OK)) {...}
> {code}
> When code==401, it is also true that code!=200. Intuitively there should be 
> both a sendErrora and return RETURN before line #484 in the first if block?
> {quote}
> This causes any and all {{REJECTED}} AuditEvent messages to be accompanied by 
> a coresponding {{UNAUTHORIZED}} AuditEvent.  
> It's not yet clear if, from the perspective of the external client, there are 
> any other bugs in behavior (TBD)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13835) HttpSolrCall produces incorrect extra AuditEvent on AuthorizationResponse.PROMPT

2019-10-13 Thread Jira


[ 
https://issues.apache.org/jira/browse/SOLR-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950452#comment-16950452
 ] 

Jan Høydahl commented on SOLR-13835:


PR [GitHub Pull Request #946|https://github.com/apache/lucene-solr/pull/946] 
ready for review, also folding in changes from SOLR-13840 which is now marked 
as duplicate.

> HttpSolrCall produces incorrect extra AuditEvent on 
> AuthorizationResponse.PROMPT
> 
>
> Key: SOLR-13835
> URL: https://issues.apache.org/jira/browse/SOLR-13835
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Authentication, Authorization
>Reporter: Chris M. Hostetter
>Assignee: Jan Høydahl
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> spinning this out of SOLR-13741...
> {quote}
> Wrt the REJECTED + UNAUTHORIZED events I see the same as you, and I believe 
> there is a code bug, not a test bug. In HttpSolrCall#471 in the 
> {{authorize()}} call, if authResponse == PROMPT, it will actually match both 
> blocks and emit two audit events: 
> [https://github.com/apache/lucene-solr/blob/26ede632e6259eb9d16861a3c0f782c9c8999762/solr/core/src/java/org/apache/solr/servlet/HttpSolrCall.java#L475:L493]
>  
> {code:java}
> if (authResponse.statusCode == AuthorizationResponse.PROMPT.statusCode) {...}
> if (!(authResponse.statusCode == HttpStatus.SC_ACCEPTED) && 
> !(authResponse.statusCode == HttpStatus.SC_OK)) {...}
> {code}
> When code==401, it is also true that code!=200. Intuitively there should be 
> both a sendErrora and return RETURN before line #484 in the first if block?
> {quote}
> This causes any and all {{REJECTED}} AuditEvent messages to be accompanied by 
> a coresponding {{UNAUTHORIZED}} AuditEvent.  
> It's not yet clear if, from the perspective of the external client, there are 
> any other bugs in behavior (TBD)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Resolved] (SOLR-13840) AuditLogger issues when logged from HttpServletRequest

2019-10-13 Thread Jira


 [ 
https://issues.apache.org/jira/browse/SOLR-13840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl resolved SOLR-13840.

Resolution: Duplicate

Close as duplicate of SOLR-13835 since in order to get the test right after 
fixing SOLR-13835 I also had to fix the AdminEvent constructor mentioned as 
part of that issue.

> AuditLogger issues when logged from HttpServletRequest
> --
>
> Key: SOLR-13840
> URL: https://issues.apache.org/jira/browse/SOLR-13840
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Auditlogging
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
>
> Spinoff from SOLR-13741
> When a REJECTED event is generated from SolrDispatchFilter on failed 
> authentication, we only have the {{HttpServletRequest}} as input, no 
> SolrParams, Principal etc. In this case we parse "resource" from contextPath, 
> while we should use {{getPathInfo()}}. Also, we fail to detect admin requests 
> as such and get UNKNOWN instead. Lastly, the {{solrParams}} part of 
> {{AuditEvent}} is not filled at all from in this case, while we could have 
> filled it with the parameters in the request.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] janhoy opened a new pull request #946: SOLR-13835 HttpSolrCall produces incorrect extra AuditEvent on AuthorizationResponse.PROMPT

2019-10-13 Thread GitBox
janhoy opened a new pull request #946: SOLR-13835 HttpSolrCall produces 
incorrect extra AuditEvent on AuthorizationResponse.PROMPT
URL: https://github.com/apache/lucene-solr/pull/946
 
 
   # Description
   
   HttpSolrCall produces incorrect extra AuditEvent on 
AuthorizationResponse.PROMPT
   This also causes a 403 response instead of the correct 401 to the client
   
   # Solution
   
   Do not fall-through to next code block but return 401 response directly.
   
   # Tests
   
   Fixed the auth() test that previously expected a 403 to now expect 401
   
   # Checklist
   
   Please review the following and check all that apply:
   
   - [x] I have reviewed the guidelines for [How to 
Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms 
to the standards described there to the best of my ability.
   - [x] I have created a Jira issue and added the issue ID to my pull request 
title.
   - [x] I am authorized to contribute this code to the ASF and have removed 
any code I do not have a license to distribute.
   - [x] I have given Solr maintainers 
[access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork)
 to contribute to my PR branch. (optional but recommended)
   - [x] I have developed this patch against the `master` branch.
   - [x] I have run `ant precommit` and the appropriate test suite.
   - [x] I have added tests for my changes.
   - [ ] I have added documentation for the [Ref 
Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) 
(for Solr changes only).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Assigned] (SOLR-13835) HttpSolrCall produces incorrect extra AuditEvent on AuthorizationResponse.PROMPT

2019-10-13 Thread Jira


 [ 
https://issues.apache.org/jira/browse/SOLR-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl reassigned SOLR-13835:
--

Assignee: Jan Høydahl

> HttpSolrCall produces incorrect extra AuditEvent on 
> AuthorizationResponse.PROMPT
> 
>
> Key: SOLR-13835
> URL: https://issues.apache.org/jira/browse/SOLR-13835
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Authentication, Authorization
>Reporter: Chris M. Hostetter
>Assignee: Jan Høydahl
>Priority: Major
>
> spinning this out of SOLR-13741...
> {quote}
> Wrt the REJECTED + UNAUTHORIZED events I see the same as you, and I believe 
> there is a code bug, not a test bug. In HttpSolrCall#471 in the 
> {{authorize()}} call, if authResponse == PROMPT, it will actually match both 
> blocks and emit two audit events: 
> [https://github.com/apache/lucene-solr/blob/26ede632e6259eb9d16861a3c0f782c9c8999762/solr/core/src/java/org/apache/solr/servlet/HttpSolrCall.java#L475:L493]
>  
> {code:java}
> if (authResponse.statusCode == AuthorizationResponse.PROMPT.statusCode) {...}
> if (!(authResponse.statusCode == HttpStatus.SC_ACCEPTED) && 
> !(authResponse.statusCode == HttpStatus.SC_OK)) {...}
> {code}
> When code==401, it is also true that code!=200. Intuitively there should be 
> both a sendErrora and return RETURN before line #484 in the first if block?
> {quote}
> This causes any and all {{REJECTED}} AuditEvent messages to be accompanied by 
> a coresponding {{UNAUTHORIZED}} AuditEvent.  
> It's not yet clear if, from the perspective of the external client, there are 
> any other bugs in behavior (TBD)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13741) possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest

2019-10-13 Thread Jira


[ 
https://issues.apache.org/jira/browse/SOLR-13741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950421#comment-16950421
 ] 

Jan Høydahl commented on SOLR-13741:


Spun off SOLR-13840 for the bugs when generating event based on 
HttpServletRequest

> possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest
> --
>
> Key: SOLR-13741
> URL: https://issues.apache.org/jira/browse/SOLR-13741
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Chris M. Hostetter
>Assignee: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch, 
> SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch
>
>
> A while back i saw a weird non-reproducible failure from 
> AuditLoggerIntegrationTest.  When i started reading through that code, 2 
> things jumped out at me:
> # the way the 'delay' option works is brittle, and makes assumptions about 
> CPU scheduling that aren't neccessarily going to be true (and also suffers 
> from the problem that Thread.sleep isn't garunteed to sleep as long as you 
> ask it too)
> # the way the existing {{waitForAuditEventCallbacks(number)}} logic works by 
> checking the size of a (List) {{buffer}} of recieved events in a sleep/poll 
> loop, until it contains at least N items -- but the code that adds items to 
> that buffer in the async Callback thread async _before_ the code that updates 
> other state variables (like the global {{count}} and the patch specific 
> {{resourceCounts}}) meaning that a test waiting on 3 events could "see" 3 
> events added to the buffer, but calling {{assertEquals(3, 
> receiver.getTotalCount())}} could subsequently fail because that variable 
> hadn't been udpated yet.
> #2 was the source of the failures I was seeing, and while a quick fix for 
> that specific problem would be to update all other state _before_ adding the 
> event to the buffer, I set out to try and make more general improvements to 
> the test:
> * eliminate the dependency on sleep loops by {{await}}-ing on concurrent data 
> structures
> * harden the assertions made about the expected events recieved (updating 
> some test methods that currently just assert the number of events recieved)
> * add new assertions that _only_ the expected events are recieved.
> In the process of doing this, I've found several oddities/descrepencies 
> between things the test currently claims/asserts, and what *actually* happens 
> under more rigerous scrutiny/assertions.
> I'll attach a patch shortly that has my (in progress) updates and inlcudes 
> copious nocommits about things seem suspect.  the summary of these concerns 
> is:
> * SolrException status codes that do not match what the existing test says 
> they should (but doesn't assert)
> * extra AuditEvents occuring that the existing test does not expect
> * AuditEvents for incorrect credentials that do not at all match the expected 
> AuditEvent in the existing test -- which the current test seems to miss in 
> it's assertions because it's picking up some extra events from triggered by 
> previuos requests earlier in the test that just happen to also match the 
> asserctions.
> ...it's not clear to me if the test logic is correct and these are "code 
> bugs" or if the test is faulty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-13840) AuditLogger issues when logged from HttpServletRequest

2019-10-13 Thread Jira


 [ 
https://issues.apache.org/jira/browse/SOLR-13840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-13840:
---
Summary: AuditLogger issues when logged from HttpServletRequest  (was: 
AuditLogger issues with REJECTED state due to wrong PW)

> AuditLogger issues when logged from HttpServletRequest
> --
>
> Key: SOLR-13840
> URL: https://issues.apache.org/jira/browse/SOLR-13840
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Auditlogging
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
>
> Spinoff from SOLR-13741
> When a REJECTED event is generated from SolrDispatchFilter on failed 
> authentication, we only have the {{HttpServletRequest}} as input, no 
> SolrParams, Principal etc. In this case we parse "resource" from contextPath, 
> while we should use {{getPathInfo()}}. Also, we fail to detect admin requests 
> as such and get UNKNOWN instead. Lastly, the {{solrParams}} part of 
> {{AuditEvent}} is not filled at all from in this case, while we could have 
> filled it with the parameters in the request.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Created] (SOLR-13840) AuditLogger issues with REJECTED state due to wrong PW

2019-10-13 Thread Jira
Jan Høydahl created SOLR-13840:
--

 Summary: AuditLogger issues with REJECTED state due to wrong PW
 Key: SOLR-13840
 URL: https://issues.apache.org/jira/browse/SOLR-13840
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Auditlogging
Reporter: Jan Høydahl
Assignee: Jan Høydahl


Spinoff from SOLR-13741

When a REJECTED event is generated from SolrDispatchFilter on failed 
authentication, we only have the {{HttpServletRequest}} as input, no 
SolrParams, Principal etc. In this case we parse "resource" from contextPath, 
while we should use {{getPathInfo()}}. Also, we fail to detect admin requests 
as such and get UNKNOWN instead. Lastly, the {{solrParams}} part of 
{{AuditEvent}} is not filled at all from in this case, while we could have 
filled it with the parameters in the request.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-13741) possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest

2019-10-13 Thread Jira


 [ 
https://issues.apache.org/jira/browse/SOLR-13741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-13741:
---
Attachment: SOLR-13741.patch

> possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest
> --
>
> Key: SOLR-13741
> URL: https://issues.apache.org/jira/browse/SOLR-13741
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Chris M. Hostetter
>Assignee: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch, 
> SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch
>
>
> A while back i saw a weird non-reproducible failure from 
> AuditLoggerIntegrationTest.  When i started reading through that code, 2 
> things jumped out at me:
> # the way the 'delay' option works is brittle, and makes assumptions about 
> CPU scheduling that aren't neccessarily going to be true (and also suffers 
> from the problem that Thread.sleep isn't garunteed to sleep as long as you 
> ask it too)
> # the way the existing {{waitForAuditEventCallbacks(number)}} logic works by 
> checking the size of a (List) {{buffer}} of recieved events in a sleep/poll 
> loop, until it contains at least N items -- but the code that adds items to 
> that buffer in the async Callback thread async _before_ the code that updates 
> other state variables (like the global {{count}} and the patch specific 
> {{resourceCounts}}) meaning that a test waiting on 3 events could "see" 3 
> events added to the buffer, but calling {{assertEquals(3, 
> receiver.getTotalCount())}} could subsequently fail because that variable 
> hadn't been udpated yet.
> #2 was the source of the failures I was seeing, and while a quick fix for 
> that specific problem would be to update all other state _before_ adding the 
> event to the buffer, I set out to try and make more general improvements to 
> the test:
> * eliminate the dependency on sleep loops by {{await}}-ing on concurrent data 
> structures
> * harden the assertions made about the expected events recieved (updating 
> some test methods that currently just assert the number of events recieved)
> * add new assertions that _only_ the expected events are recieved.
> In the process of doing this, I've found several oddities/descrepencies 
> between things the test currently claims/asserts, and what *actually* happens 
> under more rigerous scrutiny/assertions.
> I'll attach a patch shortly that has my (in progress) updates and inlcudes 
> copious nocommits about things seem suspect.  the summary of these concerns 
> is:
> * SolrException status codes that do not match what the existing test says 
> they should (but doesn't assert)
> * extra AuditEvents occuring that the existing test does not expect
> * AuditEvents for incorrect credentials that do not at all match the expected 
> AuditEvent in the existing test -- which the current test seems to miss in 
> it's assertions because it's picking up some extra events from triggered by 
> previuos requests earlier in the test that just happen to also match the 
> asserctions.
> ...it's not clear to me if the test logic is correct and these are "code 
> bugs" or if the test is faulty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13741) possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest

2019-10-13 Thread Jira


[ 
https://issues.apache.org/jira/browse/SOLR-13741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950410#comment-16950410
 ] 

Jan Høydahl commented on SOLR-13741:


Hmm, will check where that json file could have hidden..

> possible AuditLogger bugs uncovered while hardening AuditLoggerIntegrationTest
> --
>
> Key: SOLR-13741
> URL: https://issues.apache.org/jira/browse/SOLR-13741
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Chris M. Hostetter
>Assignee: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-13741.patch, SOLR-13741.patch, SOLR-13741.patch, 
> SOLR-13741.patch, SOLR-13741.patch
>
>
> A while back i saw a weird non-reproducible failure from 
> AuditLoggerIntegrationTest.  When i started reading through that code, 2 
> things jumped out at me:
> # the way the 'delay' option works is brittle, and makes assumptions about 
> CPU scheduling that aren't neccessarily going to be true (and also suffers 
> from the problem that Thread.sleep isn't garunteed to sleep as long as you 
> ask it too)
> # the way the existing {{waitForAuditEventCallbacks(number)}} logic works by 
> checking the size of a (List) {{buffer}} of recieved events in a sleep/poll 
> loop, until it contains at least N items -- but the code that adds items to 
> that buffer in the async Callback thread async _before_ the code that updates 
> other state variables (like the global {{count}} and the patch specific 
> {{resourceCounts}}) meaning that a test waiting on 3 events could "see" 3 
> events added to the buffer, but calling {{assertEquals(3, 
> receiver.getTotalCount())}} could subsequently fail because that variable 
> hadn't been udpated yet.
> #2 was the source of the failures I was seeing, and while a quick fix for 
> that specific problem would be to update all other state _before_ adding the 
> event to the buffer, I set out to try and make more general improvements to 
> the test:
> * eliminate the dependency on sleep loops by {{await}}-ing on concurrent data 
> structures
> * harden the assertions made about the expected events recieved (updating 
> some test methods that currently just assert the number of events recieved)
> * add new assertions that _only_ the expected events are recieved.
> In the process of doing this, I've found several oddities/descrepencies 
> between things the test currently claims/asserts, and what *actually* happens 
> under more rigerous scrutiny/assertions.
> I'll attach a patch shortly that has my (in progress) updates and inlcudes 
> copious nocommits about things seem suspect.  the summary of these concerns 
> is:
> * SolrException status codes that do not match what the existing test says 
> they should (but doesn't assert)
> * extra AuditEvents occuring that the existing test does not expect
> * AuditEvents for incorrect credentials that do not at all match the expected 
> AuditEvent in the existing test -- which the current test seems to miss in 
> it's assertions because it's picking up some extra events from triggered by 
> previuos requests earlier in the test that just happen to also match the 
> asserctions.
> ...it's not clear to me if the test logic is correct and these are "code 
> bugs" or if the test is faulty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13823) ClassCastException when using group.query and return score

2019-10-13 Thread Jira


[ 
https://issues.apache.org/jira/browse/SOLR-13823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950376#comment-16950376
 ] 

Uwe Jäger commented on SOLR-13823:
--

If I resolve this, what will be the fix version?

> ClassCastException when using group.query and return score
> --
>
> Key: SOLR-13823
> URL: https://issues.apache.org/jira/browse/SOLR-13823
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Affects Versions: 8.2, 8.1.1
>Reporter: Uwe Jäger
>Assignee: Munendra S N
>Priority: Major
>  Labels: Grouping
> Attachments: SOLR-13823.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When grouping the results of a query with group.query there is a 
> ClassCastException in org.apache.solr.search.Grouping.CommandQuery.finish 
> (line 890) since the collector is wrapped in a MultiCollector. 
> The wanted topCollector is available in the inner class so it can be used 
> directly and the cast is not necessary at all. After that change there are 
> still the scores missing in the result, so populating the scores is 
> necessary, too.
> I will create a PR showing the error and also containing a fix.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13823) ClassCastException when using group.query and return score

2019-10-13 Thread Jira


[ 
https://issues.apache.org/jira/browse/SOLR-13823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950375#comment-16950375
 ] 

Uwe Jäger commented on SOLR-13823:
--

The patch looks good to me and can be applied. 

> ClassCastException when using group.query and return score
> --
>
> Key: SOLR-13823
> URL: https://issues.apache.org/jira/browse/SOLR-13823
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Affects Versions: 8.2, 8.1.1
>Reporter: Uwe Jäger
>Assignee: Munendra S N
>Priority: Major
>  Labels: Grouping
> Attachments: SOLR-13823.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When grouping the results of a query with group.query there is a 
> ClassCastException in org.apache.solr.search.Grouping.CommandQuery.finish 
> (line 890) since the collector is wrapped in a MultiCollector. 
> The wanted topCollector is available in the inner class so it can be used 
> directly and the cast is not necessary at all. After that change there are 
> still the scores missing in the result, so populating the scores is 
> necessary, too.
> I will create a PR showing the error and also containing a fix.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13839) MaxScore is returned as NAN when group.query doesn't match any docs

2019-10-13 Thread Munendra S N (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950333#comment-16950333
 ] 

Munendra S N commented on SOLR-13839:
-

 [^SOLR-13839.patch] 
Patch contains a failing test

> MaxScore is returned as NAN when group.query doesn't match any docs
> ---
>
> Key: SOLR-13839
> URL: https://issues.apache.org/jira/browse/SOLR-13839
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Reporter: Munendra S N
>Priority: Minor
> Attachments: SOLR-13839.patch
>
>
> When the main query matches some products but group.query doesn't match any 
> docs then maxScore=NAN would be returned in the response.
> * This happens only in standalone/single shard mode
> * score needs to fetched in the response to encounter this issue



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-13839) MaxScore is returned as NAN when group.query doesn't match any docs

2019-10-13 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-13839:

Attachment: SOLR-13839.patch

> MaxScore is returned as NAN when group.query doesn't match any docs
> ---
>
> Key: SOLR-13839
> URL: https://issues.apache.org/jira/browse/SOLR-13839
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Reporter: Munendra S N
>Priority: Minor
> Attachments: SOLR-13839.patch
>
>
> When the main query matches some products but group.query doesn't match any 
> docs then maxScore=NAN would be returned in the response.
> * This happens only in standalone/single shard mode
> * score needs to fetched in the response to encounter this issue



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Created] (SOLR-13839) MaxScore is returned as NAN when group.query doesn't match any docs

2019-10-13 Thread Munendra S N (Jira)
Munendra S N created SOLR-13839:
---

 Summary: MaxScore is returned as NAN when group.query doesn't 
match any docs
 Key: SOLR-13839
 URL: https://issues.apache.org/jira/browse/SOLR-13839
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: search
Reporter: Munendra S N


When the main query matches some products but group.query doesn't match any 
docs then maxScore=NAN would be returned in the response.

* This happens only in standalone/single shard mode
* score needs to fetched in the response to encounter this issue



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13823) ClassCastException when using group.query and return score

2019-10-13 Thread Munendra S N (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950321#comment-16950321
 ] 

Munendra S N commented on SOLR-13823:
-

 [^SOLR-13823.patch] 
* This includes fix for distributed case too
* Main query is used to populate scores. This is similar to {{group.field}} and 
{{group.func}}. Also, scores should be same even when not sorted by {{score}}
* fixes populating scores twice

[~uwej711]
Could you please review this? I have attached the patch, so that patch review 
would be run. Currently, For Github PR only precommit checks are done but for 
patches test suites is also run. Once all the tests pass, we can move changes 
to github PR or patch based on your preference


> ClassCastException when using group.query and return score
> --
>
> Key: SOLR-13823
> URL: https://issues.apache.org/jira/browse/SOLR-13823
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Affects Versions: 8.2, 8.1.1
>Reporter: Uwe Jäger
>Priority: Major
>  Labels: Grouping
> Attachments: SOLR-13823.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When grouping the results of a query with group.query there is a 
> ClassCastException in org.apache.solr.search.Grouping.CommandQuery.finish 
> (line 890) since the collector is wrapped in a MultiCollector. 
> The wanted topCollector is available in the inner class so it can be used 
> directly and the cast is not necessary at all. After that change there are 
> still the scores missing in the result, so populating the scores is 
> necessary, too.
> I will create a PR showing the error and also containing a fix.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Assigned] (SOLR-13823) ClassCastException when using group.query and return score

2019-10-13 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N reassigned SOLR-13823:
---

Assignee: Munendra S N

> ClassCastException when using group.query and return score
> --
>
> Key: SOLR-13823
> URL: https://issues.apache.org/jira/browse/SOLR-13823
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Affects Versions: 8.2, 8.1.1
>Reporter: Uwe Jäger
>Assignee: Munendra S N
>Priority: Major
>  Labels: Grouping
> Attachments: SOLR-13823.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When grouping the results of a query with group.query there is a 
> ClassCastException in org.apache.solr.search.Grouping.CommandQuery.finish 
> (line 890) since the collector is wrapped in a MultiCollector. 
> The wanted topCollector is available in the inner class so it can be used 
> directly and the cast is not necessary at all. After that change there are 
> still the scores missing in the result, so populating the scores is 
> necessary, too.
> I will create a PR showing the error and also containing a fix.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-13823) ClassCastException when using group.query and return score

2019-10-13 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-13823:

Status: Patch Available  (was: Open)

> ClassCastException when using group.query and return score
> --
>
> Key: SOLR-13823
> URL: https://issues.apache.org/jira/browse/SOLR-13823
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Affects Versions: 8.2, 8.1.1
>Reporter: Uwe Jäger
>Priority: Major
>  Labels: Grouping
> Attachments: SOLR-13823.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When grouping the results of a query with group.query there is a 
> ClassCastException in org.apache.solr.search.Grouping.CommandQuery.finish 
> (line 890) since the collector is wrapped in a MultiCollector. 
> The wanted topCollector is available in the inner class so it can be used 
> directly and the cast is not necessary at all. After that change there are 
> still the scores missing in the result, so populating the scores is 
> necessary, too.
> I will create a PR showing the error and also containing a fix.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-13823) ClassCastException when using group.query and return score

2019-10-13 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-13823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-13823:

Attachment: SOLR-13823.patch

> ClassCastException when using group.query and return score
> --
>
> Key: SOLR-13823
> URL: https://issues.apache.org/jira/browse/SOLR-13823
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Affects Versions: 8.2, 8.1.1
>Reporter: Uwe Jäger
>Priority: Major
>  Labels: Grouping
> Attachments: SOLR-13823.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When grouping the results of a query with group.query there is a 
> ClassCastException in org.apache.solr.search.Grouping.CommandQuery.finish 
> (line 890) since the collector is wrapped in a MultiCollector. 
> The wanted topCollector is available in the inner class so it can be used 
> directly and the cast is not necessary at all. After that change there are 
> still the scores missing in the result, so populating the scores is 
> necessary, too.
> I will create a PR showing the error and also containing a fix.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-9985) LukeRequestHandler doesn’t populate docFreq for PointFields

2019-10-13 Thread Bar Rotstein (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950303#comment-16950303
 ] 

Bar Rotstein commented on SOLR-9985:


I have added a simple unit test on top of this patch and created a 
[PR|https://github.com/apache/lucene-solr/pull/945]

> LukeRequestHandler doesn’t populate docFreq for PointFields
> ---
>
> Key: SOLR-9985
> URL: https://issues.apache.org/jira/browse/SOLR-9985
> Project: Solr
>  Issue Type: Improvement
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Minor
>  Labels: numeric-tries-to-points
> Attachments: SOLR-9985.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Followup task of SOLR-8396



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] barrotsteindev opened a new pull request #945: SOLR-9985: add docFreq for PointFields

2019-10-13 Thread GitBox
barrotsteindev opened a new pull request #945: SOLR-9985: add docFreq for 
PointFields
URL: https://github.com/apache/lucene-solr/pull/945
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-9985) LukeRequestHandler doesn’t populate docFreq for PointFields

2019-10-13 Thread Bar Rotstein (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950291#comment-16950291
 ] 

Bar Rotstein commented on SOLR-9985:


It seems like PointField#readableToIndexed and PointField#storedToIndexed are 
unsupported for points based fields.
 Would there be another way of using SolrIndexSearcher#docFreq for point fields?

> LukeRequestHandler doesn’t populate docFreq for PointFields
> ---
>
> Key: SOLR-9985
> URL: https://issues.apache.org/jira/browse/SOLR-9985
> Project: Solr
>  Issue Type: Improvement
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Minor
>  Labels: numeric-tries-to-points
> Attachments: SOLR-9985.patch
>
>
> Followup task of SOLR-8396



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-9985) LukeRequestHandler doesn’t populate docFreq for PointFields

2019-10-13 Thread Bar Rotstein (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950236#comment-16950236
 ] 

Bar Rotstein commented on SOLR-9985:


I'd like to try and give this a go if it is alright.

> LukeRequestHandler doesn’t populate docFreq for PointFields
> ---
>
> Key: SOLR-9985
> URL: https://issues.apache.org/jira/browse/SOLR-9985
> Project: Solr
>  Issue Type: Improvement
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Minor
>  Labels: numeric-tries-to-points
> Attachments: SOLR-9985.patch
>
>
> Followup task of SOLR-8396



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org