[jira] [Updated] (SOLR-10677) Expose a diagnostics API to gives details of how loaded each node is, according to the policies

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-10677:
-
Description: 
Expose a diagnostics API to gives details of how loaded each node is, according 
to the configured cluster policy and preferences. This should probably fold 
into the autoscaling read APIs added in SOLR-10373.

By adding {{diagnostics=true}} as a request parameter for /admin/autoscaling 
(for v1 API) or /cluster/autoscaling (for v2 API), the API returns the list of 
nodes in sorted order according to the cluster preferences configured.

API v1:
{code}
curl 
http://localhost:8983/solr/admin/autoscaling?diagnostics=true&wt=json&indent=on
{code}
or API v2:
{code}
http://localhost:8983/v2/cluster/autoscaling?diagnostics=true&wt=json&indent=on
{code}
{code}
{
  "responseHeader": {
"status": 0,
"QTime": 99
  },
  "cluster-preferences": [
{
  "minimize": "cores",
  "precision": 3
},
{
  "maximize": "freedisk",
  "precision": 100
},
{
  "minimize": "cpu",
  "precision": 10
}
  ],
  "diagnostics": [
{
  "127.0.1.1:7574_solr": [
{
  "replicas": {
"gettingstarted": {
  "shard2": [
{
  "core_node4": {}
}
  ],
  "shard1": [
{
  "core_node3": {}
}
  ]
}
  }
},
{
  "cores": 2
},
{
  "freedisk": 83619053568
},
{
  "cpu": null
},
{
  "nodeRole": null
},
{
  "node": "127.0.1.1:7574_solr"
}
  ]
},
{
  "127.0.1.1:8983_solr": [
{
  "replicas": {
"gettingstarted": {
  "shard2": [
{
  "core_node1": {}
}
  ],
  "shard1": [
{
  "core_node2": {}
}
  ]
}
  }
},
{
  "cores": 2
},
{
  "freedisk": 83619053568
},
{
  "cpu": null
},
{
  "nodeRole": null
},
{
  "node": "127.0.1.1:8983_solr"
}
  ]
}
  ],
  "WARNING": "This response format is experimental.  It is likely to change in 
the future."
}
{code}

  was:Expose a diagnostics API to gives details of how loaded each node is, 
according to the configured cluster policy and preferences. This should 
probably fold into the autoscaling read APIs added in SOLR-10373.


> Expose a diagnostics API to gives details of how loaded each node is, 
> according to the policies
> ---
>
> Key: SOLR-10677
> URL: https://issues.apache.org/jira/browse/SOLR-10677
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>  Labels: autoscaling
> Fix For: master (7.0)
>
> Attachments: SOLR-10677.patch
>
>
> Expose a diagnostics API to gives details of how loaded each node is, 
> according to the configured cluster policy and preferences. This should 
> probably fold into the autoscaling read APIs added in SOLR-10373.
> By adding {{diagnostics=true}} as a request parameter for /admin/autoscaling 
> (for v1 API) or /cluster/autoscaling (for v2 API), the API returns the list 
> of nodes in sorted order according to the cluster preferences configured.
> API v1:
> {code}
> curl 
> http://localhost:8983/solr/admin/autoscaling?diagnostics=true&wt=json&indent=on
> {code}
> or API v2:
> {code}
> http://localhost:8983/v2/cluster/autoscaling?diagnostics=true&wt=json&indent=on
> {code}
> {code}
> {
>   "responseHeader": {
> "status": 0,
> "QTime": 99
>   },
>   "cluster-preferences": [
> {
>   "minimize": "cores",
>   "precision": 3
> },
> {
>   "maximize": "freedisk",
>   "precision": 100
> },
> {
>   "minimize": "cpu",
>   "precision": 10
> }
>   ],
>   "diagnostics": [
> {
>   "127.0.1.1:7574_solr": [
> {
>   "replicas": {
> "gettingstarted": {
>   "shard2": [
> {
>   "core_node4": {}
> }
>   ],
>   "shard1": [
> {
>   "core_node3": {}
> }
>   ]
> }
>   }
> },
> {
>   "cores": 2
> },
> {
>   "freedisk": 83619053568
> },
>

[jira] [Updated] (LUCENE-7800) Remove code that potentially rethrows checked exceptions from methods that don't declare them ("sneaky throw" hack)

[Dawid Weiss (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Weiss updated LUCENE-7800:

Attachment: LUCENE-7800.patch

Precommit (javadoc) and ParseException unwrapping in the compiler pulled up.

> Remove code that potentially rethrows checked exceptions from methods that 
> don't declare them ("sneaky throw" hack)
> ---
>
> Key: LUCENE-7800
> URL: https://issues.apache.org/jira/browse/LUCENE-7800
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Minor
> Fix For: 6.x, master (7.0)
>
> Attachments: LUCENE-7800.patch, LUCENE-7800.patch, LUCENE-7800.patch
>
>
> For a long time I considered the "sneaky" throw hack to be a nice way of 
> coding around some of Java's limitations (especially with invoking methods 
> via reflection or method handles), but with time I started to see how it can 
> be potentially dangerous and is nearly always confusing. If you have a Java 
> method and its signature doesn't indicate the possibility of a checked 
> exception you, as a programmer, simply don't expect it to happen. Never. So, 
> for example, you could write:
> {code}
> try {
>  luceneApi();
> } catch (RuntimeException | Error e) {
>   // Handle unchecked exceptions here.
> }
> {code}
> and consider the code above to be absolutely bullet-proof in terms of 
> handling exceptions. Unfortunately with sneaky throws anywhere in the 
> "luceneApi" this is no longer the case -- you can receive a checked exception 
> that will simply fall through and hit some code frame above.
> So I suggest we remove sneaky throw from the core entirely. It only exists in 
> two places -- private methods inside Snowball programs invoked via method 
> handles (these don't even declare checked exceptions so I assume they can't 
> occur) and AttributeFactory -- here there is a real possibility somebody 
> could declare an attribute class's constructor that does throw an unchecked 
> exception. In that case I think it is more reasonable to wrap it in a 
> RuntimeException than rethrow it as original.
> Alternatively, we can modify the signature of {{createAttributeInstance}} and 
> {{getStaticImplementation}} to declare some kind of checked exception (either 
> a wrapper or even a Throwable), but I see little reason for it and it'd 
> change the API.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10694) IteratorWriter and MapWriter interfaces should be supported by all response formats

[Shalin Shekhar Mangar (JIRA)]

Shalin Shekhar Mangar created SOLR-10694:


 Summary: IteratorWriter and MapWriter interfaces should be 
supported by all response formats
 Key: SOLR-10694
 URL: https://issues.apache.org/jira/browse/SOLR-10694
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Response Writers
Reporter: Shalin Shekhar Mangar
 Fix For: master (7.0), 6.7


Initially written for streaming, the serialization of IteratorWriter and 
MapWriter objects are only implemented for JSON and javabin response writers. 
This is kinda trappy as I found in SOLR-10677.

All response writers should implement serialization of these two types.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Vivek Narang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011811#comment-16011811
 ] 

Vivek Narang commented on SOLR-10317:
-

Hi [~ichattopadhyaya] I was trying to create an improved UI. Please access 
[http://162.243.101.83/IndexingBenchmarkStandalone.html] and let me know what 
you think. This is not yet final and I will be adding 
features/options/controls/data (Like an option to view test environment related 
information etc.) on this soon. Thanks. 

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: changes-lucene-20160907.json, 
> changes-solr-20160907.json, managed-schema, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf, 
> solrconfig.xml
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Vivek Narang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011691#comment-16011691
 ] 

Vivek Narang commented on SOLR-10317:
-

Yes [~arafalov] :)

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: changes-lucene-20160907.json, 
> changes-solr-20160907.json, managed-schema, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf, 
> solrconfig.xml
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10317) Solr Nightly Benchmarks

[Vivek Narang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011667#comment-16011667
 ] 

Vivek Narang edited comment on SOLR-10317 at 5/16/17 3:08 AM:
--

Hello [~ichattopadhyaya] I also found out another dataset which can be used for 
some scenarios/features. Please see 
[https://www.kaggle.com/snap/amazon-fine-food-reviews]. This data set is huge 
with over half a million records and ten fields. There is a good mix of text 
and numeric fields. The current indexing time as observed is 1222 seconds on a 
standalone node. Please access the file (~250MB) here: 
[http://162.243.101.83/Reviews.csv]. I think this is awesome! Please let me 
know what you think. Thanks. 

--- Data Structure Details ---
Id
ProductId - unique identifier for the product
UserId - unqiue identifier for the user
ProfileName
HelpfulnessNumerator - number of users who found the review helpful
HelpfulnessDenominator - number of users who indicated whether they found the 
review helpful
Score - rating between 1 and 5
Time - timestamp for the review
Summary - brief summary of the review
Text - text of the review
---


was (Author: vivek.nar...@uga.edu):
Hello [~ichattopadhyaya] I also found out another dataset which can be used for 
some scenarios/features. Please see 
[https://www.kaggle.com/snap/amazon-fine-food-reviews]. This data set is huge 
with over half a million records and ten fields. There is a good mix of text 
and numeric fields. The current indexing time as observed is 1222 seconds on a 
standalone node. Please access the file (~250MB) here: 
[http://162.243.101.83/Reviews.csv]. I think this is awesome! Please let me 
know what you think. Thanks. 

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: changes-lucene-20160907.json, 
> changes-solr-20160907.json, managed-schema, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf, 
> solrconfig.xml
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011668#comment-16011668
 ] 

Alexandre Rafalovitch commented on SOLR-10317:
--

Amazon one looks interesting. And it looks like it could be made nested with 
product record as a parent.

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: changes-lucene-20160907.json, 
> changes-solr-20160907.json, managed-schema, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf, 
> solrconfig.xml
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Vivek Narang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011667#comment-16011667
 ] 

Vivek Narang commented on SOLR-10317:
-

Hello [~ichattopadhyaya] I also found out another dataset which can be used for 
some scenarios/features. Please see 
[https://www.kaggle.com/snap/amazon-fine-food-reviews]. This data set is huge 
with over half a million records and ten fields. There is a good mix of text 
and numeric fields. The current indexing time as observed is 1222 seconds on a 
standalone node. Please access the file (~250MB) here: 
[http://162.243.101.83/Reviews.csv]. I think this is awesome! Please let me 
know what you think. Thanks. 

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: changes-lucene-20160907.json, 
> changes-solr-20160907.json, managed-schema, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf, 
> solrconfig.xml
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7822) IllegalArgumentException thrown instead of a CorruptIndexException

[Martin Amirault (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011628#comment-16011628
 ] 

Martin Amirault edited comment on LUCENE-7822 at 5/16/17 2:43 AM:
--

Created a patch for master branch, along with a test to reproduce the problem.

Actually the code doing the checksum is here with CodecUtil.checkFooter but is 
not called in a finally clause. Using it in a finally clause would hide some 
exceptions, so I decided to go with [~jpountz] suggestion


was (Author: marumarutan):
Created a patch for master branch, along with a test to reproduce the problem.

Actually the code doing the checksum was here, just not called in a finally 
clause.

> IllegalArgumentException thrown instead of a CorruptIndexException
> --
>
> Key: LUCENE-7822
> URL: https://issues.apache.org/jira/browse/LUCENE-7822
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.5.1
>Reporter: Martin Amirault
>Priority: Minor
> Attachments: LUCENE-7822.patch
>
>
> Similarly to LUCENE-7592 , When an {{*.si}} file is corrupted on very 
> specific part an IllegalArgumentException is thrown instead of a 
> CorruptIndexException.
> StackTrace (Lucene 6.5.1):
> {code}
> java.lang.IllegalArgumentException: Illegal minor version: 12517381
>   at 
> __randomizedtesting.SeedInfo.seed([1FEB5987CFA44BE:B8755B5574F9F3BF]:0)
>   at org.apache.lucene.util.Version.(Version.java:385)
>   at org.apache.lucene.util.Version.(Version.java:371)
>   at org.apache.lucene.util.Version.fromBits(Version.java:353)
>   at 
> org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:97)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}
> Simple fix would be to add IllegalArgumentException to the catch list at 
> {{org/apache/lucene/index/SegmentInfos.java:289}}
> Other variations for the stacktraces:
> {code}
> java.lang.IllegalArgumentException: invalid codec filename '_�.cfs', must 
> match: _[a-z0-9]+(_.*)?\..*
>   at 
> __randomizedtesting.SeedInfo.seed([8B3FDE317B8D634A:A8EE07E5EB4B0B13]:0)
>   at 
> org.apache.lucene.index.SegmentInfo.checkFileNames(SegmentInfo.java:270)
>   at org.apache.lucene.index.SegmentInfo.addFiles(SegmentInfo.java:252)
>   at org.apache.lucene.index.SegmentInfo.setFiles(SegmentInfo.java:246)
>   at 
> org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:248)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}
> {code}
> java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.Codec with name 'LucenI62' does not exist.  You need 
> to add the corresponding JAR file supporting this SPI to your classpath.  The 
> current classpath supports the following names: [Lucene62, Lucene50, 
> Lucene53, Lucene54, Lucene60]
>   at 
> __randomizedtesting.SeedInfo.seed([925DE160F7260F99:B026EB9373CB6368]:0)
>   at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
>   at org.apache.lucene.codecs.Codec.forName(Codec.java:116)
>   at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:424)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:356)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.luce

[jira] [Updated] (LUCENE-7822) IllegalArgumentException thrown instead of a CorruptIndexException

[Martin Amirault (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martin Amirault updated LUCENE-7822:

Attachment: LUCENE-7822.patch

> IllegalArgumentException thrown instead of a CorruptIndexException
> --
>
> Key: LUCENE-7822
> URL: https://issues.apache.org/jira/browse/LUCENE-7822
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.5.1
>Reporter: Martin Amirault
>Priority: Minor
> Attachments: LUCENE-7822.patch
>
>
> Similarly to LUCENE-7592 , When an {{*.si}} file is corrupted on very 
> specific part an IllegalArgumentException is thrown instead of a 
> CorruptIndexException.
> StackTrace (Lucene 6.5.1):
> {code}
> java.lang.IllegalArgumentException: Illegal minor version: 12517381
>   at 
> __randomizedtesting.SeedInfo.seed([1FEB5987CFA44BE:B8755B5574F9F3BF]:0)
>   at org.apache.lucene.util.Version.(Version.java:385)
>   at org.apache.lucene.util.Version.(Version.java:371)
>   at org.apache.lucene.util.Version.fromBits(Version.java:353)
>   at 
> org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:97)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}
> Simple fix would be to add IllegalArgumentException to the catch list at 
> {{org/apache/lucene/index/SegmentInfos.java:289}}
> Other variations for the stacktraces:
> {code}
> java.lang.IllegalArgumentException: invalid codec filename '_�.cfs', must 
> match: _[a-z0-9]+(_.*)?\..*
>   at 
> __randomizedtesting.SeedInfo.seed([8B3FDE317B8D634A:A8EE07E5EB4B0B13]:0)
>   at 
> org.apache.lucene.index.SegmentInfo.checkFileNames(SegmentInfo.java:270)
>   at org.apache.lucene.index.SegmentInfo.addFiles(SegmentInfo.java:252)
>   at org.apache.lucene.index.SegmentInfo.setFiles(SegmentInfo.java:246)
>   at 
> org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:248)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}
> {code}
> java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.Codec with name 'LucenI62' does not exist.  You need 
> to add the corresponding JAR file supporting this SPI to your classpath.  The 
> current classpath supports the following names: [Lucene62, Lucene50, 
> Lucene53, Lucene54, Lucene60]
>   at 
> __randomizedtesting.SeedInfo.seed([925DE160F7260F99:B026EB9373CB6368]:0)
>   at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
>   at org.apache.lucene.codecs.Codec.forName(Codec.java:116)
>   at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:424)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:356)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-

[jira] [Updated] (LUCENE-7822) IllegalArgumentException thrown instead of a CorruptIndexException

[Martin Amirault (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martin Amirault updated LUCENE-7822:

Attachment: (was: LUCENE-7822.patch)

> IllegalArgumentException thrown instead of a CorruptIndexException
> --
>
> Key: LUCENE-7822
> URL: https://issues.apache.org/jira/browse/LUCENE-7822
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.5.1
>Reporter: Martin Amirault
>Priority: Minor
>
> Similarly to LUCENE-7592 , When an {{*.si}} file is corrupted on very 
> specific part an IllegalArgumentException is thrown instead of a 
> CorruptIndexException.
> StackTrace (Lucene 6.5.1):
> {code}
> java.lang.IllegalArgumentException: Illegal minor version: 12517381
>   at 
> __randomizedtesting.SeedInfo.seed([1FEB5987CFA44BE:B8755B5574F9F3BF]:0)
>   at org.apache.lucene.util.Version.(Version.java:385)
>   at org.apache.lucene.util.Version.(Version.java:371)
>   at org.apache.lucene.util.Version.fromBits(Version.java:353)
>   at 
> org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:97)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}
> Simple fix would be to add IllegalArgumentException to the catch list at 
> {{org/apache/lucene/index/SegmentInfos.java:289}}
> Other variations for the stacktraces:
> {code}
> java.lang.IllegalArgumentException: invalid codec filename '_�.cfs', must 
> match: _[a-z0-9]+(_.*)?\..*
>   at 
> __randomizedtesting.SeedInfo.seed([8B3FDE317B8D634A:A8EE07E5EB4B0B13]:0)
>   at 
> org.apache.lucene.index.SegmentInfo.checkFileNames(SegmentInfo.java:270)
>   at org.apache.lucene.index.SegmentInfo.addFiles(SegmentInfo.java:252)
>   at org.apache.lucene.index.SegmentInfo.setFiles(SegmentInfo.java:246)
>   at 
> org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:248)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}
> {code}
> java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.Codec with name 'LucenI62' does not exist.  You need 
> to add the corresponding JAR file supporting this SPI to your classpath.  The 
> current classpath supports the following names: [Lucene62, Lucene50, 
> Lucene53, Lucene54, Lucene60]
>   at 
> __randomizedtesting.SeedInfo.seed([925DE160F7260F99:B026EB9373CB6368]:0)
>   at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
>   at org.apache.lucene.codecs.Codec.forName(Codec.java:116)
>   at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:424)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:356)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev

[jira] [Closed] (SOLR-4473) Reloading a core will not close (leak) associated DIH JDBC connection

[Alexandre Rafalovitch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandre Rafalovitch closed SOLR-4473.
---
   Resolution: Won't Fix
Fix Version/s: (was: 6.0)
   (was: 4.9)

Derby is not used anywhere in Solr and will be - soon - deleted as part of DIH 
example re-implementation (SOLR-10312).

> Reloading a core will not close (leak) associated DIH JDBC connection
> -
>
> Key: SOLR-4473
> URL: https://issues.apache.org/jira/browse/SOLR-4473
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 4.1
>Reporter: Alexandre Rafalovitch
>Assignee: Alexandre Rafalovitch
>
> I have DIH configured with Derby database. After I start Solr, I can run DIH 
> import fine. After I reload the core, DIH can no longer run with the 
> following message (excerpts): 
> ...
> EVERE: Exception while processing: vac document : 
> SolrInputDocument[]:org.apache.solr.handler.dataimport.DataImportHandlerException:
>  Unable to execute query: select * from ALERTS Processing Document # 1
>   at 
> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:71)
>   at 
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:253)
>   at 
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
>   at 
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:38)
>   at 
> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
>   at 
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
>   at 
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
>   at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:465)
> Caused by: java.sql.SQLException: Another instance of Derby may have already 
> booted the database .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10655) refactor duplicate ref guide content into "snippet" files that can be included

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011629#comment-16011629
 ] 

Hoss Man commented on SOLR-10655:
-

bq. For a lot of our content, this may be a simpler approach. The syntax for 
using tags like this is explained in: 
http://asciidoctor.org/docs/user-manual/#include-partial

the "include tagged portion" does look like a perfect fit for what we were 
using in cwiki -- and i agree that model of including a tagged portion of 
another file is probably a better fit for what we want to do, because in all 
the cases i can think of one page should really "own" the content, and it's 
just a matter of other pages "citing" that content.

but in practice it seems to have bugs that will break links between the 2 pages 
-- which seems like a deal breaker, since we explicitly draw attention to this 
relationship in our pages (and i don't think we should stop doing that)...

https://github.com/asciidoctor/asciidoctor/issues/2200

I'm attaching a patch that tries to use the 
{{include::foo.adoc\[tags=some_excerpt\]}} syntax in all places that currently 
have a "TODO SOLR-10655" related comment -- see nocommits in the patch and the 
ultimate build failure messages from {{ant build-site}} because of the our 
automatic link checking.

bq. I personally split this problem into two separate use cases - first, we 
used excerpt to save having to remember where we were maintaining a list of 
related pages...like a copied TOC. This use case doesn't actually interest me 
that much going forward, because I think it's a one-off that we are unlikely to 
repeat. With better global find/replace at our disposal now, the task of 
maintaining those pages is 1000x easier even if it doesn't reside in a single 
place.

I disagree ... i think that unless asciidoctor issue#2200 has an easy 
workaround we should move forward with refactoring this duplicated 
documentation into some "snippet" files that can be included in multiple 
places.  Even if search and replace is much easier with adoc then with cwiki, 
it's too easy for someone who doesn't realize the content is duplicated to make 
an edit in to only one copy.  we can have all the {{// NOTE: this table is 
duplicated in defining-fields.adoc}} type comments we want, but those may still 
be easily overlooked by someone who (for example) knows they need to edit one 
cell of a table in {{field-type-definitions-and-properties.adoc}} and uses uses 
their editor's search feature to jump to the relevant line -- never realizing 
there is a big ass comment off screen above that table informing them about 
another page they may not have ever paid attention to.

(I would much rather use the "include tag" syntax then having these one off 
snippet files, but given the viable options i'd much rather someone opening a 
file to edit it find an {{include::snippet/foo.adoc}} they didn't expect -- 
drawing attention to the fact that the content is re-used in multiple places -- 
then find the content they expect, but overlook that it's duplicated in 
multiple places)



bq. ensuring anchors and relative links are generated in such a way that they 
are still unique per-section

Note for future, there are easy ways to deal with this as long as the included 
file _expects_ to be included in multiple places and explicitly declares all 
it's anchors -- and our existing link checker code should automatically ensure 
that's never problematic if someone forgets...

http://asciidoctor.org/docs/user-manual/#include-multiple

bq. that some convention / page attributes are used such that the 
BuildNavAndPDFBody code doesn't try to include them as "real" pages (or errors 
that they have no parent)

FYI: I looked into this and confirmed that BuildNavAndPDFBody (currently) only 
looks at "top level" {{src/*.adoc}} files, nothing in subdirectories -- so we 
could easily start organizing "snippet" type files in whatever subdirs we want 
w/o needing any code changes)

> refactor duplicate ref guide content into "snippet" files that can be included
> --
>
> Key: SOLR-10655
> URL: https://issues.apache.org/jira/browse/SOLR-10655
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Hoss Man
>
> in cwiki, we were using the "excerpt" and "excerpt-include" macros to mirror 
> some content across multiple pages.
> as part of the cwiki->adoc migration, these macros were just evaluated 
> durring export, and the content is now duplicated.
> moving forward, we should refactor this duplicated content into "snippet" 
> files that can be included in multiple places.  A few things we need to be 
> careful about when doing this:
> * ensuring anchors and re

[jira] [Comment Edited] (LUCENE-7822) IllegalArgumentException thrown instead of a CorruptIndexException

[Martin Amirault (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011628#comment-16011628
 ] 

Martin Amirault edited comment on LUCENE-7822 at 5/16/17 2:12 AM:
--

Created a patch for master branch, along with a test to reproduce the problem.

Actually the code doing the checksum was here, just not called in a finally 
clause.


was (Author: marumarutan):
Created a patch for master branch, along with a test to reproduce the problem.

Actually the code for doing the checksum was here, just not called in a finally 
clause.

> IllegalArgumentException thrown instead of a CorruptIndexException
> --
>
> Key: LUCENE-7822
> URL: https://issues.apache.org/jira/browse/LUCENE-7822
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.5.1
>Reporter: Martin Amirault
>Priority: Minor
> Attachments: LUCENE-7822.patch
>
>
> Similarly to LUCENE-7592 , When an {{*.si}} file is corrupted on very 
> specific part an IllegalArgumentException is thrown instead of a 
> CorruptIndexException.
> StackTrace (Lucene 6.5.1):
> {code}
> java.lang.IllegalArgumentException: Illegal minor version: 12517381
>   at 
> __randomizedtesting.SeedInfo.seed([1FEB5987CFA44BE:B8755B5574F9F3BF]:0)
>   at org.apache.lucene.util.Version.(Version.java:385)
>   at org.apache.lucene.util.Version.(Version.java:371)
>   at org.apache.lucene.util.Version.fromBits(Version.java:353)
>   at 
> org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:97)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}
> Simple fix would be to add IllegalArgumentException to the catch list at 
> {{org/apache/lucene/index/SegmentInfos.java:289}}
> Other variations for the stacktraces:
> {code}
> java.lang.IllegalArgumentException: invalid codec filename '_�.cfs', must 
> match: _[a-z0-9]+(_.*)?\..*
>   at 
> __randomizedtesting.SeedInfo.seed([8B3FDE317B8D634A:A8EE07E5EB4B0B13]:0)
>   at 
> org.apache.lucene.index.SegmentInfo.checkFileNames(SegmentInfo.java:270)
>   at org.apache.lucene.index.SegmentInfo.addFiles(SegmentInfo.java:252)
>   at org.apache.lucene.index.SegmentInfo.setFiles(SegmentInfo.java:246)
>   at 
> org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:248)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}
> {code}
> java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.Codec with name 'LucenI62' does not exist.  You need 
> to add the corresponding JAR file supporting this SPI to your classpath.  The 
> current classpath supports the following names: [Lucene62, Lucene50, 
> Lucene53, Lucene54, Lucene60]
>   at 
> __randomizedtesting.SeedInfo.seed([925DE160F7260F99:B026EB9373CB6368]:0)
>   at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
>   at org.apache.lucene.codecs.Codec.forName(Codec.java:116)
>   at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:424)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:356)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(S

[jira] [Updated] (LUCENE-7822) IllegalArgumentException thrown instead of a CorruptIndexException

[Martin Amirault (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martin Amirault updated LUCENE-7822:

Attachment: LUCENE-7822.patch

Created a patch for master branch, along with a test to reproduce the problem.

Actually the code for doing the checksum was here, just not called in a finally 
clause.

> IllegalArgumentException thrown instead of a CorruptIndexException
> --
>
> Key: LUCENE-7822
> URL: https://issues.apache.org/jira/browse/LUCENE-7822
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.5.1
>Reporter: Martin Amirault
>Priority: Minor
> Attachments: LUCENE-7822.patch
>
>
> Similarly to LUCENE-7592 , When an {{*.si}} file is corrupted on very 
> specific part an IllegalArgumentException is thrown instead of a 
> CorruptIndexException.
> StackTrace (Lucene 6.5.1):
> {code}
> java.lang.IllegalArgumentException: Illegal minor version: 12517381
>   at 
> __randomizedtesting.SeedInfo.seed([1FEB5987CFA44BE:B8755B5574F9F3BF]:0)
>   at org.apache.lucene.util.Version.(Version.java:385)
>   at org.apache.lucene.util.Version.(Version.java:371)
>   at org.apache.lucene.util.Version.fromBits(Version.java:353)
>   at 
> org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:97)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}
> Simple fix would be to add IllegalArgumentException to the catch list at 
> {{org/apache/lucene/index/SegmentInfos.java:289}}
> Other variations for the stacktraces:
> {code}
> java.lang.IllegalArgumentException: invalid codec filename '_�.cfs', must 
> match: _[a-z0-9]+(_.*)?\..*
>   at 
> __randomizedtesting.SeedInfo.seed([8B3FDE317B8D634A:A8EE07E5EB4B0B13]:0)
>   at 
> org.apache.lucene.index.SegmentInfo.checkFileNames(SegmentInfo.java:270)
>   at org.apache.lucene.index.SegmentInfo.addFiles(SegmentInfo.java:252)
>   at org.apache.lucene.index.SegmentInfo.setFiles(SegmentInfo.java:246)
>   at 
> org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:248)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:260)
> {code}
> {code}
> java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.Codec with name 'LucenI62' does not exist.  You need 
> to add the corresponding JAR file supporting this SPI to your classpath.  The 
> current classpath supports the following names: [Lucene62, Lucene50, 
> Lucene53, Lucene54, Lucene60]
>   at 
> __randomizedtesting.SeedInfo.seed([925DE160F7260F99:B026EB9373CB6368]:0)
>   at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
>   at org.apache.lucene.codecs.Codec.forName(Codec.java:116)
>   at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:424)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:356)
>   at 
> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:448)
>   at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:445)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644)
>   at 
> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:450)
>   at 
> org.apache.lucene.index.DirectoryReader.list

Re: Release 6.6

[Ishan Chattopadhyaya]

I attempted to build an RC, but failed repeatedly before realizing it was
due to, apart from test failures, LUCENE-7830 / LUCENE-6438. I've manually
cleared up the dead symlinks now. I'll build the RC on Tuesday, as I'm
about to wrap up the day's work.

On Mon, May 15, 2017 at 10:12 PM, Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I wish to backport a fix from SOLR-8440 (last commit) to the release
> branch. It affects only the feature introduced in SOLR-8440. Please let me
> know if someone has any objections.
>
> Also, I'm planning to build the RC in another 3-4 hours. Please let me
> know if there's something that someone is working on which needs to get in
> before that.
>
> Thanks and regards,
> Ishan
>
>
> On Sun, May 14, 2017 at 5:02 PM, Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> Sure Steve! Thanks!
>>
>> On Sun, May 14, 2017 at 2:34 PM, Steve Rowe  wrote:
>>
>>> Ishan,
>>>
>>> Okay to include https://issues.apache.org/jira/browse/LUCENE-7821 for
>>> 6.6?
>>>
>>> --
>>> Steve
>>> www.lucidworks.com
>>>
>>> > On May 12, 2017, at 12:51 PM, jim ferenczi 
>>> wrote:
>>> >
>>> > Ok thanks Ishan.
>>> >
>>> > Le 12 mai 2017 18:44, "Ishan Chattopadhyaya" <
>>> ichattopadhy...@gmail.com> a écrit :
>>> > Sure, Jim. Please go ahead!
>>> >
>>> > On Fri, May 12, 2017 at 10:01 PM, jim ferenczi 
>>> wrote:
>>> > Hi,
>>> > Would be great to have https://issues.apache.org/jira
>>> /browse/LUCENE-7824 included for 6.6. Can I merge the fix this week end
>>> Ishan ?
>>> >
>>> > 2017-05-11 16:45 GMT+02:00 Ishan Chattopadhyaya <
>>> ichattopadhy...@gmail.com>:
>>> > Done, Adrien. Thanks!
>>> >
>>> > On Thu, May 11, 2017 at 7:34 PM, Adrien Grand 
>>> wrote:
>>> > Ishan, wdyt about running addVersion on the branch_6x/master as well?
>>> I think it would help realize that the 6.6 branch has been cut when looking
>>> at the CHANGES.txt file and not forget about backporting?
>>> >
>>> > Le mar. 9 mai 2017 à 17:58, Ishan Chattopadhyaya <
>>> ichattopadhy...@gmail.com> a écrit :
>>> > I've cut the branch for 6.6. (branch_6_6). Please feel free to add
>>> bugfixes to the branch, if needed.
>>> > Planning to build the first RC on 15 May. Please let me know if there
>>> are any objections.
>>> >
>>> > Thanks,
>>> > Ishan
>>> >
>>> > On Tue, May 9, 2017 at 11:10 AM, Ishan Chattopadhyaya <
>>> ichattopadhy...@gmail.com> wrote:
>>> > Planning to cut the release branch in another 10-12 hours.
>>> >
>>> > On Mon, May 1, 2017 at 4:00 PM, Martin Gainty 
>>> wrote:
>>> > i was wondering if there was a specific test for SpanPayloadCheckQuery
>>> method
>>> >
>>> > matches = payloadToMatch.get(upto).bytesEquals(payload);
>>> >
>>> >
>>> >
>>> > the only implementation i could locate was in collectLeaf of
>>> SpanPayloadCheckQuery
>>> >
>>> >
>>> > I will submit JIRA with Patch
>>> >
>>> >
>>> > as a CS *dinosaur* I am more familiar with LISP for dissecting
>>> sentence fragments (what we called phenomes) than current SEO
>>> implementations
>>> >
>>> >
>>> > Can you suggest a book to read to better understand Lucenes pattern
>>> dissection and match algorithms?
>>> >
>>> >
>>> > Many Thanks for helpful guidance
>>> > Martin
>>> > __
>>> >
>>> >
>>> >
>>> > From: Erik Hatcher 
>>> > Sent: Sunday, April 30, 2017 2:06 PM
>>> >
>>> > To: dev@lucene.apache.org
>>> > Subject: Re: Release 6.6
>>> >
>>> > Martin -
>>> >
>>> > I have to admit to still being unsure what the gist is here - is there
>>> a bug?   Apologies for not catching what you’re saying/showing here.
>>> Again, as far as I can tell SpanPayloadCheckQuery is working as expected
>>> now.
>>> >
>>> > I’m going to focus purely on SOLR-1485 this week to get it committed
>>> for 6.6.  If there is an issue to address with your work would you please
>>> open a JIRA and include your patch there?
>>> >
>>> > Thanks,
>>> > Erik
>>> >
>>> >
>>> >> On Apr 30, 2017, at 7:47 AM, Martin Gainty 
>>> wrote:
>>> >>
>>> >> Mornin' Erik
>>> >>
>>> >> there is a collectLeaf  override in 
>>> >> org.apache.lucene.queries.payloads.TestPayloadSpans
>>> ..but its never called:
>>> >>
>>> >> static class VerifyingCollector implements SpanCollector {
>>> >> List payloads = new ArrayList<>();
>>> >> @Override
>>> >> public void collectLeaf(PostingsEnum postings, int position, Term
>>> term) throws IOException {
>>> >>  
>>> >>  }
>>> >> }
>>> >>
>>> >> the modification in 
>>> >> org.apache.lucene.queries.payloads.TestPayloadCheckQuery
>>> tests collectLeaf for query
>>> >>
>>> >> //initialise term
>>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 231
>>> before term1=new org.apache.lucene.index.Term('field','withPayload')");
>>> >> org.apache.lucene.index.Term term1=new 
>>> >> org.apache.lucene.index.Term("field",
>>> "withPayload");
>>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 233
>>> term1="+term1);
>>> >>
>>> >> //assume position is 5
>>> >> int position=5;
>>> 

[jira] [Commented] (LUCENE-7830) "ant clean-jars" doesn't remove symlinks that point to non existent files

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011616#comment-16011616
 ] 

Ishan Chattopadhyaya commented on LUCENE-7830:
--

Indeed, I had copied the working tree from another machine which caused the 
stale symlinks.

> "ant clean-jars" doesn't remove symlinks that point to non existent files
> -
>
> Key: LUCENE-7830
> URL: https://issues.apache.org/jira/browse/LUCENE-7830
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Hoss Man
>
> Discovered this helping ishan troubleshoot weird failures he was getting on 
> his machine today.
> Steps to reproduce...
> {code}
> $ touch /tmp/foo.jar
> $ touch /tmp/bar.jar
> $ ln -s /tmp/foo.jar solr/core/lib/
> $ ln -s /tmp/bar.jar solr/core/lib/
> $ ls solr/core/lib/
> $ ls -l solr/core/lib/
> 18:10 bar.jar -> /tmp/bar.jar
> 18:10 foo.jar -> /tmp/foo.jar
> $ rm /tmp/bar.jar
> $ ant clean-jars
> Buildfile: /home/hossman/lucene/dev/build.xml
> clean-jars:
> BUILD SUCCESSFUL
> Total time: 0 seconds
> $ ls -l solr/core/lib/
> total 0
> lrwxrwxrwx 1 hossman hossman 12 May 15 18:10 bar.jar -> /tmp/bar.jar
> {code}
> The situation that (i speculate) led to ishan's problems was (smething like) 
> ...
> # at some point does a build that resolves all jars, ivy creates symlinks to 
> it's cache
> # at some later point, either the ivy cache (or a subset of jars in it) gets 
> blown away (possibly intentionally to save disk space) or the working dir is 
> copied from one machine to another
> # the version of the dependency gets updated and/or the branch is changed so 
> a diff version of some jar is needed - so "ant resolve" & all compilation and 
> testing are happy even though there is a symlink to a file that isn't in the 
> ivy cache anymore.
> # check-licenses will fail however, because when it attempts to resolve the 
> list of all jar resources found by ant, the underlying file doesn't actaully 
> exist, so the build fails with an error about an "old" jar that you don't 
> expect to be used anyway...{code}
> // from LicenseCheckTask.java
>   if (!r.isExists()) { 
> throw new BuildException("JAR resource does not exist: " + 
> r.getName());
>   }
> {code}
> # {{ant clean-jars}} does nothing to fix the underlying problem



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Vivek Narang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011599#comment-16011599
 ] 

Vivek Narang commented on SOLR-10317:
-

Thanks [~arafalov] :)


> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: changes-lucene-20160907.json, 
> changes-solr-20160907.json, managed-schema, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf, 
> solrconfig.xml
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7830) "ant clean-jars" doesn't remove symlinks that point to non existent files

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011592#comment-16011592
 ] 

Steve Rowe commented on LUCENE-7830:


I think this is a duplicate of LUCENE-6438 - I put a patch up there a ways back.

> "ant clean-jars" doesn't remove symlinks that point to non existent files
> -
>
> Key: LUCENE-7830
> URL: https://issues.apache.org/jira/browse/LUCENE-7830
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Hoss Man
>
> Discovered this helping ishan troubleshoot weird failures he was getting on 
> his machine today.
> Steps to reproduce...
> {code}
> $ touch /tmp/foo.jar
> $ touch /tmp/bar.jar
> $ ln -s /tmp/foo.jar solr/core/lib/
> $ ln -s /tmp/bar.jar solr/core/lib/
> $ ls solr/core/lib/
> $ ls -l solr/core/lib/
> 18:10 bar.jar -> /tmp/bar.jar
> 18:10 foo.jar -> /tmp/foo.jar
> $ rm /tmp/bar.jar
> $ ant clean-jars
> Buildfile: /home/hossman/lucene/dev/build.xml
> clean-jars:
> BUILD SUCCESSFUL
> Total time: 0 seconds
> $ ls -l solr/core/lib/
> total 0
> lrwxrwxrwx 1 hossman hossman 12 May 15 18:10 bar.jar -> /tmp/bar.jar
> {code}
> The situation that (i speculate) led to ishan's problems was (smething like) 
> ...
> # at some point does a build that resolves all jars, ivy creates symlinks to 
> it's cache
> # at some later point, either the ivy cache (or a subset of jars in it) gets 
> blown away (possibly intentionally to save disk space) or the working dir is 
> copied from one machine to another
> # the version of the dependency gets updated and/or the branch is changed so 
> a diff version of some jar is needed - so "ant resolve" & all compilation and 
> testing are happy even though there is a symlink to a file that isn't in the 
> ivy cache anymore.
> # check-licenses will fail however, because when it attempts to resolve the 
> list of all jar resources found by ant, the underlying file doesn't actaully 
> exist, so the build fails with an error about an "old" jar that you don't 
> expect to be used anyway...{code}
> // from LicenseCheckTask.java
>   if (!r.isExists()) { 
> throw new BuildException("JAR resource does not exist: " + 
> r.getName());
>   }
> {code}
> # {{ant clean-jars}} does nothing to fix the underlying problem



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7830) "ant clean-jars" doesn't remove symlinks that point to non existent files

[Hoss Man (JIRA)]

Hoss Man created LUCENE-7830:


 Summary: "ant clean-jars" doesn't remove symlinks that point to 
non existent files
 Key: LUCENE-7830
 URL: https://issues.apache.org/jira/browse/LUCENE-7830
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Hoss Man


Discovered this helping ishan troubleshoot weird failures he was getting on his 
machine today.

Steps to reproduce...
{code}
$ touch /tmp/foo.jar
$ touch /tmp/bar.jar
$ ln -s /tmp/foo.jar solr/core/lib/
$ ln -s /tmp/bar.jar solr/core/lib/
$ ls solr/core/lib/

$ ls -l solr/core/lib/

18:10 bar.jar -> /tmp/bar.jar
18:10 foo.jar -> /tmp/foo.jar
$ rm /tmp/bar.jar
$ ant clean-jars
Buildfile: /home/hossman/lucene/dev/build.xml

clean-jars:

BUILD SUCCESSFUL
Total time: 0 seconds
$ ls -l solr/core/lib/
total 0
lrwxrwxrwx 1 hossman hossman 12 May 15 18:10 bar.jar -> /tmp/bar.jar
{code}

The situation that (i speculate) led to ishan's problems was (smething like) ...

# at some point does a build that resolves all jars, ivy creates symlinks to 
it's cache
# at some later point, either the ivy cache (or a subset of jars in it) gets 
blown away (possibly intentionally to save disk space) or the working dir is 
copied from one machine to another
# the version of the dependency gets updated and/or the branch is changed so a 
diff version of some jar is needed - so "ant resolve" & all compilation and 
testing are happy even though there is a symlink to a file that isn't in the 
ivy cache anymore.
# check-licenses will fail however, because when it attempts to resolve the 
list of all jar resources found by ant, the underlying file doesn't actaully 
exist, so the build fails with an error about an "old" jar that you don't 
expect to be used anyway...{code}
// from LicenseCheckTask.java
  if (!r.isExists()) { 
throw new BuildException("JAR resource does not exist: " + r.getName());
  }
{code}
# {{ant clean-jars}} does nothing to fix the underlying problem



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1301 - Still unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1301/

1 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1494891787746,generation=2,filelist=[_9i.cfe, _9i.cfs, 
_9i.si, _9n.fdt, _9n.fdx, _9n.fnm, _9n.nvd, _9n.nvm, _9n.si, 
_9n_TestBloomFilteredLucenePostings_0.blm, 
_9n_TestBloomFilteredLucenePostings_0.doc, 
_9n_TestBloomFilteredLucenePostings_0.tim, 
_9n_TestBloomFilteredLucenePostings_0.tip, _9p.cfe, _9p.cfs, _9p.si, _9t.cfe, 
_9t.cfs, _9t.si, _9w.cfe, _9w.cfs, _9w.si, _9x.cfe, _9x.cfs, _9x.si, _a0.cfe, 
_a0.cfs, _a0.si, _a1.cfe, _a1.cfs, _a1.si, _a2.cfe, _a2.cfs, _a2.si, _a3.cfe, 
_a3.cfs, _a3.si, _a4.cfe, _a4.cfs, _a4.si, _a5.cfe, _a5.cfs, _a5.si, _a6.cfe, 
_a6.cfs, _a6.si, _a7.cfe, _a7.cfs, _a7.si, _a8.cfe, _a8.cfs, _a8.si, _a9.cfe, 
_a9.cfs, _a9.si, _aa.cfe, _aa.cfs, _aa.si, _ab.cfe, _ab.cfs, _ab.si, _ac.cfe, 
_ac.cfs, _ac.si, _ad.cfe, _ad.cfs, _ad.si, _ae.cfe, _ae.cfs, _ae.si, _af.cfe, 
_af.cfs, _af.si, _ag.cfe, _ag.cfs, _ag.si, _ah.cfe, _ah.cfs, _ah.si, _ai.cfe, 
_ai.cfs, _ai.si, _aj.cfe, _aj.cfs, _aj.si, _ak.cfe, _ak.cfs, _ak.si, 
segments_2]}]> but 
was:<[{indexVersion=1494891787746,generation=2,filelist=[_9i.cfe, _9i.cfs, 
_9i.si, _9n.fdt, _9n.fdx, _9n.fnm, _9n.nvd, _9n.nvm, _9n.si, 
_9n_TestBloomFilteredLucenePostings_0.blm, 
_9n_TestBloomFilteredLucenePostings_0.doc, 
_9n_TestBloomFilteredLucenePostings_0.tim, 
_9n_TestBloomFilteredLucenePostings_0.tip, _9p.cfe, _9p.cfs, _9p.si, _9t.cfe, 
_9t.cfs, _9t.si, _9w.cfe, _9w.cfs, _9w.si, _9x.cfe, _9x.cfs, _9x.si, _a0.cfe, 
_a0.cfs, _a0.si, _a1.cfe, _a1.cfs, _a1.si, _a2.cfe, _a2.cfs, _a2.si, _a3.cfe, 
_a3.cfs, _a3.si, _a4.cfe, _a4.cfs, _a4.si, _a5.cfe, _a5.cfs, _a5.si, _a6.cfe, 
_a6.cfs, _a6.si, _a7.cfe, _a7.cfs, _a7.si, _a8.cfe, _a8.cfs, _a8.si, _a9.cfe, 
_a9.cfs, _a9.si, _aa.cfe, _aa.cfs, _aa.si, _ab.cfe, _ab.cfs, _ab.si, _ac.cfe, 
_ac.cfs, _ac.si, _ad.cfe, _ad.cfs, _ad.si, _ae.cfe, _ae.cfs, _ae.si, _af.cfe, 
_af.cfs, _af.si, _ag.cfe, _ag.cfs, _ag.si, _ah.cfe, _ah.cfs, _ah.si, _ai.cfe, 
_ai.cfs, _ai.si, _aj.cfe, _aj.cfs, _aj.si, _ak.cfe, _ak.cfs, _ak.si, 
segments_2]}, {indexVersion=1494891787746,generation=3,filelist=[_9i.cfe, 
_9i.cfs, _9i.si, _9p.cfe, _9p.cfs, _9p.si, _9t.cfe, _9t.cfs, _9t.si, _9w.cfe, 
_9w.cfs, _9w.si, _9x.cfe, _9x.cfs, _9x.si, _a0.cfe, _a0.cfs, _a0.si, _a1.cfe, 
_a1.cfs, _a1.si, _a3.cfe, _a3.cfs, _a3.si, _a4.cfe, _a4.cfs, _a4.si, _a5.cfe, 
_a5.cfs, _a5.si, _a6.cfe, _a6.cfs, _a6.si, _a7.cfe, _a7.cfs, _a7.si, _a8.cfe, 
_a8.cfs, _a8.si, _a9.cfe, _a9.cfs, _a9.si, _aa.cfe, _aa.cfs, _aa.si, _ab.cfe, 
_ab.cfs, _ab.si, _ac.cfe, _ac.cfs, _ac.si, _ad.cfe, _ad.cfs, _ad.si, _ag.cfe, 
_ag.cfs, _ag.si, _ah.cfe, _ah.cfs, _ah.si, _ai.cfe, _ai.cfs, _ai.si, _ak.cfe, 
_ak.cfs, _ak.si, _al.cfe, _al.cfs, _al.si, segments_3]}]>

Stack Trace:
java.lang.AssertionError: 
expected:<[{indexVersion=1494891787746,generation=2,filelist=[_9i.cfe, _9i.cfs, 
_9i.si, _9n.fdt, _9n.fdx, _9n.fnm, _9n.nvd, _9n.nvm, _9n.si, 
_9n_TestBloomFilteredLucenePostings_0.blm, 
_9n_TestBloomFilteredLucenePostings_0.doc, 
_9n_TestBloomFilteredLucenePostings_0.tim, 
_9n_TestBloomFilteredLucenePostings_0.tip, _9p.cfe, _9p.cfs, _9p.si, _9t.cfe, 
_9t.cfs, _9t.si, _9w.cfe, _9w.cfs, _9w.si, _9x.cfe, _9x.cfs, _9x.si, _a0.cfe, 
_a0.cfs, _a0.si, _a1.cfe, _a1.cfs, _a1.si, _a2.cfe, _a2.cfs, _a2.si, _a3.cfe, 
_a3.cfs, _a3.si, _a4.cfe, _a4.cfs, _a4.si, _a5.cfe, _a5.cfs, _a5.si, _a6.cfe, 
_a6.cfs, _a6.si, _a7.cfe, _a7.cfs, _a7.si, _a8.cfe, _a8.cfs, _a8.si, _a9.cfe, 
_a9.cfs, _a9.si, _aa.cfe, _aa.cfs, _aa.si, _ab.cfe, _ab.cfs, _ab.si, _ac.cfe, 
_ac.cfs, _ac.si, _ad.cfe, _ad.cfs, _ad.si, _ae.cfe, _ae.cfs, _ae.si, _af.cfe, 
_af.cfs, _af.si, _ag.cfe, _ag.cfs, _ag.si, _ah.cfe, _ah.cfs, _ah.si, _ai.cfe, 
_ai.cfs, _ai.si, _aj.cfe, _aj.cfs, _aj.si, _ak.cfe, _ak.cfs, _ak.si, 
segments_2]}]> but 
was:<[{indexVersion=1494891787746,generation=2,filelist=[_9i.cfe, _9i.cfs, 
_9i.si, _9n.fdt, _9n.fdx, _9n.fnm, _9n.nvd, _9n.nvm, _9n.si, 
_9n_TestBloomFilteredLucenePostings_0.blm, 
_9n_TestBloomFilteredLucenePostings_0.doc, 
_9n_TestBloomFilteredLucenePostings_0.tim, 
_9n_TestBloomFilteredLucenePostings_0.tip, _9p.cfe, _9p.cfs, _9p.si, _9t.cfe, 
_9t.cfs, _9t.si, _9w.cfe, _9w.cfs, _9w.si, _9x.cfe, _9x.cfs, _9x.si, _a0.cfe, 
_a0.cfs, _a0.si, _a1.cfe, _a1.cfs, _a1.si, _a2.cfe, _a2.cfs, _a2.si, _a3.cfe, 
_a3.cfs, _a3.si, _a4.cfe, _a4.cfs, _a4.si, _a5.cfe, _a5.cfs, _a5.si, _a6.cfe, 
_a6.cfs, _a6.si, _a7.cfe, _a7.cfs, _a7.si, _a8.cfe, _a8.cfs, _a8.si, _a9.cfe, 
_a9.cfs, _a9.si, _aa.cfe, _aa.cfs, _aa.si, _ab.cfe, _ab.cfs, _ab.si, _ac.cfe, 
_ac.cfs, _ac.si, _ad.cfe, _ad.cfs, _ad.si, _ae.cfe, _ae.cfs, _ae.si, _af.cfe, 
_af.cfs, _af.si, _ag.cfe, _ag.cfs, _ag.si, _ah.cfe, _ah.cfs, _ah.si, _ai.cfe, 
_ai.cfs, _ai.si, _aj.cfe, _aj.cfs, _aj.si, _ak.cfe, _ak.cfs, _ak.si, 
segments_2]}, {indexVersion=1494891787746,generation=3,filelist=[_9i.cfe, 
_9i.cfs, _9i.si, _9p.cfe, _9p.cfs, _9p.si, _9t.cfe, _9t.cfs, _9

[jira] [Updated] (SOLR-10317) Solr Nightly Benchmarks

[Alexandre Rafalovitch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandre Rafalovitch updated SOLR-10317:
-
Attachment: managed-schema
solrconfig.xml
changes-solr-20160907.json
changes-lucene-20160907.json

Attached are my experiment's outputs for Lucene and Solr's CHANGES.txt 
converted into nested Solr-ready input format. managed-schema and 
solrconfig.xml are also attached.

It's been a while since I retested this, but I can have another look if it is 
useful.

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: changes-lucene-20160907.json, 
> changes-solr-20160907.json, managed-schema, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf, 
> solrconfig.xml
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10693) Add copyOfRange Stream Evaluator

[Joel Bernstein (JIRA)]

Joel Bernstein created SOLR-10693:
-

 Summary: Add copyOfRange Stream Evaluator
 Key: SOLR-10693
 URL: https://issues.apache.org/jira/browse/SOLR-10693
 Project: Solr
  Issue Type: New Feature
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Joel Bernstein


The copyOfRange Stream Evaluator copies a range of an array to a new array.

Syntax:
{code}
a = copyOfRange(colA, 1, 4)
{code}





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10612) make ':toclevels:' work in our jekyll templates

[Cassandra Targett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011367#comment-16011367
 ] 

Cassandra Targett commented on SOLR-10612:
--

I just spent a little bit of time trying to make the native TOC in the 
jekyll-asciidoc gem work, and I believe based on what I've read about it that 
only two positions for the TOC are supported: preamble (at the top of the page) 
and macro (explicitly positioned inline with the page). Any right or left side 
placement won't work.

See:
https://github.com/asciidoctor/jekyll-asciidoc#generating-a-table-of-contents
https://github.com/asciidoctor/jekyll-asciidoc/issues/135
http://discuss.asciidoctor.org/Jekyll-Plugin-toc-Problem-td4070.html

But, maybe there is some stuff we could do with JS and/or the liquid template 
with a new param and/or CSS. I'm not throwing away the idea, just one of the 
obvious ideas probably isn't workable.

> make ':toclevels:' work in our jekyll templates
> ---
>
> Key: SOLR-10612
> URL: https://issues.apache.org/jira/browse/SOLR-10612
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Hoss Man
>Assignee: Hoss Man
> Attachments: lang-analysis-1-level-toc.png
>
>
> asciidoc has a concept called {{:toclevels:}} which is suppose to determine 
> which how deep down a page's section/header depth the generated table of 
> contents will show -- ie: some LONG pages have a huge number of level 2 
> headings, and on those pages we only want to show level 1.
> but in jekyll, asciidoctor isn't responsible for generating the TOC -- 
> instead it's done by some javascript (which is better for a variety of 
> reasons) and at the moment this javascript doesn't know anything about 
> {{:toclevels:}}
> But it should be possible to tweak our rendering templates to include 
> {{:toclevels:}} as an attribute in the generated HTML, and then we can tweak 
> the javascript call made to generate the TOC so that it respects it on a 
> per-page basis



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7829) make "Apache TLP Website Link Checks" all-green

[Christine Poerschke (JIRA)]

Christine Poerschke created LUCENE-7829:
---

 Summary: make "Apache TLP Website Link Checks" all-green
 Key: LUCENE-7829
 URL: https://issues.apache.org/jira/browse/LUCENE-7829
 Project: Lucene - Core
  Issue Type: Task
Reporter: Christine Poerschke
Priority: Minor


See https://whimsy.apache.org/site/project/lucene for details.
* Events - red (???)
* License - yellow (drop the LICENSE-2.0)
* Sponsorship - red (instead of "Become a Sponsor" the link should read 
"Sponsorship" or "Donate" looks like)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10691) Allow to not commit index on core close

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011353#comment-16011353
 ] 

Erick Erickson commented on SOLR-10691:
---

Could you explain your use-case for this functionality? On the surface, it 
seems very trappy. With any kind of autocommit happening, you really don't have 
a good way to know what docs are already committed. In SolrCloud there's no 
guarantee that docs that have been committed on one replica are also committed 
on another. Not to mention transaction logs, so just not calling commit has to 
deal with rolling back the tlogs as well or the docs will be re-indexed when 
the node starts up.

99.9% of the time, configuring this would cause unexpected behavior. SolrCloud 
in particular tries very hard to guarantee that any doc sent to SolrCloud is 
there when the system restarts for whatever reason. If implemented such that 
documents were lost, this kind of change would cause that contract to be 
voilated.

This seems like it's too little benefit for too much risk unless there's a 
compelling use-case, other than "it'd be a little more efficient".

Plus, the tests do not verify the commit _behavior_, just that the config was 
properly read. My claim is that if there were tests adding a bunch of docs such 
that the commits happened at unpredictable intervals, say with some kind of 
commitWithin some random number between 250 and 1,000 ms it would be difficult 
to write a reliable test showing that predictable docs were thrown away.

Or am I completely misunderstanding the point?

> Allow to not commit index on core close
> ---
>
> Key: SOLR-10691
> URL: https://issues.apache.org/jira/browse/SOLR-10691
> Project: Solr
>  Issue Type: Improvement
>Reporter: Ivan Mamontov
>Priority: Trivial
> Attachments: SOLR-10691.patch
>
>
> As a Solr user I would like to avoid unnecessary commits into Solr/Lucene 
> index on {{org.apache.solr.update.SolrIndexWriter#close}} in case IW has 
> uncommitted changes.
> In {{org.apache.lucene.index.IndexWriterConfig}}(LUCENE-5871) there is a 
> property which is currently used to decide whether to commit or discard 
> uncommitted changes  when you call close(). Unfortunately Solr does not 
> support this property in {{org.apache.solr.update.SolrIndexConfig}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Lucene-3.0.0 God Class Detection Report and Feedback Questionnaire

[Khalid AlKharabsheh]

Dear Respondent,


This questionnaire is part of my PhD thesis in the area of improving design
smell detection. The purpose of the study is to measure the concordance
level of human experts against the design smell detection tools.
We analyzed the Lucene-3.0.0 project and detected the set of God Class
(Large Class or The Blob) that you can find in this questionnaire.
Completing this questionnaire should take about 15 minutes.

I would appreciate your kind attention and cooperation on sending your
questionnaire feedback as soon as possible. Notice that all information
related to respondent or contained in this questionnaire will be treated in
strict confidence.

If you are not currently involved in the project, please forward this
message to developers you think are a better target for this questionnaire

To fill the questionnire click on:

https://docs.google.com/forms/d/e/1FAIpQLSeXqH8oI_SEmgL3oDsd0G-boCgJqv-Q75e63hF_KS7chi4G2A/viewform?c=0&w=1



Thank you in advance.
Yours sincerely,
Khalid Alkharabsheh
PhD Student at CITIUS, Santiago de Compostela University (Spain).
https://citius.usc.es/equipo/investigadores-en-formacion/khalid-alkharabsheh


-- 

Kha lid Alkharabsheh
LP2 [image: E-mail:] khalid.alkharabs...@usc.es 
[image: Phone:] +34 881816474
[image: Website:] citius.usc.es  · [image: Twitter:]
 citiususc 

[jira] [Assigned] (SOLR-8762) DIH entity child=true should respond nested documents on debug

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev reassigned SOLR-8762:
--

Assignee: Mikhail Khludnev

> DIH entity child=true should respond nested documents on debug
> --
>
> Key: SOLR-8762
> URL: https://issues.apache.org/jira/browse/SOLR-8762
> Project: Solr
>  Issue Type: Improvement
>  Components: contrib - DataImportHandler
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Minor
>  Labels: newbie, newdev
> Attachments: SOLR-8762.patch, SOLR-8762.patch
>
>
> Problem is described in 
> [comment|https://issues.apache.org/jira/browse/SOLR-5147?focusedCommentId=14744852&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14744852]
>  of SOLR-5147 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-3702) String concatenation function

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev reassigned SOLR-3702:
--

Assignee: Mikhail Khludnev  (was: Shalin Shekhar Mangar)

> String concatenation function
> -
>
> Key: SOLR-3702
> URL: https://issues.apache.org/jira/browse/SOLR-3702
> Project: Solr
>  Issue Type: New Feature
>  Components: query parsers
>Affects Versions: 4.0-ALPHA
>Reporter: Ted Strauss
>Assignee: Mikhail Khludnev
> Fix For: 4.9, 6.0
>
> Attachments: SOLR-3702.patch, SOLR-3702.patch, SOLR-3702.patch, 
> SOLR-3702.patch, SOLR-3702.patch
>
>
> Related to https://issues.apache.org/jira/browse/SOLR-2526
> Add query function to support concatenation of Strings.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011210#comment-16011210
 ] 

Alexandre Rafalovitch commented on SOLR-10317:
--

I've already mentioned my https://github.com/arafalov/git-to-solr in this JIRA 
back in March. It has *full* access to GIT information and produces nested 
records, so I think it might be better than the GIT-to-JSON route. Or at least 
worth comparing.

I also did parsing of the CHANGES file, which was ugly and hacky, but did 
produce a nested dataset to play with. It is not publicly available now, but I 
could look for it in backups to see if that's useful.

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 4 - Still unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/4/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexSorting

Error Message:
Suite timeout exceeded (>= 720 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 720 msec).
at __randomizedtesting.SeedInfo.seed([79DC68A757851F38]:0)


FAILED:  org.apache.solr.cloud.TestRandomRequestDistribution.test

Error Message:
Shard a1x2_shard1_replica1 received all 10 requests

Stack Trace:
java.lang.AssertionError: Shard a1x2_shard1_replica1 received all 10 requests
at 
__randomizedtesting.SeedInfo.seed([FB98FBAB5B60D44:87EDB0601B4A60BC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.TestRandomRequestDistribution.testRequestTracking(TestRandomRequestDistribution.java:121)
at 
org.apache.solr.cloud.TestRandomRequestDistribution.test(TestRandomRequestDistribution.java:65)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.jav

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011200#comment-16011200
 ] 

Ishan Chattopadhyaya commented on SOLR-10317:
-

Vivek, as [~michael.sun] mentioned, 

bq.  2. In report, it's useful to specify the test environment, test parameters 
used in test. These metadata make test result meaningful.

reports like these are meaningless, unless the test conditions are specified 
along with the report. Perhaps every graph should accompany those details?

X-Axis doesn't seem to be commit time, as it should be. Time of metric 
collection is useless.

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10317) Solr Nightly Benchmarks

[Vivek Narang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011172#comment-16011172
 ] 

Vivek Narang edited comment on SOLR-10317 at 5/15/17 7:27 PM:
--

Hi [~ichattopadhyaya], A small additional update. Using this set of documents 
(~25K) and the data structure I am getting the following metrics against recent 
commits [http://162.243.101.83/IndexingBenchmark.html]. Y-Axis is in seconds 
and X-Axis is time. On hover, you will be able to see the commit ID.  


was (Author: vivek.nar...@uga.edu):
Hi [~ichattopadhyaya], A small additional update. Using this set of documents 
(~25K) and the data structure I am getting the following metrics against recent 
commits [http://162.243.101.83/IndexingBenchmark.html]. Y-Axis is in seconds 
this time and X-Axis is time. On hover, you will be able to see the commit ID.  

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Vivek Narang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011172#comment-16011172
 ] 

Vivek Narang commented on SOLR-10317:
-

Hi [~ichattopadhyaya], A small additional update. Using this set of documents 
(~25K) and the data structure I am getting the following metrics against recent 
commits [http://162.243.101.83/IndexingBenchmark.html]. Y-Axis is in seconds 
this time and X-Axis is time. On hover, you will be able to see the commit ID.  

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10317) Solr Nightly Benchmarks

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011152#comment-16011152
 ] 

Ishan Chattopadhyaya edited comment on SOLR-10317 at 5/15/17 7:18 PM:
--

bq. Solr commit history for indexing purposes
Looks great for a start. Can you include the files touched in each commit and 
%age of the each file changed?
Also, if it can be joined with solr/CHANGES.txt and lucene/CHANGES.txt text, 
then contributor info from there can be gathered. That will be more meaningful, 
since one issue is committed by a committer, but various folks collaborate on 
that issue. Also, information as to whether a commit is a bugfix, feature, 
optimization or other change can be obtained from the CHANGES.txt. Also, the 
Solr version in which it was released can be leveraged from there.

[~arafalov], any more thoughts?


was (Author: ichattopadhyaya):
bq. Solr commit history for indexing purposes
Looks great for a start. Can you include the files touched in each commit and 
%age of the each file changed?
Also, if it can be joined with solr/CHANGES.txt and lucene/CHANGES.txt text, 
then contributor info from there can be gathered. That will be more meaningful, 
since one issue is committed by a committer, but various folks collaborate on 
that issue. [~arafalov], any more thoughts?

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011152#comment-16011152
 ] 

Ishan Chattopadhyaya commented on SOLR-10317:
-

bq. Solr commit history for indexing purposes
Looks great for a start. Can you include the files touched in each commit and 
%age of the each file changed?
Also, if it can be joined with solr/CHANGES.txt and lucene/CHANGES.txt text, 
then contributor info from there can be gathered. That will be more meaningful, 
since one issue is committed by a committer, but various folks collaborate on 
that issue. [~arafalov], any more thoughts?

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10317) Solr Nightly Benchmarks

[Vivek Narang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011141#comment-16011141
 ] 

Vivek Narang edited comment on SOLR-10317 at 5/15/17 7:12 PM:
--

Hi [~ichattopadhyaya] I also wanted to share that I was trying to figure a way 
out to get the Solr commit history for indexing purposes. After searching I 
came up with a script 
[https://gist.github.com/viveknarang/141ab289789b0fe55b09409f99d84c75] and with 
this, created a JSON file [http://162.243.101.83/solrcommit.log] having around 
25K documents. Please let me know what you think about the data structure. 
Thanks!

--- Sample Record ---

{
  "commit": "9c6279d439a231d9ec8c9564b0ab76f616d10076",
  "author": "Joel Bernstein",
  "date": "Sun May 14 15:54:32 2017 -0400",
  "message": "SOLR-10663-Add-distance-Stream-Evaluator",
 "author email": "jbern...@apache.org",
 "timestamp": "1494791672",
 "committer name": "Joel Bernstein",
 "committer email": "jbern...@apache.org",
 "commit date": "Mon May 15 11:26:05 2017 -0400"
}


was (Author: vivek.nar...@uga.edu):
Hi [~ichattopadhyaya] I also wanted to share that I was trying to figure a way 
out to get the Solr commit history for indexing purposes. After searching I 
came up with a script 
[https://gist.github.com/viveknarang/141ab289789b0fe55b09409f99d84c75] and with 
this, created a JSON file [http://162.243.101.83/solrcommit.log] having around 
25K documents. Please let me know what you think about the data structure. 
Thanks!

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Vivek Narang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011141#comment-16011141
 ] 

Vivek Narang commented on SOLR-10317:
-

Hi [~ichattopadhyaya] I also wanted to share that I was trying to figure a way 
out to get the Solr commit history for indexing purposes. After searching I 
came up with a script 
[https://gist.github.com/viveknarang/141ab289789b0fe55b09409f99d84c75] and with 
this, created a JSON file [http://162.243.101.83/solrcommit.log] having around 
25K documents. Please let me know what you think about the data structure. 
Thanks!

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10661) Add copyOf Stream Evaluator

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011140#comment-16011140
 ] 

ASF subversion and git services commented on SOLR-10661:


Commit 17210362736c220ff4c12a448b238eed1212a9a8 in lucene-solr's branch 
refs/heads/master from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1721036 ]

SOLR-10661: Add copyOf Stream Evaluator


> Add copyOf Stream Evaluator
> ---
>
> Key: SOLR-10661
> URL: https://issues.apache.org/jira/browse/SOLR-10661
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: master (7.0)
>
> Attachments: SOLR-10661.patch
>
>
> The copyOf Stream Evaluator returns a copy of an array.
> Syntax:
> {code}
> a = copyOf(colA)
> {code}
> {code}
> a = copyOf(colA, length)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10692) Split Streaming Expressions docs into more manageable parts

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1606#comment-1606
 ] 

Joel Bernstein commented on SOLR-10692:
---

+1

I'll start reviewing the tickets around the new documentation so I can get the 
6.6 Streaming Expressions documented.

I'm not sure when you plan to release the 6.6 docs. The Streaming Expressions 
documentation may not finish in time for the official 6.6 docs release. But I 
think that's OK, as I think now the goal is really to work towards a much more 
manageable Streaming Expression documentation format. Once it's more manageable 
we can catch up and expand the scope to include more usage examples. I think 
the lack of usage examples right now makes it hard for people to understand how 
to use all the different functions.

> Split Streaming Expressions docs into more manageable parts
> ---
>
> Key: SOLR-10692
> URL: https://issues.apache.org/jira/browse/SOLR-10692
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
> Fix For: 6.6, master (7.0)
>
>
> The Streaming Expressions page is too big and a mix of How To and Reference 
> materials. It should be split up a bit to make it easier to use.
> First pass at this will split all the expression types into new pages 
> according to the type, so there will be 3 new sub-pages:
> * Stream Sources
> * Stream Decorators
> * Stream Evaluators
> Once that's done, we'll see if there's more we can do with this.
> Attn [~joel.bernstein].



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Vivek Narang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1603#comment-1603
 ] 

Vivek Narang commented on SOLR-10317:
-

[~ichattopadhyaya] Yes! I agree that nanosecond scale is too precise. I will 
convert it to milliseconds. Thanks.

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Michael Sun (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011107#comment-16011107
 ] 

Michael Sun commented on SOLR-10317:


[~vivek.nar...@uga.edu] Good progress.

A couple of suggestions
1. There are nearly 30% difference between max value and min value in the 
report. I noticed they are run against different build but what is the reason 
for high variation.
2. In report, it's useful to specify the test environment, test parameters used 
in test. These metadata make test result meaningful.
3. Memory usage is probably not very important for this test but it would be 
good to think about it in your report. Some test such as faceting can be memory 
intensive.

I strongly agree with [~ichattopadhyaya] that this is a 'community bonding' 
project. One suggestion. A good way to start conversation with community would 
be to start a simply design document to describe the goals, challenges, high 
level design and how your work can address them. You can share it with 
community and get feedbacks.



> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-10661) Add copyOf Stream Evaluator

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein reassigned SOLR-10661:
-

Assignee: Joel Bernstein

> Add copyOf Stream Evaluator
> ---
>
> Key: SOLR-10661
> URL: https://issues.apache.org/jira/browse/SOLR-10661
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: master (7.0)
>
> Attachments: SOLR-10661.patch
>
>
> The copyOf Stream Evaluator returns a copy of an array.
> Syntax:
> {code}
> a = copyOf(colA)
> {code}
> {code}
> a = copyOf(colA, length)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011097#comment-16011097
 ] 

Ishan Chattopadhyaya commented on SOLR-10317:
-

bq. The X axis represents day and time when the metric was recorded. The Y axis 
in this example is the time in nanoseconds that was registered for this 
activity against each commit.
I think it would be better for every metric point to correspond to a commit, 
and the day/time be that of the commit. Nanoseconds feels too granular; for 
collection creation, seconds or milliseconds seems wiser. What do you think?

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10661) Add copyOf Stream Evaluator

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10661:
--
Attachment: SOLR-10661.patch

> Add copyOf Stream Evaluator
> ---
>
> Key: SOLR-10661
> URL: https://issues.apache.org/jira/browse/SOLR-10661
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Fix For: master (7.0)
>
> Attachments: SOLR-10661.patch
>
>
> The copyOf Stream Evaluator returns a copy of an array.
> Syntax:
> {code}
> a = copyOf(colA)
> {code}
> {code}
> a = copyOf(colA, length)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Vivek Narang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011080#comment-16011080
 ] 

Vivek Narang commented on SOLR-10317:
-

Thanks [~ichattopadhyaya]. The X axis represents day and time when the metric 
was recorded. The Y axis in this example is the time in nanoseconds that was 
registered for this activity against each commit. I will provide the link to 
the code soon. As far as community bonding activities are concerned I have been 
reading the Solr documentation and especially coding standards that I must 
adhere to. I have used also #solr-dev and #solr channels to ask few questions. 
I plan to be more active in the community bonding activities this week. 

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10661) Add copyOf Stream Evaluator

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10661:
--
Description: 
The copyOf Stream Evaluator returns a copy of an array.

Syntax:
{code}
a = copyOf(colA)
{code}

{code}
a = copyOf(colA, length)
{code}


  was:
The copyOf Stream Evaluator returns a copy of an array with a specific range.

Syntax:
{code}
a = copyOf(colA, offset, length)
{code}

{code}
a = copyOf(colA, offset)
{code}



> Add copyOf Stream Evaluator
> ---
>
> Key: SOLR-10661
> URL: https://issues.apache.org/jira/browse/SOLR-10661
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Fix For: master (7.0)
>
>
> The copyOf Stream Evaluator returns a copy of an array.
> Syntax:
> {code}
> a = copyOf(colA)
> {code}
> {code}
> a = copyOf(colA, length)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10691) Allow to not commit index on core close

[Andrey Kudryavtsev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011059#comment-16011059
 ] 

Andrey Kudryavtsev commented on SOLR-10691:
---

It looks like {{IndexWriter#shutdown}} is currently used in Solr for.. correct 
core initialisation (for initial segments_* file creation at least) 

Can't just avoid it )

> Allow to not commit index on core close
> ---
>
> Key: SOLR-10691
> URL: https://issues.apache.org/jira/browse/SOLR-10691
> Project: Solr
>  Issue Type: Improvement
>Reporter: Ivan Mamontov
>Priority: Trivial
> Attachments: SOLR-10691.patch
>
>
> As a Solr user I would like to avoid unnecessary commits into Solr/Lucene 
> index on {{org.apache.solr.update.SolrIndexWriter#close}} in case IW has 
> uncommitted changes.
> In {{org.apache.lucene.index.IndexWriterConfig}}(LUCENE-5871) there is a 
> property which is currently used to decide whether to commit or discard 
> uncommitted changes  when you call close(). Unfortunately Solr does not 
> support this property in {{org.apache.solr.update.SolrIndexConfig}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10317) Solr Nightly Benchmarks

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010999#comment-16010999
 ] 

Ishan Chattopadhyaya commented on SOLR-10317:
-

[~viveknarang], looks promising!
# Can you please explain what x and y axes represent?
# Can you please point to the source code?
# Also, can you please provide an update regarding the "community bonding" 
activities that you have taken up? As per the GSoC manual, this is the time 
when you engage with the community. You should participate in the discussions 
and issues, fix bugs, add tests and documentation, review code, help users 
(over IRC, mailing lists) etc. I may not have emphasized it enough, but this 
"community bonding period" is very important in order to make this GSoC project 
a success.

> Solr Nightly Benchmarks
> ---
>
> Key: SOLR-10317
> URL: https://issues.apache.org/jira/browse/SOLR-10317
> Project: Solr
>  Issue Type: Task
>Reporter: Ishan Chattopadhyaya
>  Labels: gsoc2017, mentor
> Attachments: Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, 
> Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be 
> found here, https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr 
> nodes, both in SolrCloud and standalone mode, and record timing information 
> of various operations like indexing, querying, faceting, grouping, 
> replication etc.
> # It should be possible to run them either as an independent suite or as a 
> Jenkins job, and we should be able to report timings as graphs (Jenkins has 
> some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it 
> never goes out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md 
> (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr 
> in some of these frameworks above. However, the benchmarks run are very 
> limited. Any of these can be a starting point, or a new framework can as well 
> be used. The motivation is to be able to cover every functionality of Solr 
> with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure 
> [~shalinmangar] and [~markrmil...@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10655) refactor duplicate ref guide content into "snippet" files that can be included

[Cassandra Targett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010976#comment-16010976
 ] 

Cassandra Targett edited comment on SOLR-10655 at 5/15/17 5:46 PM:
---

I personally split this problem into two separate use cases - first, we used 
{{excerpt}} to save having to remember where we were maintaining a list of 
related pages...like a copied TOC. This use case doesn't actually interest me 
that much going forward, because I think it's a one-off that we are unlikely to 
repeat. With better global find/replace at our disposal now, the task of 
maintaining those pages is 1000x easier even if it doesn't reside in a single 
place.

The other use case I find more useful, which is to split a page into smaller 
bits and use {{include::}} syntax to include the snippet in a broader page. The 
way I've thought of it is that there is a sub-dir somewhere that has these 
little snippets and we do something like {{include::some-dir/y.adoc[]}}. I'm 
wary of that approach, just in terms of document sprawl with lots of possible 
contributors - it could get messy.

However, it occurs to me now that there is another approach using {{include::}} 
where you mark up a document with tags where you'll want to pull content, and 
then in your other document use the same {{include::}} macro but request only 
that tagged section, as in {{include::y.adoc\[tags=section1]}}. For a lot of 
our content, this may be a simpler approach. The syntax for using tags like 
this is explained in: http://asciidoctor.org/docs/user-manual/#include-partial


was (Author: ctargett):
I personally split this problem into two separate use cases - first, we used 
{{excerpt}} to save having to remember where we were maintaining a list of 
related pages...like a copied TOC. This use case doesn't actually interest me 
that much going forward, because I think it's a one-off that we are unlikely to 
repeat. With better global find/replace at our disposal now, the task of 
maintaining those pages is 1000x easier even if it doesn't reside in a single 
place.

The other use case I find more useful, which is to split a page into smaller 
bits and use {{include::}} syntax to include the snippet in a broader page. The 
way I've thought of it is that there is a sub-dir somewhere that has these 
little snippets and we do something like {{include::some-dir/y.adoc[]}}. I'm 
wary of that approach, just in terms of document sprawl with lots of possible 
contributors - it could get messy.

However, it occurs to me now that there is another approach using 
{{includes::}} where you mark up a document with tags where you'll want to pull 
content, and then in your other document use the same {{include::}} macro but 
request only that tagged section, as in {{include::y.adoc\[tags=section1]}}. 
For a lot of our content, this may be a simpler approach. The syntax for using 
tags like this is explained in: 
http://asciidoctor.org/docs/user-manual/#include-partial

> refactor duplicate ref guide content into "snippet" files that can be included
> --
>
> Key: SOLR-10655
> URL: https://issues.apache.org/jira/browse/SOLR-10655
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Hoss Man
>
> in cwiki, we were using the "excerpt" and "excerpt-include" macros to mirror 
> some content across multiple pages.
> as part of the cwiki->adoc migration, these macros were just evaluated 
> durring export, and the content is now duplicated.
> moving forward, we should refactor this duplicated content into "snippet" 
> files that can be included in multiple places.  A few things we need to be 
> careful about when doing this:
> * ensuring anchors and relative links are generated in such a way that they 
> are still unique per-section
> * that some convention / page attributes are used such that the 
> BuildNavAndPDFBody code doesn't try to include them as "real" pages (or 
> errors that they have no parent)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10655) refactor duplicate ref guide content into "snippet" files that can be included

[Cassandra Targett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010976#comment-16010976
 ] 

Cassandra Targett commented on SOLR-10655:
--

I personally split this problem into two separate use cases - first, we used 
{{excerpt}} to save having to remember where we were maintaining a list of 
related pages...like a copied TOC. This use case doesn't actually interest me 
that much going forward, because I think it's a one-off that we are unlikely to 
repeat. With better global find/replace at our disposal now, the task of 
maintaining those pages is 1000x easier even if it doesn't reside in a single 
place.

The other use case I find more useful, which is to split a page into smaller 
bits and use {{include::}} syntax to include the snippet in a broader page. The 
way I've thought of it is that there is a sub-dir somewhere that has these 
little snippets and we do something like {{include::some-dir/y.adoc[]}}. I'm 
wary of that approach, just in terms of document sprawl with lots of possible 
contributors - it could get messy.

However, it occurs to me now that there is another approach using 
{{includes::}} where you mark up a document with tags where you'll want to pull 
content, and then in your other document use the same {{include::}} macro but 
request only that tagged section, as in {{include::y.adoc\[tags=section1]}}. 
For a lot of our content, this may be a simpler approach. The syntax for using 
tags like this is explained in: 
http://asciidoctor.org/docs/user-manual/#include-partial

> refactor duplicate ref guide content into "snippet" files that can be included
> --
>
> Key: SOLR-10655
> URL: https://issues.apache.org/jira/browse/SOLR-10655
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Hoss Man
>
> in cwiki, we were using the "excerpt" and "excerpt-include" macros to mirror 
> some content across multiple pages.
> as part of the cwiki->adoc migration, these macros were just evaluated 
> durring export, and the content is now duplicated.
> moving forward, we should refactor this duplicated content into "snippet" 
> files that can be included in multiple places.  A few things we need to be 
> careful about when doing this:
> * ensuring anchors and relative links are generated in such a way that they 
> are still unique per-section
> * that some convention / page attributes are used such that the 
> BuildNavAndPDFBody code doesn't try to include them as "real" pages (or 
> errors that they have no parent)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10692) Split Streaming Expressions docs into more manageable parts

[Cassandra Targett (JIRA)]

Cassandra Targett created SOLR-10692:


 Summary: Split Streaming Expressions docs into more manageable 
parts
 Key: SOLR-10692
 URL: https://issues.apache.org/jira/browse/SOLR-10692
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: documentation
Reporter: Cassandra Targett
 Fix For: 6.6, master (7.0)


The Streaming Expressions page is too big and a mix of How To and Reference 
materials. It should be split up a bit to make it easier to use.

First pass at this will split all the expression types into new pages according 
to the type, so there will be 3 new sub-pages:

* Stream Sources
* Stream Decorators
* Stream Evaluators

Once that's done, we'll see if there's more we can do with this.

Attn [~joel.bernstein].



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10691) Allow to not commit index on core close

[Ivan Mamontov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Mamontov updated SOLR-10691:
-
Attachment: SOLR-10691.patch

Here is a patch without full test coverage, I'll update it later.

> Allow to not commit index on core close
> ---
>
> Key: SOLR-10691
> URL: https://issues.apache.org/jira/browse/SOLR-10691
> Project: Solr
>  Issue Type: Improvement
>Reporter: Ivan Mamontov
>Priority: Trivial
> Attachments: SOLR-10691.patch
>
>
> As a Solr user I would like to avoid unnecessary commits into Solr/Lucene 
> index on {{org.apache.solr.update.SolrIndexWriter#close}} in case IW has 
> uncommitted changes.
> In {{org.apache.lucene.index.IndexWriterConfig}}(LUCENE-5871) there is a 
> property which is currently used to decide whether to commit or discard 
> uncommitted changes  when you call close(). Unfortunately Solr does not 
> support this property in {{org.apache.solr.update.SolrIndexConfig}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8440) Script support for enabling basic auth

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010917#comment-16010917
 ] 

ASF subversion and git services commented on SOLR-8440:
---

Commit b30a042bcfbc24db8eac31d65997098ac7c8c2d9 in lucene-solr's branch 
refs/heads/branch_6_6 from [~ichattopadhyaya]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b30a042 ]

SOLR-8440: Support for enabling basic authentication using bin/solr|bin/solr.cmd


> Script support for enabling basic auth
> --
>
> Key: SOLR-8440
> URL: https://issues.apache.org/jira/browse/SOLR-8440
> Project: Solr
>  Issue Type: New Feature
>  Components: scripts and tools
>Reporter: Jan Høydahl
>Assignee: Ishan Chattopadhyaya
>  Labels: authentication, security
> Fix For: 6.6, master (7.0)
>
> Attachments: SOLR-8440-follow-up.patch, SOLR-8440-follow-up.patch, 
> SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8840_opt_parsing.patch
>
>
> Now that BasicAuthPlugin will be able to work without an AuthorizationPlugin 
> (SOLR-8429), it would be sweet to provide a super simple way to "Password 
> protect Solr"™ right from the command line:
> {noformat}
> bin/solr basicAuth -adduser -user solr -pass SolrRocks
> {noformat}
> It would take the mystery out of enabling one single password across the 
> board. The command would do something like this
> # Check if HTTPS is enabled, and if not, print a friendly warning
> # Check if {{/security.json}} already exists
> ## NO => create one with only plugin class defined
> ## YES => Abort if exists but plugin is not {{BasicAuthPlugin}}
> # Using security REST API, add the new user



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10691) Allow to not commit index on core close

[Ivan Mamontov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Mamontov updated SOLR-10691:
-
Security: (was: Public)

> Allow to not commit index on core close
> ---
>
> Key: SOLR-10691
> URL: https://issues.apache.org/jira/browse/SOLR-10691
> Project: Solr
>  Issue Type: Improvement
>Reporter: Ivan Mamontov
>Priority: Trivial
>
> As a Solr user I would like to avoid unnecessary commits into Solr/Lucene 
> index on {{org.apache.solr.update.SolrIndexWriter#close}} in case IW has 
> uncommitted changes.
> In {{org.apache.lucene.index.IndexWriterConfig}}(LUCENE-5871) there is a 
> property which is currently used to decide whether to commit or discard 
> uncommitted changes  when you call close(). Unfortunately Solr does not 
> support this property in {{org.apache.solr.update.SolrIndexConfig}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10691) Allow to not commit index on core close

[Ivan Mamontov (JIRA)]

Ivan Mamontov created SOLR-10691:


 Summary: Allow to not commit index on core close
 Key: SOLR-10691
 URL: https://issues.apache.org/jira/browse/SOLR-10691
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Ivan Mamontov
Priority: Trivial


As a Solr user I would like to avoid unnecessary commits into Solr/Lucene index 
on {{org.apache.solr.update.SolrIndexWriter#close}} in case IW has uncommitted 
changes.
In {{org.apache.lucene.index.IndexWriterConfig}}(LUCENE-5871) there is a 
property which is currently used to decide whether to commit or discard 
uncommitted changes  when you call close(). Unfortunately Solr does not support 
this property in {{org.apache.solr.update.SolrIndexConfig}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Release 6.6

[Ishan Chattopadhyaya]

I wish to backport a fix from SOLR-8440 (last commit) to the release
branch. It affects only the feature introduced in SOLR-8440. Please let me
know if someone has any objections.

Also, I'm planning to build the RC in another 3-4 hours. Please let me know
if there's something that someone is working on which needs to get in
before that.

Thanks and regards,
Ishan


On Sun, May 14, 2017 at 5:02 PM, Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Sure Steve! Thanks!
>
> On Sun, May 14, 2017 at 2:34 PM, Steve Rowe  wrote:
>
>> Ishan,
>>
>> Okay to include https://issues.apache.org/jira/browse/LUCENE-7821 for
>> 6.6?
>>
>> --
>> Steve
>> www.lucidworks.com
>>
>> > On May 12, 2017, at 12:51 PM, jim ferenczi 
>> wrote:
>> >
>> > Ok thanks Ishan.
>> >
>> > Le 12 mai 2017 18:44, "Ishan Chattopadhyaya" 
>> a écrit :
>> > Sure, Jim. Please go ahead!
>> >
>> > On Fri, May 12, 2017 at 10:01 PM, jim ferenczi 
>> wrote:
>> > Hi,
>> > Would be great to have https://issues.apache.org/jira
>> /browse/LUCENE-7824 included for 6.6. Can I merge the fix this week end
>> Ishan ?
>> >
>> > 2017-05-11 16:45 GMT+02:00 Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com>:
>> > Done, Adrien. Thanks!
>> >
>> > On Thu, May 11, 2017 at 7:34 PM, Adrien Grand 
>> wrote:
>> > Ishan, wdyt about running addVersion on the branch_6x/master as well? I
>> think it would help realize that the 6.6 branch has been cut when looking
>> at the CHANGES.txt file and not forget about backporting?
>> >
>> > Le mar. 9 mai 2017 à 17:58, Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com> a écrit :
>> > I've cut the branch for 6.6. (branch_6_6). Please feel free to add
>> bugfixes to the branch, if needed.
>> > Planning to build the first RC on 15 May. Please let me know if there
>> are any objections.
>> >
>> > Thanks,
>> > Ishan
>> >
>> > On Tue, May 9, 2017 at 11:10 AM, Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com> wrote:
>> > Planning to cut the release branch in another 10-12 hours.
>> >
>> > On Mon, May 1, 2017 at 4:00 PM, Martin Gainty 
>> wrote:
>> > i was wondering if there was a specific test for SpanPayloadCheckQuery
>> method
>> >
>> > matches = payloadToMatch.get(upto).bytesEquals(payload);
>> >
>> >
>> >
>> > the only implementation i could locate was in collectLeaf of
>> SpanPayloadCheckQuery
>> >
>> >
>> > I will submit JIRA with Patch
>> >
>> >
>> > as a CS *dinosaur* I am more familiar with LISP for dissecting sentence
>> fragments (what we called phenomes) than current SEO implementations
>> >
>> >
>> > Can you suggest a book to read to better understand Lucenes pattern
>> dissection and match algorithms?
>> >
>> >
>> > Many Thanks for helpful guidance
>> > Martin
>> > __
>> >
>> >
>> >
>> > From: Erik Hatcher 
>> > Sent: Sunday, April 30, 2017 2:06 PM
>> >
>> > To: dev@lucene.apache.org
>> > Subject: Re: Release 6.6
>> >
>> > Martin -
>> >
>> > I have to admit to still being unsure what the gist is here - is there
>> a bug?   Apologies for not catching what you’re saying/showing here.
>> Again, as far as I can tell SpanPayloadCheckQuery is working as expected
>> now.
>> >
>> > I’m going to focus purely on SOLR-1485 this week to get it committed
>> for 6.6.  If there is an issue to address with your work would you please
>> open a JIRA and include your patch there?
>> >
>> > Thanks,
>> > Erik
>> >
>> >
>> >> On Apr 30, 2017, at 7:47 AM, Martin Gainty 
>> wrote:
>> >>
>> >> Mornin' Erik
>> >>
>> >> there is a collectLeaf  override in 
>> >> org.apache.lucene.queries.payloads.TestPayloadSpans
>> ..but its never called:
>> >>
>> >> static class VerifyingCollector implements SpanCollector {
>> >> List payloads = new ArrayList<>();
>> >> @Override
>> >> public void collectLeaf(PostingsEnum postings, int position, Term
>> term) throws IOException {
>> >>  
>> >>  }
>> >> }
>> >>
>> >> the modification in 
>> >> org.apache.lucene.queries.payloads.TestPayloadCheckQuery
>> tests collectLeaf for query
>> >>
>> >> //initialise term
>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 231
>> before term1=new org.apache.lucene.index.Term('field','withPayload')");
>> >> org.apache.lucene.index.Term term1=new 
>> >> org.apache.lucene.index.Term("field",
>> "withPayload");
>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 233
>> term1="+term1);
>> >>
>> >> //assume position is 5
>> >> int position=5;
>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 235
>> position="+position);
>> >>
>> >> BytesRef pay = new BytesRef("pos: " + position);
>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 237
>> pay="+pay);
>> >>
>> >> //build spanQuery with term parameter
>> >> org.apache.lucene.search.spans.SpanQuery spanQuery1 = new
>> SpanTermQuery(term1);
>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 239
>> spanQuery1="+spanQuery1);
>> >>
>> >> //add BytesRef to payloadToMatch 

[jira] [Commented] (SOLR-8440) Script support for enabling basic auth

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010815#comment-16010815
 ] 

ASF subversion and git services commented on SOLR-8440:
---

Commit 89dc9c5e748fdebfed51ca33c3207233c9983836 in lucene-solr's branch 
refs/heads/branch_6x from [~ichattopadhyaya]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=89dc9c5 ]

SOLR-8440: Support for enabling basic authentication using bin/solr|bin/solr.cmd


> Script support for enabling basic auth
> --
>
> Key: SOLR-8440
> URL: https://issues.apache.org/jira/browse/SOLR-8440
> Project: Solr
>  Issue Type: New Feature
>  Components: scripts and tools
>Reporter: Jan Høydahl
>Assignee: Ishan Chattopadhyaya
>  Labels: authentication, security
> Fix For: 6.6, master (7.0)
>
> Attachments: SOLR-8440-follow-up.patch, SOLR-8440-follow-up.patch, 
> SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8840_opt_parsing.patch
>
>
> Now that BasicAuthPlugin will be able to work without an AuthorizationPlugin 
> (SOLR-8429), it would be sweet to provide a super simple way to "Password 
> protect Solr"™ right from the command line:
> {noformat}
> bin/solr basicAuth -adduser -user solr -pass SolrRocks
> {noformat}
> It would take the mystery out of enabling one single password across the 
> board. The command would do something like this
> # Check if HTTPS is enabled, and if not, print a friendly warning
> # Check if {{/security.json}} already exists
> ## NO => create one with only plugin class defined
> ## YES => Abort if exists but plugin is not {{BasicAuthPlugin}}
> # Using security REST API, add the new user



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8440) Script support for enabling basic auth

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010811#comment-16010811
 ] 

ASF subversion and git services commented on SOLR-8440:
---

Commit 9be68cc3079a320d323e46c570de3e9e883052ed in lucene-solr's branch 
refs/heads/master from [~ichattopadhyaya]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9be68cc ]

SOLR-8440: Support for enabling basic authentication using bin/solr|bin/solr.cmd


> Script support for enabling basic auth
> --
>
> Key: SOLR-8440
> URL: https://issues.apache.org/jira/browse/SOLR-8440
> Project: Solr
>  Issue Type: New Feature
>  Components: scripts and tools
>Reporter: Jan Høydahl
>Assignee: Ishan Chattopadhyaya
>  Labels: authentication, security
> Fix For: 6.6, master (7.0)
>
> Attachments: SOLR-8440-follow-up.patch, SOLR-8440-follow-up.patch, 
> SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8840_opt_parsing.patch
>
>
> Now that BasicAuthPlugin will be able to work without an AuthorizationPlugin 
> (SOLR-8429), it would be sweet to provide a super simple way to "Password 
> protect Solr"™ right from the command line:
> {noformat}
> bin/solr basicAuth -adduser -user solr -pass SolrRocks
> {noformat}
> It would take the mystery out of enabling one single password across the 
> board. The command would do something like this
> # Check if HTTPS is enabled, and if not, print a friendly warning
> # Check if {{/security.json}} already exists
> ## NO => create one with only plugin class defined
> ## YES => Abort if exists but plugin is not {{BasicAuthPlugin}}
> # Using security REST API, add the new user



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10661) Add copyOf Stream Evaluator

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10661:
--
Description: 
The copyOf Stream Evaluator returns a copy of an array with a specific range.

Syntax:
{code}
a = copyOf(colA, offset, length)
{code}

{code}
a = copyOf(colA, offset)
{code}


  was:
The subarray Stream Evaluator returns a sub range of an array.

Syntax:
{code}
a = copyOf(colA, offset, length)
{code}

{code}
a = copyOf(colA, offset)
{code}



> Add copyOf Stream Evaluator
> ---
>
> Key: SOLR-10661
> URL: https://issues.apache.org/jira/browse/SOLR-10661
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Fix For: master (7.0)
>
>
> The copyOf Stream Evaluator returns a copy of an array with a specific range.
> Syntax:
> {code}
> a = copyOf(colA, offset, length)
> {code}
> {code}
> a = copyOf(colA, offset)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10661) Add copyOf Stream Evaluator

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10661:
--
Summary: Add copyOf Stream Evaluator  (was: Add copyof Stream Evaluator)

> Add copyOf Stream Evaluator
> ---
>
> Key: SOLR-10661
> URL: https://issues.apache.org/jira/browse/SOLR-10661
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Fix For: master (7.0)
>
>
> The subarray Stream Evaluator returns a sub range of an array.
> Syntax:
> {code}
> a = copyOf(colA, offset, length)
> {code}
> {code}
> a = copyOf(colA, offset)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10661) Add copyof Stream Evaluator

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10661:
--
Description: 
The subarray Stream Evaluator returns a sub range of an array.

Syntax:
{code}
a = copyOf(colA, offset, length)
{code}

{code}
a = copyOf(colA, offset)
{code}


  was:
The subarray Stream Evaluator returns a sub range of an array.

Syntax:
{code}
a = copyof(colA, offset, length)
{code}

{code}
a = copyof(colA, offset)
{code}



> Add copyof Stream Evaluator
> ---
>
> Key: SOLR-10661
> URL: https://issues.apache.org/jira/browse/SOLR-10661
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Fix For: master (7.0)
>
>
> The subarray Stream Evaluator returns a sub range of an array.
> Syntax:
> {code}
> a = copyOf(colA, offset, length)
> {code}
> {code}
> a = copyOf(colA, offset)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10661) Add copyof Stream Evaluator

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10661:
--
Summary: Add copyof Stream Evaluator  (was: Add subarray Stream Evaluator)

> Add copyof Stream Evaluator
> ---
>
> Key: SOLR-10661
> URL: https://issues.apache.org/jira/browse/SOLR-10661
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Fix For: master (7.0)
>
>
> The subarray Stream Evaluator returns a sub range of an array.
> Syntax:
> {code}
> a = subarray(colA, offset, length)
> {code}
> {code}
> a = subarray(colA, offset)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10661) Add copyof Stream Evaluator

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10661:
--
Description: 
The subarray Stream Evaluator returns a sub range of an array.

Syntax:
{code}
a = copyof(colA, offset, length)
{code}

{code}
a = copyof(colA, offset)
{code}


  was:
The subarray Stream Evaluator returns a sub range of an array.

Syntax:
{code}
a = subarray(colA, offset, length)
{code}

{code}
a = subarray(colA, offset)
{code}



> Add copyof Stream Evaluator
> ---
>
> Key: SOLR-10661
> URL: https://issues.apache.org/jira/browse/SOLR-10661
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Fix For: master (7.0)
>
>
> The subarray Stream Evaluator returns a sub range of an array.
> Syntax:
> {code}
> a = copyof(colA, offset, length)
> {code}
> {code}
> a = copyof(colA, offset)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10662) Add length Stream Evaluator

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010716#comment-16010716
 ] 

ASF subversion and git services commented on SOLR-10662:


Commit 680dcc140d5c15be996f00278addab4c4e0b288c in lucene-solr's branch 
refs/heads/master from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=680dcc1 ]

SOLR-10662: Add length Stream Evaluator


> Add length Stream Evaluator 
> 
>
> Key: SOLR-10662
> URL: https://issues.apache.org/jira/browse/SOLR-10662
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: master (7.0)
>
> Attachments: SOLR-10662.patch
>
>
> The *length* Stream Evaluator returns the length of an array.
> Syntax:
> {code}
> a = length(colA)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10663) Add distance Stream Evaluator

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010719#comment-16010719
 ] 

ASF subversion and git services commented on SOLR-10663:


Commit 9c6279d439a231d9ec8c9564b0ab76f616d10076 in lucene-solr's branch 
refs/heads/master from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9c6279d ]

SOLR-10663: Add distance Stream Evaluator


> Add distance Stream Evaluator
> -
>
> Key: SOLR-10663
> URL: https://issues.apache.org/jira/browse/SOLR-10663
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: master (7.0)
>
> Attachments: SOLR-10663.patch
>
>
> The *distance* Stream Evaluator will find the Euclidean distance between two 
> arrays of numbers.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10666) Add rank transformation Stream Evaluator

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010717#comment-16010717
 ] 

ASF subversion and git services commented on SOLR-10666:


Commit 6e41ac7a3ba7a45a4d42b9f6b047350ff5d769ae in lucene-solr's branch 
refs/heads/master from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6e41ac7 ]

SOLR-10666: Add rank transformation Stream Evaluator


> Add rank transformation Stream Evaluator
> 
>
> Key: SOLR-10666
> URL: https://issues.apache.org/jira/browse/SOLR-10666
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: master (7.0)
>
> Attachments: SOLR-10666.patch
>
>
> The *rank* Stream Evaluator performs a rank transformation on an array of 
> numbers.
> Syntax:
> {code}
> a = rank(colA)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10664) Add scale Stream Evaluator

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010718#comment-16010718
 ] 

ASF subversion and git services commented on SOLR-10664:


Commit a6c516519ea4426dd3178766e9ac821f2f1d5581 in lucene-solr's branch 
refs/heads/master from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a6c5165 ]

SOLR-10664: Add scale Stream Evaluator


> Add scale Stream Evaluator
> --
>
> Key: SOLR-10664
> URL: https://issues.apache.org/jira/browse/SOLR-10664
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: master (7.0)
>
> Attachments: SOLR-10664.patch
>
>
> The *scale* Stream Evaluator scales an array by multiplying all the values of 
> an array by a number and returns the scaled array.
> Syntax:
> {code}
> a = scale(10, colA)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10585) Remove defaultSearchField completely in 7.0

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-10585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-10585:
---
Attachment: SOLR-10585.patch

First patch. Please review

> Remove defaultSearchField completely in 7.0
> ---
>
> Key: SOLR-10585
> URL: https://issues.apache.org/jira/browse/SOLR-10585
> Project: Solr
>  Issue Type: Sub-task
>  Components: Schema and Analysis
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Fix For: master (7.0)
>
> Attachments: SOLR-10585.patch
>
>
> Sub task of SOLR-7041.
> This will remove code related to defaultSearchField in schema and will throw 
> a hard exception if a schema using this config is attempted loaded.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7828) Improve PointValues visitor calls when all docs in a leaf share a value

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010649#comment-16010649
 ] 

Alan Woodward commented on LUCENE-7828:
---

I'm trying out a few ideas here; the one I think shows the most promise is to 
change IntersectVisitor.visit(int, byte[]) to take an array of docids.  This 
also opens up the possibility of speeding things up when a leaf only contains a 
few different values.

> Improve PointValues visitor calls when all docs in a leaf share a value
> ---
>
> Key: LUCENE-7828
> URL: https://issues.apache.org/jira/browse/LUCENE-7828
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>
> When all the docs in a leaf node have the same value, range queries can waste 
> a lot of processing if the node itself returns CELL_CROSSES_QUERY when 
> compare() is called, in effect performing the same calculation in visit(int, 
> byte[]) over and over again.  In the case I'm looking at (very low 
> cardinality indexed LongRange fields), this causes something of a perfect 
> storm for performance.  PointValues can detect up front if a given node has a 
> single value (because it's min value and max value will be equal), so this 
> case should be fairly simple to identify and shortcut.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7828) Improve PointValues visitor calls when all docs in a leaf share a value

[Alan Woodward (JIRA)]

Alan Woodward created LUCENE-7828:
-

 Summary: Improve PointValues visitor calls when all docs in a leaf 
share a value
 Key: LUCENE-7828
 URL: https://issues.apache.org/jira/browse/LUCENE-7828
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Alan Woodward


When all the docs in a leaf node have the same value, range queries can waste a 
lot of processing if the node itself returns CELL_CROSSES_QUERY when compare() 
is called, in effect performing the same calculation in visit(int, byte[]) over 
and over again.  In the case I'm looking at (very low cardinality indexed 
LongRange fields), this causes something of a perfect storm for performance.  
PointValues can detect up front if a given node has a single value (because 
it's min value and max value will be equal), so this case should be fairly 
simple to identify and shortcut.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10689) migrate in collections api not working

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved SOLR-10689.
---
Resolution: Fixed

Please raise this question on the user's list at solr-u...@lucene.apache.org, 
there are a _lot_ more people watching that list who may be able to help. 

If it's determined that this is a code issue in Solr and not a 
configuration/usage problem, we can raise a JIRA.

When you do raise the issue on the JIRA list, you must include enough detail to 
help us help you. Exactly what command did you use? Was there any useful 
information in the logs? etc.

> migrate in collections api not working
> --
>
> Key: SOLR-10689
> URL: https://issues.apache.org/jira/browse/SOLR-10689
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Admin UI, clients - java
>Affects Versions: 6.5.1
>Reporter: chandru
>
> When migrating with the same query that was given in collection api , There 
> was no docs that were migrated from A -> B. Please help me to proceed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10644) solr.in.sh installed by install script should not be world readable

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-10644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl resolved SOLR-10644.

Resolution: Fixed

Closing again after adjustments

> solr.in.sh installed by install script should not be world readable
> ---
>
> Key: SOLR-10644
> URL: https://issues.apache.org/jira/browse/SOLR-10644
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Fix For: 6.6, master (7.0)
>
> Attachments: SOLR-10644.patch, SOLR-10644-reopen.patch
>
>
> Spinoff from SOLR-8440
> {{install_solr_service.sh}} installs {{solr.in.sh}} as world-readable but not 
> solr user writable:
> {noformat}
> -rw-r--r-- 1 root root 5968 Feb 15 14:55 /etc/default/solr.in.sh
> {noformat}
> For better security, and ease for scripts to update solr.in.sh, this should 
> change to:
> {noformat}
> -rw-rw 1 root solr 5968 Feb 15 14:55 /etc/default/solr.in.sh
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10644) solr.in.sh installed by install script should not be world readable

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010588#comment-16010588
 ] 

ASF subversion and git services commented on SOLR-10644:


Commit a5597a98b5ea5f52c45b7fc55e31ec7e8633b907 in lucene-solr's branch 
refs/heads/branch_6_6 from [~janhoy]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a5597a9 ]

SOLR-10644: Make solr.in.sh owned by root, readable to solr (group) but not 
world-readable

(cherry picked from commit 50a0804)


> solr.in.sh installed by install script should not be world readable
> ---
>
> Key: SOLR-10644
> URL: https://issues.apache.org/jira/browse/SOLR-10644
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Fix For: 6.6, master (7.0)
>
> Attachments: SOLR-10644.patch, SOLR-10644-reopen.patch
>
>
> Spinoff from SOLR-8440
> {{install_solr_service.sh}} installs {{solr.in.sh}} as world-readable but not 
> solr user writable:
> {noformat}
> -rw-r--r-- 1 root root 5968 Feb 15 14:55 /etc/default/solr.in.sh
> {noformat}
> For better security, and ease for scripts to update solr.in.sh, this should 
> change to:
> {noformat}
> -rw-rw 1 root solr 5968 Feb 15 14:55 /etc/default/solr.in.sh
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10644) solr.in.sh installed by install script should not be world readable

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010587#comment-16010587
 ] 

ASF subversion and git services commented on SOLR-10644:


Commit 45627d48f99ee014dda8f781befecfbd76669126 in lucene-solr's branch 
refs/heads/branch_6x from [~janhoy]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=45627d4 ]

SOLR-10644: Make solr.in.sh owned by root, readable to solr (group) but not 
world-readable

(cherry picked from commit 50a0804)


> solr.in.sh installed by install script should not be world readable
> ---
>
> Key: SOLR-10644
> URL: https://issues.apache.org/jira/browse/SOLR-10644
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Fix For: 6.6, master (7.0)
>
> Attachments: SOLR-10644.patch, SOLR-10644-reopen.patch
>
>
> Spinoff from SOLR-8440
> {{install_solr_service.sh}} installs {{solr.in.sh}} as world-readable but not 
> solr user writable:
> {noformat}
> -rw-r--r-- 1 root root 5968 Feb 15 14:55 /etc/default/solr.in.sh
> {noformat}
> For better security, and ease for scripts to update solr.in.sh, this should 
> change to:
> {noformat}
> -rw-rw 1 root solr 5968 Feb 15 14:55 /etc/default/solr.in.sh
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10690) Remove unused theme files

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett resolved SOLR-10690.
--
Resolution: Fixed

In addition to removing some old include files that aren't used, I removed 
remaining traces of "feedback" links, which I had disabled earlier but didn't 
fully remove. This meant removing the options from {{_config.yml.template}}.

> Remove unused theme files
> -
>
> Key: SOLR-10690
> URL: https://issues.apache.org/jira/browse/SOLR-10690
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
> Fix For: 6.6, master (7.0)
>
>
> I found more files that came from the initial Jekyll theme I borrowed that we 
> aren't using because AsciiDoc has support for those features (the theme used 
> Markdown). 
> This issue is to remove those.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10644) solr.in.sh installed by install script should not be world readable

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010573#comment-16010573
 ] 

ASF subversion and git services commented on SOLR-10644:


Commit 50a08042630c28fb2b85a05440dae170a46cbf49 in lucene-solr's branch 
refs/heads/master from [~janhoy]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=50a0804 ]

SOLR-10644: Make solr.in.sh owned by root, readable to solr (group) but not 
world-readable


> solr.in.sh installed by install script should not be world readable
> ---
>
> Key: SOLR-10644
> URL: https://issues.apache.org/jira/browse/SOLR-10644
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Fix For: 6.6, master (7.0)
>
> Attachments: SOLR-10644.patch, SOLR-10644-reopen.patch
>
>
> Spinoff from SOLR-8440
> {{install_solr_service.sh}} installs {{solr.in.sh}} as world-readable but not 
> solr user writable:
> {noformat}
> -rw-r--r-- 1 root root 5968 Feb 15 14:55 /etc/default/solr.in.sh
> {noformat}
> For better security, and ease for scripts to update solr.in.sh, this should 
> change to:
> {noformat}
> -rw-rw 1 root solr 5968 Feb 15 14:55 /etc/default/solr.in.sh
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10690) Remove unused theme files

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010576#comment-16010576
 ] 

ASF subversion and git services commented on SOLR-10690:


Commit 13395a6fc6ac2ec1c8dcde56f338bfa2fb6fc00a in lucene-solr's branch 
refs/heads/branch_6_6 from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=13395a6 ]

SOLR-10690: remove unused files left over from theme adaptation


> Remove unused theme files
> -
>
> Key: SOLR-10690
> URL: https://issues.apache.org/jira/browse/SOLR-10690
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
> Fix For: 6.6, master (7.0)
>
>
> I found more files that came from the initial Jekyll theme I borrowed that we 
> aren't using because AsciiDoc has support for those features (the theme used 
> Markdown). 
> This issue is to remove those.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10690) Remove unused theme files

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010574#comment-16010574
 ] 

ASF subversion and git services commented on SOLR-10690:


Commit e5053f87b25c2ec91089a20eb224261eb2bf42a7 in lucene-solr's branch 
refs/heads/branch_6x from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e5053f8 ]

SOLR-10690: remove unused files left over from theme adaptation


> Remove unused theme files
> -
>
> Key: SOLR-10690
> URL: https://issues.apache.org/jira/browse/SOLR-10690
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
> Fix For: 6.6, master (7.0)
>
>
> I found more files that came from the initial Jekyll theme I borrowed that we 
> aren't using because AsciiDoc has support for those features (the theme used 
> Markdown). 
> This issue is to remove those.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10690) Remove unused theme files

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010569#comment-16010569
 ] 

ASF subversion and git services commented on SOLR-10690:


Commit 08711cf4fbe845f5d28b1b4f1bc6d890a5595089 in lucene-solr's branch 
refs/heads/master from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=08711cf ]

SOLR-10690: remove unused files left over from theme adaptation


> Remove unused theme files
> -
>
> Key: SOLR-10690
> URL: https://issues.apache.org/jira/browse/SOLR-10690
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
> Fix For: 6.6, master (7.0)
>
>
> I found more files that came from the initial Jekyll theme I borrowed that we 
> aren't using because AsciiDoc has support for those features (the theme used 
> Markdown). 
> This issue is to remove those.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10584) Remove defaultOperator completely in 7.0

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-10584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010562#comment-16010562
 ] 

Jan Høydahl commented on SOLR-10584:


It was incredibly nice to be able to edit RefGuide in the same commit as code. 
And to search all *.adoc files in IntelliJ to make sure all occurrences of 
defaultOperator in docs were discovered :)

> Remove defaultOperator completely in 7.0
> 
>
> Key: SOLR-10584
> URL: https://issues.apache.org/jira/browse/SOLR-10584
> Project: Solr
>  Issue Type: Sub-task
>  Components: Schema and Analysis
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Fix For: master (7.0)
>
> Attachments: SOLR-10584.patch, SOLR-10584.patch
>
>
> Sub task of SOLR-7041.
> This will remove code related to defaultOperator in schema and will throw a 
> hard exception if a schema using this config is attempted loaded.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10584) Remove defaultOperator completely in 7.0

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010557#comment-16010557
 ] 

ASF subversion and git services commented on SOLR-10584:


Commit 295602ce7b925ef5e35a6cb361f581923ee45aab in lucene-solr's branch 
refs/heads/master from [~janhoy]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=295602c ]

SOLR-10584: RefGuide updates, removal of some more dead code and a failing test


> Remove defaultOperator completely in 7.0
> 
>
> Key: SOLR-10584
> URL: https://issues.apache.org/jira/browse/SOLR-10584
> Project: Solr
>  Issue Type: Sub-task
>  Components: Schema and Analysis
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Fix For: master (7.0)
>
> Attachments: SOLR-10584.patch, SOLR-10584.patch
>
>
> Sub task of SOLR-7041.
> This will remove code related to defaultOperator in schema and will throw a 
> hard exception if a schema using this config is attempted loaded.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10690) Remove unused theme files

[Cassandra Targett (JIRA)]

Cassandra Targett created SOLR-10690:


 Summary: Remove unused theme files
 Key: SOLR-10690
 URL: https://issues.apache.org/jira/browse/SOLR-10690
 Project: Solr
  Issue Type: Sub-task
  Security Level: Public (Default Security Level. Issues are Public)
  Components: documentation
Reporter: Cassandra Targett
Assignee: Cassandra Targett
 Fix For: 6.6, master (7.0)


I found more files that came from the initial Jekyll theme I borrowed that we 
aren't using because AsciiDoc has support for those features (the theme used 
Markdown). 

This issue is to remove those.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10670) [ref-guide] Various top-bar fixes

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett resolved SOLR-10670.
--
   Resolution: Fixed
Fix Version/s: master (7.0)

> [ref-guide] Various top-bar fixes
> -
>
> Key: SOLR-10670
> URL: https://issues.apache.org/jira/browse/SOLR-10670
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Jan Høydahl
>Assignee: Cassandra Targett
>  Labels: asciidoc, html
> Fix For: 6.6, master (7.0)
>
> Attachments: SOLR-10670-home.patch, SOLR-10670.hoss.patch, 
> SOLR-10670.patch, SOLR-10670.patch, SOLR-10670.patch
>
>
> A few suggestions for the new ref-guide HTML format:
> * Favicon is not displayed, image missing in folder
> * Topnav link to community should point to 
> http://lucene.apache.org/solr/community.html
> * Replace "Solr News" link with a "Solr Website" link - we should link to the 
> website
> * Instead of pointint the Source Code link to cryptic apache GIT, point to 
> https://lucene.apache.org/solr/community.html#version-control where people 
> get more context and can also find the GitHub link



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-10670) [ref-guide] Various top-bar fixes

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett reassigned SOLR-10670:


Assignee: Cassandra Targett

> [ref-guide] Various top-bar fixes
> -
>
> Key: SOLR-10670
> URL: https://issues.apache.org/jira/browse/SOLR-10670
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Jan Høydahl
>Assignee: Cassandra Targett
>  Labels: asciidoc, html
> Fix For: 6.6
>
> Attachments: SOLR-10670-home.patch, SOLR-10670.hoss.patch, 
> SOLR-10670.patch, SOLR-10670.patch, SOLR-10670.patch
>
>
> A few suggestions for the new ref-guide HTML format:
> * Favicon is not displayed, image missing in folder
> * Topnav link to community should point to 
> http://lucene.apache.org/solr/community.html
> * Replace "Solr News" link with a "Solr Website" link - we should link to the 
> website
> * Instead of pointint the Source Code link to cryptic apache GIT, point to 
> https://lucene.apache.org/solr/community.html#version-control where people 
> get more context and can also find the GitHub link



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10670) [ref-guide] Various top-bar fixes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010528#comment-16010528
 ] 

ASF subversion and git services commented on SOLR-10670:


Commit 663fd9b32dd173076824f1cd6cd575ccbaa29a22 in lucene-solr's branch 
refs/heads/branch_6_6 from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=663fd9b ]

SOLR-10670: move logo and background images & fix paths


> [ref-guide] Various top-bar fixes
> -
>
> Key: SOLR-10670
> URL: https://issues.apache.org/jira/browse/SOLR-10670
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Jan Høydahl
>  Labels: asciidoc, html
> Fix For: 6.6
>
> Attachments: SOLR-10670-home.patch, SOLR-10670.hoss.patch, 
> SOLR-10670.patch, SOLR-10670.patch, SOLR-10670.patch
>
>
> A few suggestions for the new ref-guide HTML format:
> * Favicon is not displayed, image missing in folder
> * Topnav link to community should point to 
> http://lucene.apache.org/solr/community.html
> * Replace "Solr News" link with a "Solr Website" link - we should link to the 
> website
> * Instead of pointint the Source Code link to cryptic apache GIT, point to 
> https://lucene.apache.org/solr/community.html#version-control where people 
> get more context and can also find the GitHub link



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10670) [ref-guide] Various top-bar fixes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010526#comment-16010526
 ] 

ASF subversion and git services commented on SOLR-10670:


Commit 2bb7b2734982b361af25502b83b854d1a34e0dc1 in lucene-solr's branch 
refs/heads/branch_6x from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2bb7b27 ]

SOLR-10670: move logo and background images & fix paths


> [ref-guide] Various top-bar fixes
> -
>
> Key: SOLR-10670
> URL: https://issues.apache.org/jira/browse/SOLR-10670
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Jan Høydahl
>  Labels: asciidoc, html
> Fix For: 6.6
>
> Attachments: SOLR-10670-home.patch, SOLR-10670.hoss.patch, 
> SOLR-10670.patch, SOLR-10670.patch, SOLR-10670.patch
>
>
> A few suggestions for the new ref-guide HTML format:
> * Favicon is not displayed, image missing in folder
> * Topnav link to community should point to 
> http://lucene.apache.org/solr/community.html
> * Replace "Solr News" link with a "Solr Website" link - we should link to the 
> website
> * Instead of pointint the Source Code link to cryptic apache GIT, point to 
> https://lucene.apache.org/solr/community.html#version-control where people 
> get more context and can also find the GitHub link



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10670) [ref-guide] Various top-bar fixes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010525#comment-16010525
 ] 

ASF subversion and git services commented on SOLR-10670:


Commit a1078143eddadf40c275594e335e063eacfc6e93 in lucene-solr's branch 
refs/heads/master from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a107814 ]

SOLR-10670: move logo and background images & fix paths


> [ref-guide] Various top-bar fixes
> -
>
> Key: SOLR-10670
> URL: https://issues.apache.org/jira/browse/SOLR-10670
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Jan Høydahl
>  Labels: asciidoc, html
> Fix For: 6.6
>
> Attachments: SOLR-10670-home.patch, SOLR-10670.hoss.patch, 
> SOLR-10670.patch, SOLR-10670.patch, SOLR-10670.patch
>
>
> A few suggestions for the new ref-guide HTML format:
> * Favicon is not displayed, image missing in folder
> * Topnav link to community should point to 
> http://lucene.apache.org/solr/community.html
> * Replace "Solr News" link with a "Solr Website" link - we should link to the 
> website
> * Instead of pointint the Source Code link to cryptic apache GIT, point to 
> https://lucene.apache.org/solr/community.html#version-control where people 
> get more context and can also find the GitHub link



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10670) [ref-guide] Various top-bar fixes

[Cassandra Targett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010503#comment-16010503
 ] 

Cassandra Targett commented on SOLR-10670:
--

I intend to do one more change on this issue, which is to move the icons into 
the images directory where they should have been to begin with. I won't attach 
a patch, because it's just moving a couple files and updating paths.

Then I think we will have covered all the things that came up initially, 
besides the sidebar stuff but I think that grew out of figuring out where to 
put "Home", which since it doesn't exist isn't a problem :) I'm sure there are 
styling things we can do to the sidebar to make it better - Hoss & I only took 
it as far as "good enough". Let's take those up in a separate issue, though.

> [ref-guide] Various top-bar fixes
> -
>
> Key: SOLR-10670
> URL: https://issues.apache.org/jira/browse/SOLR-10670
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Jan Høydahl
>  Labels: asciidoc, html
> Fix For: 6.6
>
> Attachments: SOLR-10670-home.patch, SOLR-10670.hoss.patch, 
> SOLR-10670.patch, SOLR-10670.patch, SOLR-10670.patch
>
>
> A few suggestions for the new ref-guide HTML format:
> * Favicon is not displayed, image missing in folder
> * Topnav link to community should point to 
> http://lucene.apache.org/solr/community.html
> * Replace "Solr News" link with a "Solr Website" link - we should link to the 
> website
> * Instead of pointint the Source Code link to cryptic apache GIT, point to 
> https://lucene.apache.org/solr/community.html#version-control where people 
> get more context and can also find the GitHub link



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8440) Script support for enabling basic auth

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-8440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010496#comment-16010496
 ] 

Jan Høydahl commented on SOLR-8440:
---

Did a quick visual read-through of the new patch (did not apply and test), and 
did not spot any red flags...

> Script support for enabling basic auth
> --
>
> Key: SOLR-8440
> URL: https://issues.apache.org/jira/browse/SOLR-8440
> Project: Solr
>  Issue Type: New Feature
>  Components: scripts and tools
>Reporter: Jan Høydahl
>Assignee: Ishan Chattopadhyaya
>  Labels: authentication, security
> Fix For: 6.6, master (7.0)
>
> Attachments: SOLR-8440-follow-up.patch, SOLR-8440-follow-up.patch, 
> SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8840_opt_parsing.patch
>
>
> Now that BasicAuthPlugin will be able to work without an AuthorizationPlugin 
> (SOLR-8429), it would be sweet to provide a super simple way to "Password 
> protect Solr"™ right from the command line:
> {noformat}
> bin/solr basicAuth -adduser -user solr -pass SolrRocks
> {noformat}
> It would take the mystery out of enabling one single password across the 
> board. The command would do something like this
> # Check if HTTPS is enabled, and if not, print a friendly warning
> # Check if {{/security.json}} already exists
> ## NO => create one with only plugin class defined
> ## YES => Abort if exists but plugin is not {{BasicAuthPlugin}}
> # Using security REST API, add the new user



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10584) Remove defaultOperator completely in 7.0

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010451#comment-16010451
 ] 

ASF subversion and git services commented on SOLR-10584:


Commit 66cd20e98a611bd28ed642be518d34edc4da5d20 in lucene-solr's branch 
refs/heads/master from [~janhoy]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=66cd20e ]

SOLR-10584: Remove defaultOperator completely in 7.0


> Remove defaultOperator completely in 7.0
> 
>
> Key: SOLR-10584
> URL: https://issues.apache.org/jira/browse/SOLR-10584
> Project: Solr
>  Issue Type: Sub-task
>  Components: Schema and Analysis
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Fix For: master (7.0)
>
> Attachments: SOLR-10584.patch, SOLR-10584.patch
>
>
> Sub task of SOLR-7041.
> This will remove code related to defaultOperator in schema and will throw a 
> hard exception if a schema using this config is attempted loaded.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10584) Remove defaultOperator completely in 7.0

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-10584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl resolved SOLR-10584.

Resolution: Fixed

> Remove defaultOperator completely in 7.0
> 
>
> Key: SOLR-10584
> URL: https://issues.apache.org/jira/browse/SOLR-10584
> Project: Solr
>  Issue Type: Sub-task
>  Components: Schema and Analysis
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Fix For: master (7.0)
>
> Attachments: SOLR-10584.patch, SOLR-10584.patch
>
>
> Sub task of SOLR-7041.
> This will remove code related to defaultOperator in schema and will throw a 
> hard exception if a schema using this config is attempted loaded.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8440) Script support for enabling basic auth

[Ishan Chattopadhyaya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya updated SOLR-8440:
---
Attachment: SOLR-8440-follow-up.patch

Updated the patch:
# Changed {{-blockUnknown}} and {{-prompt}} to have a true|false argument.
# Accepts -s and -d (home and server directory), where basicAuth.conf is kept.
# Updated the Windows script to parse the parameters.
# Introduced {{-updateIncludeFileOnly true|false}} parameter, which avoids 
updating security.json and only updates the solr.in.sh / solr.in.cmd.

Tested manually on GNU/Linux and Windows. [~janhoy], please review. I'll test a 
bit more before committing.

> Script support for enabling basic auth
> --
>
> Key: SOLR-8440
> URL: https://issues.apache.org/jira/browse/SOLR-8440
> Project: Solr
>  Issue Type: New Feature
>  Components: scripts and tools
>Reporter: Jan Høydahl
>Assignee: Ishan Chattopadhyaya
>  Labels: authentication, security
> Fix For: 6.6, master (7.0)
>
> Attachments: SOLR-8440-follow-up.patch, SOLR-8440-follow-up.patch, 
> SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8840_opt_parsing.patch
>
>
> Now that BasicAuthPlugin will be able to work without an AuthorizationPlugin 
> (SOLR-8429), it would be sweet to provide a super simple way to "Password 
> protect Solr"™ right from the command line:
> {noformat}
> bin/solr basicAuth -adduser -user solr -pass SolrRocks
> {noformat}
> It would take the mystery out of enabling one single password across the 
> board. The command would do something like this
> # Check if HTTPS is enabled, and if not, print a friendly warning
> # Check if {{/security.json}} already exists
> ## NO => create one with only plugin class defined
> ## YES => Abort if exists but plugin is not {{BasicAuthPlugin}}
> # Using security REST API, add the new user



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7800) Remove code that potentially rethrows checked exceptions from methods that don't declare them ("sneaky throw" hack)

[Dawid Weiss (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Weiss updated LUCENE-7800:

Attachment: LUCENE-7800.patch

I return to this issue to either resolve it or have it vetoed, because 
otherwise it'll be forgotted forever.

Attached is a patch which removes unchecked-exception throwing hacks from the 
codebase for reasons I already mentioned. The changes are:

1) SnowballProgram throws an AssertionError should an exception happen in 
{{find_among}} (it shouldn't).

2) {{AttributeFactory}} rethrows Error and RuntimeException classes, but wraps 
checked exceptions in a {{AttributeInstantiationException}} that is a subclass 
of {{RuntimeException}}. I made the {{AttributeInstantiationException}} class 
public just in case somebody desperately wanted to instanceof it and get the 
cause.

3) JavascriptCompiler throws throws a {{RuntimeException}} with a cause set to 
{{ParseException}}. The code unwraps such checked parse exceptions in 
{{getAntlrParseTree}} anyway, so this is only an internal change.



> Remove code that potentially rethrows checked exceptions from methods that 
> don't declare them ("sneaky throw" hack)
> ---
>
> Key: LUCENE-7800
> URL: https://issues.apache.org/jira/browse/LUCENE-7800
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Minor
> Fix For: 6.x, master (7.0)
>
> Attachments: LUCENE-7800.patch, LUCENE-7800.patch
>
>
> For a long time I considered the "sneaky" throw hack to be a nice way of 
> coding around some of Java's limitations (especially with invoking methods 
> via reflection or method handles), but with time I started to see how it can 
> be potentially dangerous and is nearly always confusing. If you have a Java 
> method and its signature doesn't indicate the possibility of a checked 
> exception you, as a programmer, simply don't expect it to happen. Never. So, 
> for example, you could write:
> {code}
> try {
>  luceneApi();
> } catch (RuntimeException | Error e) {
>   // Handle unchecked exceptions here.
> }
> {code}
> and consider the code above to be absolutely bullet-proof in terms of 
> handling exceptions. Unfortunately with sneaky throws anywhere in the 
> "luceneApi" this is no longer the case -- you can receive a checked exception 
> that will simply fall through and hit some code frame above.
> So I suggest we remove sneaky throw from the core entirely. It only exists in 
> two places -- private methods inside Snowball programs invoked via method 
> handles (these don't even declare checked exceptions so I assume they can't 
> occur) and AttributeFactory -- here there is a real possibility somebody 
> could declare an attribute class's constructor that does throw an unchecked 
> exception. In that case I think it is more reasonable to wrap it in a 
> RuntimeException than rethrow it as original.
> Alternatively, we can modify the signature of {{createAttributeInstance}} and 
> {{getStaticImplementation}} to declare some kind of checked exception (either 
> a wrapper or even a Throwable), but I see little reason for it and it'd 
> change the API.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7800) Remove code that potentially rethrows checked exceptions from methods that don't declare them ("sneaky throw" hack)

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010414#comment-16010414
 ] 

Dawid Weiss edited comment on LUCENE-7800 at 5/15/17 12:30 PM:
---

I return to this issue to either resolve it or have it vetoed, because 
otherwise it'll be forgotten forever.

Attached is a patch which removes unchecked-exception throwing hacks from the 
codebase for reasons I already mentioned. The changes are:

1) SnowballProgram throws an AssertionError should an exception happen in 
{{find_among}} (it shouldn't).

2) {{AttributeFactory}} rethrows Error and RuntimeException classes, but wraps 
checked exceptions in a {{AttributeInstantiationException}} that is a subclass 
of {{RuntimeException}}. I made the {{AttributeInstantiationException}} class 
public just in case somebody desperately wanted to instanceof it and get the 
cause.

3) JavascriptCompiler throws throws a {{RuntimeException}} with a cause set to 
{{ParseException}}. The code unwraps such checked parse exceptions in 
{{getAntlrParseTree}} anyway, so this is only an internal change.




was (Author: dweiss):
I return to this issue to either resolve it or have it vetoed, because 
otherwise it'll be forgotted forever.

Attached is a patch which removes unchecked-exception throwing hacks from the 
codebase for reasons I already mentioned. The changes are:

1) SnowballProgram throws an AssertionError should an exception happen in 
{{find_among}} (it shouldn't).

2) {{AttributeFactory}} rethrows Error and RuntimeException classes, but wraps 
checked exceptions in a {{AttributeInstantiationException}} that is a subclass 
of {{RuntimeException}}. I made the {{AttributeInstantiationException}} class 
public just in case somebody desperately wanted to instanceof it and get the 
cause.

3) JavascriptCompiler throws throws a {{RuntimeException}} with a cause set to 
{{ParseException}}. The code unwraps such checked parse exceptions in 
{{getAntlrParseTree}} anyway, so this is only an internal change.



> Remove code that potentially rethrows checked exceptions from methods that 
> don't declare them ("sneaky throw" hack)
> ---
>
> Key: LUCENE-7800
> URL: https://issues.apache.org/jira/browse/LUCENE-7800
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Minor
> Fix For: 6.x, master (7.0)
>
> Attachments: LUCENE-7800.patch, LUCENE-7800.patch
>
>
> For a long time I considered the "sneaky" throw hack to be a nice way of 
> coding around some of Java's limitations (especially with invoking methods 
> via reflection or method handles), but with time I started to see how it can 
> be potentially dangerous and is nearly always confusing. If you have a Java 
> method and its signature doesn't indicate the possibility of a checked 
> exception you, as a programmer, simply don't expect it to happen. Never. So, 
> for example, you could write:
> {code}
> try {
>  luceneApi();
> } catch (RuntimeException | Error e) {
>   // Handle unchecked exceptions here.
> }
> {code}
> and consider the code above to be absolutely bullet-proof in terms of 
> handling exceptions. Unfortunately with sneaky throws anywhere in the 
> "luceneApi" this is no longer the case -- you can receive a checked exception 
> that will simply fall through and hit some code frame above.
> So I suggest we remove sneaky throw from the core entirely. It only exists in 
> two places -- private methods inside Snowball programs invoked via method 
> handles (these don't even declare checked exceptions so I assume they can't 
> occur) and AttributeFactory -- here there is a real possibility somebody 
> could declare an attribute class's constructor that does throw an unchecked 
> exception. In that case I think it is more reasonable to wrap it in a 
> RuntimeException than rethrow it as original.
> Alternatively, we can modify the signature of {{createAttributeInstance}} and 
> {{getStaticImplementation}} to declare some kind of checked exception (either 
> a wrapper or even a Throwable), but I see little reason for it and it'd 
> change the API.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7823) Have a naive bayes classifier which uses plain BM25 scores instead of plain frequencies

[Tommaso Teofili (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili resolved LUCENE-7823.
-
Resolution: Fixed

> Have a naive bayes classifier which uses plain BM25 scores instead of plain 
> frequencies
> ---
>
> Key: LUCENE-7823
> URL: https://issues.apache.org/jira/browse/LUCENE-7823
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/classification
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: master (7.0)
>
>
> {{SimpleNaiveBayesClassifier}} users term frequencies with add one smoothing 
> to calculate likelihood and just tf for prior. Given Lucene has switched to 
> BM25 it would be better to have a different impl which uses BM25 
> scoring as a probability measure of both prior and likelihood.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9546) There is a lot of unnecessary boxing/unboxing going on in {{SolrParams}} class

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-9546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010401#comment-16010401
 ] 

Jan Høydahl commented on SOLR-9546:
---

This jira appears in CHANGES.txt for 6.3.0, please tag appropriately and close.

> There is a lot of unnecessary boxing/unboxing going on in {{SolrParams}} class
> --
>
> Key: SOLR-9546
> URL: https://issues.apache.org/jira/browse/SOLR-9546
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
>Priority: Minor
> Attachments: SOLR-9546_CloudMLTQParser.patch, SOLR-9546.patch
>
>
> Here is an excerpt 
> {code}
>   public Long getLong(String param, Long def) {
> String val = get(param);
> try {
>   return val== null ? def : Long.parseLong(val);
> }
> catch( Exception ex ) {
>   throw new SolrException( SolrException.ErrorCode.BAD_REQUEST, 
> ex.getMessage(), ex );
> }
>   }
> {code}
> {{Long.parseLong()}} returns a primitive type but since method expect to 
> return a {{Long}}, it needs to be wrapped. There are many more method like 
> that. We might be creating a lot of unnecessary objects here.
> I am not sure if JVM catches upto it and somehow optimizes it if these 
> methods are called enough times (or may be compiler does some modifications 
> at compile time)
> Let me know if I am thinking of some premature optimization



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-6.x - Build # 895 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/895/

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.update.HdfsTransactionLog  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.update.HdfsTransactionLog.(HdfsTransactionLog.java:130)  
at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:203)  at 
org.apache.solr.update.UpdateHandler.(UpdateHandler.java:153)  at 
org.apache.solr.update.UpdateHandler.(UpdateHandler.java:110)  at 
org.apache.solr.update.DirectUpdateHandler2.(DirectUpdateHandler2.java:108)
  at sun.reflect.GeneratedConstructorAccessor192.newInstance(Unknown Source)  
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)  at 
org.apache.solr.core.SolrCore.createInstance(SolrCore.java:760)  at 
org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:822)  at 
org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1088)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:947)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:830)  at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:920)  at 
org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:558)  at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
 at java.lang.Thread.run(Thread.java:745)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not 
released!!! [HdfsTransactionLog]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.update.HdfsTransactionLog
at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at 
org.apache.solr.update.HdfsTransactionLog.(HdfsTransactionLog.java:130)
at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:203)
at org.apache.solr.update.UpdateHandler.(UpdateHandler.java:153)
at org.apache.solr.update.UpdateHandler.(UpdateHandler.java:110)
at 
org.apache.solr.update.DirectUpdateHandler2.(DirectUpdateHandler2.java:108)
at sun.reflect.GeneratedConstructorAccessor192.newInstance(Unknown 
Source)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:760)
at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:822)
at org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1088)
at org.apache.solr.core.SolrCore.(SolrCore.java:947)
at org.apache.solr.core.SolrCore.(SolrCore.java:830)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:920)
at 
org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:558)
at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)


at __randomizedtesting.SeedInfo.seed([F958E43157F19911]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:302)
at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.ca

[jira] [Created] (SOLR-10689) migrate in collections api not working

[chandru (JIRA)]

chandru created SOLR-10689:
--

 Summary: migrate in collections api not working
 Key: SOLR-10689
 URL: https://issues.apache.org/jira/browse/SOLR-10689
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Admin UI, clients - java
Affects Versions: 6.5.1
Reporter: chandru


When migrating with the same query that was given in collection api , There was 
no docs that were migrated from A -> B. Please help me to proceed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10678) Clustering can be executed multiple times in distributed mode

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010338#comment-16010338
 ] 

Dawid Weiss commented on SOLR-10678:


I looked at it, here's a summary of my findings. 

1) Clustering isn't currently run twice in distributed mode, so this is a 
non-issue functionally. This is so because a distributed request goes through 
{{modifyRequest}} and {{ClusteringComponent}} removes itself (by disabling the 
{{clustering}} attribute) from subsequent shard requests:
{code}
  public void modifyRequest(ResponseBuilder rb, SearchComponent who, 
ShardRequest sreq) {
SolrParams params = rb.req.getParams();
if (!params.getBool(COMPONENT_NAME, false) || 
!params.getBool(ClusteringParams.USE_SEARCH_RESULTS, false)) {
  return;
}
sreq.params.remove(COMPONENT_NAME);
{code}

This is then checked in {{process}}:
{code}
  public void process(ResponseBuilder rb) throws IOException {
SolrParams params = rb.req.getParams();
if (!params.getBool(COMPONENT_NAME, false)) {
  return;
}
{code}

2. What confused me a *lot* was why {{process}} is invoked during the 
distributed test (and why the test was executed so darn many times). Turned out 
it's because of the default {{ShardsRepeatRule}} and this gem inside inside 
{{BaseDistributedSearchTestCase}}:
{code}
  protected QueryResponse query(boolean setDistribParams, SolrParams p) throws 
Exception {
final ModifiableSolrParams params = new ModifiableSolrParams(p);
// TODO: look into why passing true causes fails
params.set("distrib", "false");
final QueryResponse controlRsp = controlClient.query(params);
validateControlData(controlRsp);
params.remove("distrib");
{code}

So the distributed test is running a forced-non-distributed request first, 
followed by the distributed request, hence the confusing logs.

> Clustering can be executed multiple times in distributed mode
> -
>
> Key: SOLR-10678
> URL: https://issues.apache.org/jira/browse/SOLR-10678
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Minor
>
> As reported on SO: 
> http://stackoverflow.com/questions/43877284/how-does-solr-clustering-component-work/43937064#43937064



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org