date:20130911

Mark Miller created SOLR-5232:
-

 Summary: SolrCloud should distribute updates via streaming rather 
than buffering.
 Key: SOLR-5232
 URL: https://issues.apache.org/jira/browse/SOLR-5232
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.5, 5.0


The current approach was never the best for SolrCloud - it was designed for a 
pre SolrCloud Solr - it also uses too many connections and threads - nailing 
that down is likely wasted effort when we should really move away from 
explicitly buffering docs and sending small batches per thread as we have been 
doing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5232) SolrCloud should distribute updates via streaming rather than buffering.


 [ 
https://issues.apache.org/jira/browse/SOLR-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-5232:
--

Attachment: SOLR-5232.patch

I hacked together an initial patch that uses a ConcurrentSolrServer per url 
with 1 thread and a queue of 10. I originally played around with sharing them 
across update requests as a static, but it ends up being difficult to wait 
until you know the updates are accepted, so I abandoned that.

 SolrCloud should distribute updates via streaming rather than buffering.
 

 Key: SOLR-5232
 URL: https://issues.apache.org/jira/browse/SOLR-5232
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.5, 5.0

 Attachments: SOLR-5232.patch


 The current approach was never the best for SolrCloud - it was designed for a 
 pre SolrCloud Solr - it also uses too many connections and threads - nailing 
 that down is likely wasted effort when we should really move away from 
 explicitly buffering docs and sending small batches per thread as we have 
 been doing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5216) Document updates to SolrCloud can cause a distributed deadlock.


[ 
https://issues.apache.org/jira/browse/SOLR-5216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764057#comment-13764057
 ] 

Mark Miller commented on SOLR-5216:
---

I think the only thing that this patch could possibly do is eliminate the 
deadlock, but allow for more threads. I only expect up to a 2x max use in 
threads, but it could be off and allow for more than that if there is some 
small bug I'm missing. In any case, I'm sure the idea is the right one for the 
deadlock. I worry the problems you get after many hours might be due to the 
sheer number of threads and requests. I worry about spending too much time 
trying to get this solution working though - it might be energy better spent 
changing things towards a better direction - SOLR-5232: SolrCloud should 
distribute updates via streaming rather than buffering.

 Document updates to SolrCloud can cause a distributed deadlock.
 ---

 Key: SOLR-5216
 URL: https://issues.apache.org/jira/browse/SOLR-5216
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
Priority: Critical
 Fix For: 4.5, 5.0

 Attachments: SOLR-5216.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: org.apache.lucene.index.CorruptIndexException: Corrupted: docID=582140, docBase=0, chunkDocs=0, numDocs=649475

The luke request handler and your JaccardSimilarityQuery use the same
API to retrieve documents (according to the stack trace) so I'm
surprised that you see errors in one case only. Are you sure you are
looking at the same documents? If background merges happened between
your M/R job and your request to the luke request handler, it could be
that document IDs have been re-assigned. Ideally, you should run
CheckIndex[1] on your indices to make sure everything is fine.

[1] 
http://lucene.apache.org/core/4_3_1/core/org/apache/lucene/index/CheckIndex.html

-- 
Adrien

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5232) SolrCloud should distribute updates via streaming rather than buffering.

2013-09-11 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-5232:
--

Attachment: SOLR-5232.patch

Another quick pass - biggest change is sharing thread executor across requests 
to reduce the need to spin up new threads.

 SolrCloud should distribute updates via streaming rather than buffering.
 

 Key: SOLR-5232
 URL: https://issues.apache.org/jira/browse/SOLR-5232
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.5, 5.0

 Attachments: SOLR-5232.patch, SOLR-5232.patch


 The current approach was never the best for SolrCloud - it was designed for a 
 pre SolrCloud Solr - it also uses too many connections and threads - nailing 
 that down is likely wasted effort when we should really move away from 
 explicitly buffering docs and sending small batches per thread as we have 
 been doing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5233) admin/collections?action=DELETESHARD broken

2013-09-11 Thread Christine Poerschke (JIRA)

Christine Poerschke created SOLR-5233:
-

 Summary: admin/collections?action=DELETESHARD broken
 Key: SOLR-5233
 URL: https://issues.apache.org/jira/browse/SOLR-5233
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.4
Reporter: Christine Poerschke


The problem we saw was that deleting a shard did not actually delete it, 
although it semi-claimed to have done so.
{noformat} 
...
o.a.s.c.OverseerCollectionProcessor [OverseerCollectionProcessor.java:723]
Delete shard invoked
...
o.a.s.c.OverseerCollectionProcessor [OverseerCollectionProcessor.java:781]
Successfully deleted collection collection1, shard: null
...
{noformat} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5233) admin/collections?action=DELETESHARD broken

2013-09-11 Thread Christine Poerschke (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764123#comment-13764123
 ] 

Christine Poerschke commented on SOLR-5233:
---

From looking at the code, OverseerCollectionProcessor.java, in deleteShard 
it's a copy/paste problem and a few of the _message.getStr(name)_ need to be 
_sliceId_ and _sliceId_ as well as _collection_ needs to be passed as part of 
the _Overseer.REMOVESHARD_ operation.

 admin/collections?action=DELETESHARD broken
 ---

 Key: SOLR-5233
 URL: https://issues.apache.org/jira/browse/SOLR-5233
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.4
Reporter: Christine Poerschke

 The problem we saw was that deleting a shard did not actually delete it, 
 although it semi-claimed to have done so.
 {noformat} 
 ...
 o.a.s.c.OverseerCollectionProcessor [OverseerCollectionProcessor.java:723]
 Delete shard invoked
 ...
 o.a.s.c.OverseerCollectionProcessor [OverseerCollectionProcessor.java:781]
 Successfully deleted collection collection1, shard: null
 ...
 {noformat} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5234) Allow SolrResourceLoader to load resources from URLs

2013-09-11 Thread Alan Woodward (JIRA)

Alan Woodward created SOLR-5234:
---

 Summary: Allow SolrResourceLoader to load resources from URLs
 Key: SOLR-5234
 URL: https://issues.apache.org/jira/browse/SOLR-5234
 Project: Solr
  Issue Type: Improvement
Reporter: Alan Woodward
Assignee: Alan Woodward
Priority: Minor


This would allow multiple solr instance to share large configuration files.  It 
would also help resolve problems caused by attempting to store 1Mb files in 
zookeeper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5230) Call DelegatingCollector.finish() during grouping

2013-09-11 Thread Simon Endele (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764157#comment-13764157
 ] 

Simon Endele commented on SOLR-5230:


Applied the patch and it seems to work at a first glance.
Thank you very much for your quick reaction on 
https://issues.apache.org/jira/browse/SOLR-5020, Joel!

But for some scenarios (e.g. expensive post-filters) it might be a drawback 
that the phases cannot be distinguished in the finish() method.
What do you think about introducing a second method 
DelegatingCollector.finishAfterGrouping() or similar that is called in the 
second phase instead of finish()?

 Call DelegatingCollector.finish() during grouping
 -

 Key: SOLR-5230
 URL: https://issues.apache.org/jira/browse/SOLR-5230
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 4.4
Reporter: Joel Bernstein
Priority: Minor
 Fix For: 4.5, 5.0

 Attachments: SOLR-5230.patch


 This is an add-on to SOLR-5020 to call the new DelegatingCollector.finish() 
 method from inside the grouping flow. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5234) Allow SolrResourceLoader to load resources from URLs

2013-09-11 Thread Alan Woodward (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated SOLR-5234:


Attachment: SOLR-5234.patch

Quick n dirty patch.  It doesn't appear to break anything...

Unresolved questions:
- I'm not sure how best to test the new functionality.  Do the Jenkins boxes 
even allow access to the internet?
- Interactions with SOLR-4882.  Maybe I should just disallow URLs that begin 
with file://?

 Allow SolrResourceLoader to load resources from URLs
 

 Key: SOLR-5234
 URL: https://issues.apache.org/jira/browse/SOLR-5234
 Project: Solr
  Issue Type: Improvement
Reporter: Alan Woodward
Assignee: Alan Woodward
Priority: Minor
 Attachments: SOLR-5234.patch


 This would allow multiple solr instance to share large configuration files.  
 It would also help resolve problems caused by attempting to store 1Mb files 
 in zookeeper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-5230) Call DelegatingCollector.finish() during grouping

2013-09-11 Thread Simon Endele (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764157#comment-13764157
 ] 

Simon Endele edited comment on SOLR-5230 at 9/11/13 10:26 AM:
--

Applied the patch and it seems to work at a first glance.
Thank you very much for your quick reaction on SOLR-5020, Joel!

But for some scenarios (e.g. expensive post-filters) it might be a drawback 
that the phases cannot be distinguished in the finish() method.
What do you think about introducing a second method 
DelegatingCollector.finishAfterGrouping() or similar that is called in the 
second phase instead of finish()?

  was (Author: simon.endele):
Applied the patch and it seems to work at a first glance.
Thank you very much for your quick reaction on 
https://issues.apache.org/jira/browse/SOLR-5020, Joel!

But for some scenarios (e.g. expensive post-filters) it might be a drawback 
that the phases cannot be distinguished in the finish() method.
What do you think about introducing a second method 
DelegatingCollector.finishAfterGrouping() or similar that is called in the 
second phase instead of finish()?
  
 Call DelegatingCollector.finish() during grouping
 -

 Key: SOLR-5230
 URL: https://issues.apache.org/jira/browse/SOLR-5230
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 4.4
Reporter: Joel Bernstein
Priority: Minor
 Fix For: 4.5, 5.0

 Attachments: SOLR-5230.patch


 This is an add-on to SOLR-5020 to call the new DelegatingCollector.finish() 
 method from inside the grouping flow. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (LUCENE-4768) Child Traversable To Parent Block Join Query

2013-09-11 Thread Vadim Kirilchuk (JIRA)

[
https://issues.apache.org/jira/browse/LUCENE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vadim Kirilchuk closed LUCENE-4768.
---

Resolution: Won't Fix

As i didn't came up with a more meaningful example, i am closing the issue.
Thanks for your attention.

Child Traversable To Parent Block Join Query

Key: LUCENE-4768
URL: https://issues.apache.org/jira/browse/LUCENE-4768
Project: Lucene - Core
Issue Type: Improvement
Components: core/query/scoring
Environment: trunk
git rev-parse HEAD
5cc88eaa41eb66236a0d4203cc81f1eed97c9a41
Reporter: Vadim Kirilchuk
Attachments: LUCENE-4768-draft.patch

Hi everyone!
Let me describe what i am trying to do:
I have hierarchical documents ('car model' as parent, 'trim' as child) and
use block join queries to retrieve them. However, i am not happy with current
behavior of ToParentBlockJoinQuery which goes through all parent childs
during nextDoc call (accumulating scores and freqs).
Consider the following example, you have a query with a custom post condition
on top of such bjq: and during post condition you traverse scorers tree
(doc-at-time) and want to manually push child scorers of bjq one by one until
condition passes or current parent have no more childs.
I am attaching the patch with query(and some tests) similar to
ToParentBlockJoin but with an ability to traverse childs. (i have to do weird
instance of check and cast inside my code) This is a draft only and i will be
glad to hear if someone need it or to hear how we can improve it.
P.s i believe that proposed query is more generic (low level) than
ToParentBJQ and ToParentBJQ can be extended from it and call nextChild()
internally during nextDoc().
Also, i think that the problem of traversing hierarchical documents is more
complex as lucene have only nextDoc API. What do you think about making api
more hierarchy aware? One level document is a special case of multi level
document but not vice versa. WDYT?
Thanks in advance.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-5233) admin/collections?action=DELETESHARD broken

2013-09-11 Thread Shalin Shekhar Mangar (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar reassigned SOLR-5233:
---

Assignee: Shalin Shekhar Mangar

 admin/collections?action=DELETESHARD broken
 ---

 Key: SOLR-5233
 URL: https://issues.apache.org/jira/browse/SOLR-5233
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.4
Reporter: Christine Poerschke
Assignee: Shalin Shekhar Mangar

 The problem we saw was that deleting a shard did not actually delete it, 
 although it semi-claimed to have done so.
 {noformat} 
 ...
 o.a.s.c.OverseerCollectionProcessor [OverseerCollectionProcessor.java:723]
 Delete shard invoked
 ...
 o.a.s.c.OverseerCollectionProcessor [OverseerCollectionProcessor.java:781]
 Successfully deleted collection collection1, shard: null
 ...
 {noformat} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5189) Numeric DocValues Updates

2013-09-11 Thread Shai Erera (JIRA)

[
https://issues.apache.org/jira/browse/LUCENE-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shai Erera updated LUCENE-5189:
---

Attachment: LUCENE-5189.patch

I've been busy hunting down concurrency bugs and making field updates work with
index sorting (thought it will be an easy nocommit to handle... boy, I was
wrong!). Patch adds field updates to TestSortingMP as well as improves
TestNumericDVUpdates.testSegmentMerges to stress that more.

There are some {{nocommit (RENAME)}} which I'd love to get some feedback on.
Also there are two standing out nocommits around FieldInfos.gen and an
optimization to SegmentCoreReaders/SegmentReader around reusing DVProducers of
update gens. I think these two can be handled in separate issues, just to get
this issue focused.

I'd like to commit this to trunk, so that I can move on to the other issues.
Therefore I'd appreciate some review on the patch.

Numeric DocValues Updates
-

Key: LUCENE-5189
URL: https://issues.apache.org/jira/browse/LUCENE-5189
Project: Lucene - Core
Issue Type: New Feature
Components: core/index
Reporter: Shai Erera
Assignee: Shai Erera
Attachments: LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch,
LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch,
LUCENE-5189.patch

In LUCENE-4258 we started to work on incremental field updates, however the
amount of changes are immense and hard to follow/consume. The reason is that
we targeted postings, stored fields, DV etc., all from the get go.
I'd like to start afresh here, with numeric-dv-field updates only. There are
a couple of reasons to that:
* NumericDV fields should be easier to update, if e.g. we write all the
values of all the documents in a segment for the updated field (similar to
how livedocs work, and previously norms).
* It's a fairly contained issue, attempting to handle just one data type to
update, yet requires many changes to core code which will also be useful for
updating other data types.
* It has value in and on itself, and we don't need to allow updating all the
data types in Lucene at once ... we can do that gradually.
I have some working patch already which I'll upload next, explaining the
changes.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5232) SolrCloud should distribute updates via streaming rather than buffering.

2013-09-11 Thread Erick Erickson (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764215#comment-13764215
 ] 

Erick Erickson commented on SOLR-5232:
--

Mark:

Just so it's recorded, there has been anecdotal evidence that the batch size of 
10 we were using is a bottleneck, but there were reasons to not bump it up. Do 
you expect streaming to alter the throughput here?

 SolrCloud should distribute updates via streaming rather than buffering.
 

 Key: SOLR-5232
 URL: https://issues.apache.org/jira/browse/SOLR-5232
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.5, 5.0

 Attachments: SOLR-5232.patch, SOLR-5232.patch


 The current approach was never the best for SolrCloud - it was designed for a 
 pre SolrCloud Solr - it also uses too many connections and threads - nailing 
 that down is likely wasted effort when we should really move away from 
 explicitly buffering docs and sending small batches per thread as we have 
 been doing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5202) Support easier overrides of Carrot2 clustering attributes via XML data sets exported from the Workbench.


[ 
https://issues.apache.org/jira/browse/SOLR-5202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764216#comment-13764216
 ] 

ASF subversion and git services commented on SOLR-5202:
---

Commit 1521804 from [~dawidweiss] in branch 'dev/trunk'
[ https://svn.apache.org/r1521804 ]

SOLR-5202: cleaned up comments surrounding clustering component. Left all the 
algorithms in there, but added a pointer to integration-strategies which 
includes more exhaustive configuration examples.

 Support easier overrides of Carrot2 clustering attributes via XML data sets 
 exported from the Workbench.
 

 Key: SOLR-5202
 URL: https://issues.apache.org/jira/browse/SOLR-5202
 Project: Solr
  Issue Type: New Feature
Reporter: Dawid Weiss
Assignee: Dawid Weiss
 Fix For: 4.5, 5.0

 Attachments: SOLR-5202.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-4.x - Build # 107 - Still Failing

2013-09-11 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-4.x/107/

No tests ran.

Build Log:
[...truncated 34293 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease
 [copy] Copying 416 files to 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease/lucene
 [copy] Copying 194 files to 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease/solr
 [exec] JAVA6_HOME is /home/hudson/tools/java/latest1.6
 [exec] JAVA7_HOME is /home/hudson/tools/java/latest1.7
 [exec] NOTE: output encoding is US-ASCII
 [exec] 
 [exec] Load release URL 
file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease/...
 [exec] 
 [exec] Test Lucene...
 [exec]   test basics...
 [exec]   get KEYS
 [exec] 0.1 MB in 0.02 sec (4.9 MB/sec)
 [exec]   check changes HTML...
 [exec]   download lucene-4.5.0-src.tgz...
 [exec] 27.1 MB in 0.04 sec (659.7 MB/sec)
 [exec] verify md5/sha1 digests
 [exec]   download lucene-4.5.0.tgz...
 [exec] 49.1 MB in 0.07 sec (661.3 MB/sec)
 [exec] verify md5/sha1 digests
 [exec]   download lucene-4.5.0.zip...
 [exec] 58.9 MB in 0.09 sec (646.8 MB/sec)
 [exec] verify md5/sha1 digests
 [exec]   unpack lucene-4.5.0.tgz...
 [exec] verify JAR/WAR metadata...
 [exec] test demo with 1.6...
 [exec]   got 5723 hits for query lucene
 [exec] test demo with 1.7...
 [exec]   got 5723 hits for query lucene
 [exec] check Lucene's javadoc JAR
 [exec] 
 [exec] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeReleaseTmp/unpack/lucene-4.5.0/docs/core/org/apache/lucene/util/AttributeSource.html
 [exec]   broken details HTML: Method Detail: addAttributeImpl: closing 
/code does not match opening T
 [exec]   broken details HTML: Method Detail: getAttribute: closing 
/code does not match opening T
 [exec] Traceback (most recent call last):
 [exec]   File 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py,
 line 1450, in module
 [exec] main()
 [exec]   File 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py,
 line 1394, in main
 [exec] smokeTest(baseURL, svnRevision, version, tmpDir, isSigned, 
testArgs)
 [exec]   File 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py,
 line 1431, in smokeTest
 [exec] unpackAndVerify('lucene', tmpDir, artifact, svnRevision, 
version, testArgs)
 [exec]   File 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py,
 line 607, in unpackAndVerify
 [exec] verifyUnpacked(project, artifact, unpackPath, svnRevision, 
version, testArgs)
 [exec]   File 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py,
 line 786, in verifyUnpacked
 [exec] checkJavadocpath('%s/docs' % unpackPath)
 [exec]   File 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py,
 line 904, in checkJavadocpath
 [exec] raise RuntimeError('missing javadocs package summaries!')
 [exec] RuntimeError: missing javadocs package summaries!

BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/build.xml:321:
 exec returned: 1

Total time: 20 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Lucene/Solr 4.5

2013-09-11 Thread ASF subversion and git services (JIRA)

Hi,

I was looking at the changelogs for Lucene and Solr and I think they
look pretty good. What would you think about realeasing Lucene/Solr
4.5, are there issues you would like to get in before the release?

-- 
Adrien

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-26) Wildcard query only accepts 1 or more characters for prefixes (not 0 or more)


[ 
https://issues.apache.org/jira/browse/LUCENE-26?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764321#comment-13764321
 ] 

ASF subversion and git services commented on LUCENE-26:
---

Commit 1521851 from [~vajda] in branch 'pylucene/trunk'
[ https://svn.apache.org/r1521851 ]

fixed bug PYLUCENE-26 (Martin Anon)

 Wildcard query only accepts 1 or more characters for prefixes (not 0 or more)
 -

 Key: LUCENE-26
 URL: https://issues.apache.org/jira/browse/LUCENE-26
 Project: Lucene - Core
  Issue Type: Bug
  Components: core/search
 Environment: Operating System: Linux
 Platform: PC
Reporter: Lee Mallabone
Assignee: Lucene Developers
 Attachments: ASF.LICENSE.NOT.GRANTED--WildcardTermEnum.java.diff, 
 ASF.LICENSE.NOT.GRANTED--WildcardTest.java


 When using the WildcardQuery class to do a wildcard search with the wildcard 
 at
 the end of the string, the wildcard is treated as 1 or more characters. This
 is inconsistent with other wildcard behaviour, and undesirable. The correct
 behaviour is for the wildcard to represent 0 or more characters.
 The error is due to a problem in the wildcard comparison method in
 org.apache.lucene.search.WildcardTermEnum.
 Patch to follow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (PYLUCENE-26) extern keyword missing in reserved keywords in JCC cpp.py unit

2013-09-11 Thread Andi Vajda (JIRA)


 [ 
https://issues.apache.org/jira/browse/PYLUCENE-26?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda resolved PYLUCENE-26.


Resolution: Fixed

rev 1521851
Thank you for the patch.


 extern keyword missing in reserved keywords in JCC cpp.py unit
 

 Key: PYLUCENE-26
 URL: https://issues.apache.org/jira/browse/PYLUCENE-26
 Project: PyLucene
  Issue Type: Bug
 Environment: jcc-svn-head
Reporter: Martin
Priority: Minor
  Labels: patch
 Attachments: extern.patch

   Original Estimate: 5m
  Remaining Estimate: 5m

 The python unit cpp.py misses the reserved c++ keyword extern, which leads 
 to problems

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (SOLR-5230) Call DelegatingCollector.finish() during grouping


[ 
https://issues.apache.org/jira/browse/SOLR-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764330#comment-13764330
 ] 

Joel Bernstein commented on SOLR-5230:
--

After reviewing the code, I believe the finish() method needs to be called in 
both phases. It looks like in Phase1 the group heads are collected and in 
phase2 the sub-documents are collected. In both situation the postfilter needs 
to be applied and finish called.

 Call DelegatingCollector.finish() during grouping
 -

 Key: SOLR-5230
 URL: https://issues.apache.org/jira/browse/SOLR-5230
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 4.4
Reporter: Joel Bernstein
Priority: Minor
 Fix For: 4.5, 5.0

 Attachments: SOLR-5230.patch


 This is an add-on to SOLR-5020 to call the new DelegatingCollector.finish() 
 method from inside the grouping flow. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-5230) Call DelegatingCollector.finish() during grouping

[
https://issues.apache.org/jira/browse/SOLR-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764330#comment-13764330
]

Joel Bernstein edited comment on SOLR-5230 at 9/11/13 2:05 PM:
---

After reviewing the code, I believe the finish() method needs to be called in
both phases. It looks like in Phase1 the group heads are collected and in
phase2 the sub-documents are collected. In both situations the postfilter needs
to be applied and finish called.

was (Author: joel.bernstein):
After reviewing the code, I believe the finish() method needs to be called
in both phases. It looks like in Phase1 the group heads are collected and in
phase2 the sub-documents are collected. In both situation the postfilter needs
to be applied and finish called.

Call DelegatingCollector.finish() during grouping
-

Key: SOLR-5230
URL: https://issues.apache.org/jira/browse/SOLR-5230
Project: Solr
Issue Type: New Feature
Components: search
Affects Versions: 4.4
Reporter: Joel Bernstein
Priority: Minor
Fix For: 4.5, 5.0

Attachments: SOLR-5230.patch

This is an add-on to SOLR-5020 to call the new DelegatingCollector.finish()
method from inside the grouping flow.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5232) SolrCloud should distribute updates via streaming rather than buffering.

2013-09-11 Thread Yonik Seeley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764351#comment-13764351
 ] 

Yonik Seeley commented on SOLR-5232:


bq. there has been anecdotal evidence that the batch size of 10 we were using 
is a bottleneck

batch size != queue size... and it's not even clear the queue does anything 
unless one is somehow producing documents in a very bursty fashion.
Streaming rather than buffering is definitely the way to go.

 SolrCloud should distribute updates via streaming rather than buffering.
 

 Key: SOLR-5232
 URL: https://issues.apache.org/jira/browse/SOLR-5232
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.5, 5.0

 Attachments: SOLR-5232.patch, SOLR-5232.patch


 The current approach was never the best for SolrCloud - it was designed for a 
 pre SolrCloud Solr - it also uses too many connections and threads - nailing 
 that down is likely wasted effort when we should really move away from 
 explicitly buffering docs and sending small batches per thread as we have 
 been doing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5232) SolrCloud should distribute updates via streaming rather than buffering.


 [ 
https://issues.apache.org/jira/browse/SOLR-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-5232:
--

Attachment: SOLR-5232.patch

This patch adds back in the retry on forwarding to leader logic (had hacked it 
out initially) and a new simple test for retries.

 SolrCloud should distribute updates via streaming rather than buffering.
 

 Key: SOLR-5232
 URL: https://issues.apache.org/jira/browse/SOLR-5232
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.5, 5.0

 Attachments: SOLR-5232.patch, SOLR-5232.patch, SOLR-5232.patch


 The current approach was never the best for SolrCloud - it was designed for a 
 pre SolrCloud Solr - it also uses too many connections and threads - nailing 
 that down is likely wasted effort when we should really move away from 
 explicitly buffering docs and sending small batches per thread as we have 
 been doing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4816) Add document routing to CloudSolrServer


[ 
https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764404#comment-13764404
 ] 

Mark Miller commented on SOLR-4816:
---

Some recent jenkins failures in org.apache.solr.cloud.SyncSliceTest to look at: 
https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java6/

 Add document routing to CloudSolrServer
 ---

 Key: SOLR-4816
 URL: https://issues.apache.org/jira/browse/SOLR-4816
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Affects Versions: 4.3
Reporter: Joel Bernstein
Assignee: Mark Miller
Priority: Minor
 Fix For: 4.5, 5.0

 Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816-sriesenberg.patch


 This issue adds the following enhancements to CloudSolrServer's update logic:
 1) Document routing: Updates are routed directly to the correct shard leader 
 eliminating document routing at the server.
 2) Optional parallel update execution: Updates for each shard are executed in 
 a separate thread so parallel indexing can occur across the cluster.
 These enhancements should allow for near linear scalability on indexing 
 throughput.
 Usage:
 CloudSolrServer cloudClient = new CloudSolrServer(zkAddress);
 cloudClient.setParallelUpdates(true); 
 SolrInputDocument doc1 = new SolrInputDocument();
 doc1.addField(id, 0);
 doc1.addField(a_t, hello1);
 SolrInputDocument doc2 = new SolrInputDocument();
 doc2.addField(id, 2);
 doc2.addField(a_t, hello2);
 UpdateRequest request = new UpdateRequest();
 request.add(doc1);
 request.add(doc2);
 request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false);
 NamedList response = cloudClient.request(request); // Returns a backwards 
 compatible condensed response.
 //To get more detailed response down cast to RouteResponse:
 CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response;

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5235) Update Log replay does not use the processor chain for commit


[ 
https://issues.apache.org/jira/browse/SOLR-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764381#comment-13764381
 ] 

ludovic Boutros commented on SOLR-5235:
---

The portion code of UpdateLog.java for reference :

{code:title=UpdateLog.java|borderStyle=solid}
CommitUpdateCommand cmd = new CommitUpdateCommand(req, false);
cmd.setVersion(commitVersion);
cmd.softCommit = false;
cmd.waitSearcher = true;
cmd.setFlags(UpdateCommand.REPLAY);
try {
  if (debug) log.debug(commit  +  cmd);
  uhandler.commit(cmd);  // this should cause a commit to be 
added to the incomplete log and avoid it being replayed again after a restart.
} catch (IOException ex) {
  recoveryInfo.errors++;
  loglog.error(Replay exception: final commit., ex);
}
{code} 

 Update Log replay does not use the processor chain for commit
 -

 Key: SOLR-5235
 URL: https://issues.apache.org/jira/browse/SOLR-5235
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.3.1, 4.4
Reporter: ludovic Boutros

 The update log replay process commit commands directly with the Update 
 Handler.
 The update processor chain is not used. I may be wrong but I think this is to 
 prevent to log this commit command again in the LogUpdateProcessor.
 But this commit command is flagged with the flag UpdateCommand.REPLAY. I 
 think this flag should be checked in the LogUpdateProcessor in order to adapt 
 its behavior.
 Currently, commit actions in custom Update Processors are not applied in case 
 of a crash without an explicit commit.
 A workaround can be done with the finish function but this is not ideal.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5230) Call DelegatingCollector.finish() during grouping

[
https://issues.apache.org/jira/browse/SOLR-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764412#comment-13764412
]

Joel Bernstein commented on SOLR-5230:
--

This is the way it likely has to be. Both of the grouping phases need to call
the postFilter to return the proper results. And if the postFilter is called,
the finish() needs to be called.

For expensive postFilters, this is not ideal. But, that's the implementation.

Take a look at SOLR-5027 which was the original reason why the finish() method
was added to DelegatingCollector. This an alternative to field collapsing and
grouping which performs much better then ngroups and group.truncate combined.

SOLR-5027 will be getting a lot of my attention this month so any feedback you
have on this would be appreciated.

Call DelegatingCollector.finish() during grouping
-

Attachments: SOLR-5230.patch

This is an add-on to SOLR-5020 to call the new DelegatingCollector.finish()
method from inside the grouping flow.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5230) Call DelegatingCollector.finish() during grouping

2013-09-11 Thread Simon Endele (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764390#comment-13764390
 ] 

Simon Endele commented on SOLR-5230:


Hm, I'm quite sure that collect() is called for all docs in both phases.

Excerpt from my final result:
lst name=grouped
  lst name=group_id
int name=matches61/int
int name=ngroups35/int
arr name=groups
  [...]

And collect() is called twice 61 times, followed by a call of finish() each.

 Call DelegatingCollector.finish() during grouping
 -

 Key: SOLR-5230
 URL: https://issues.apache.org/jira/browse/SOLR-5230
 Project: Solr
  Issue Type: New Feature
  Components: search
Affects Versions: 4.4
Reporter: Joel Bernstein
Priority: Minor
 Fix For: 4.5, 5.0

 Attachments: SOLR-5230.patch


 This is an add-on to SOLR-5020 to call the new DelegatingCollector.finish() 
 method from inside the grouping flow. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5235) Update Log replay does not use the processor chain for commit


 [ 
https://issues.apache.org/jira/browse/SOLR-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ludovic Boutros updated SOLR-5235:
--

Summary: Update Log replay does not use the processor chain for commit  
(was: Update Log replay does use the processor chain for commit)

 Update Log replay does not use the processor chain for commit
 -

 Key: SOLR-5235
 URL: https://issues.apache.org/jira/browse/SOLR-5235
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.3.1, 4.4
Reporter: ludovic Boutros

 The update log replay process commit commands directly with the Update 
 Handler.
 The update processor chain is not used. I may be wrong but I think this is to 
 prevent to log this commit command again in the LogUpdateProcessor.
 But this commit command is flagged with the flag UpdateCommand.REPLAY. I 
 think this flag should be checked in the LogUpdateProcessor in order to adapt 
 its behavior.
 Currently, commit actions in custom Update Processors are not applied in case 
 of a crash without an explicit commit.
 A workaround can be done with the finish function but this is not ideal.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5235) Update Log replay does use the processor chain for commit

ludovic Boutros created SOLR-5235:
-

 Summary: Update Log replay does use the processor chain for commit
 Key: SOLR-5235
 URL: https://issues.apache.org/jira/browse/SOLR-5235
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.4, 4.3.1
Reporter: ludovic Boutros


The update log replay process commit commands directly with the Update Handler.
The update processor chain is not used. I may be wrong but I think this is to 
prevent to log this commit command again in the LogUpdateProcessor.

But this commit command is flagged with the flag UpdateCommand.REPLAY. I think 
this flag should be checked in the LogUpdateProcessor in order to adapt its 
behavior.

Currently, commit actions in custom Update Processors are not applied in case 
of a crash without an explicit commit.
A workaround can be done with the finish function but this is not ideal.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 4.5

2013-09-11 Thread Michael McCandless

+1 to release 4.5.

Mike McCandless

http://blog.mikemccandless.com


On Wed, Sep 11, 2013 at 9:40 AM, Adrien Grand jpou...@gmail.com wrote:
 Hi,

 I was looking at the changelogs for Lucene and Solr and I think they
 look pretty good. What would you think about realeasing Lucene/Solr
 4.5, are there issues you would like to get in before the release?

 --
 Adrien

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Issue Comment Deleted] (SOLR-4816) Add document routing to CloudSolrServer


 [ 
https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-4816:
--

Comment: was deleted

(was: Commit 1521726 from [~markrmil...@gmail.com] in branch 
'dev/branches/branch_4x'
[ https://svn.apache.org/r1521726 ]

SOLR-4816: CloudSolrServer can now route updates locally and no longer relies 
on inter-node update forwarding.
SOLR-3249: Allow CloudSolrServer and SolrCmdDistributor to use JavaBin.
SOLR-4816: CloudSolrServer now uses multiple threads to send updates by 
default.)

 Add document routing to CloudSolrServer
 ---

 Key: SOLR-4816
 URL: https://issues.apache.org/jira/browse/SOLR-4816
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Affects Versions: 4.3
Reporter: Joel Bernstein
Assignee: Mark Miller
Priority: Minor
 Fix For: 4.5, 5.0

 Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816-sriesenberg.patch


 This issue adds the following enhancements to CloudSolrServer's update logic:
 1) Document routing: Updates are routed directly to the correct shard leader 
 eliminating document routing at the server.
 2) Optional parallel update execution: Updates for each shard are executed in 
 a separate thread so parallel indexing can occur across the cluster.
 These enhancements should allow for near linear scalability on indexing 
 throughput.
 Usage:
 CloudSolrServer cloudClient = new CloudSolrServer(zkAddress);
 cloudClient.setParallelUpdates(true); 
 SolrInputDocument doc1 = new SolrInputDocument();
 doc1.addField(id, 0);
 doc1.addField(a_t, hello1);
 SolrInputDocument doc2 = new SolrInputDocument();
 doc2.addField(id, 2);
 doc2.addField(a_t, hello2);
 UpdateRequest request = new UpdateRequest();
 request.add(doc1);
 request.add(doc2);
 request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false);
 NamedList response = cloudClient.request(request); // Returns a backwards 
 compatible condensed response.
 //To get more detailed response down cast to RouteResponse:
 CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response;

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 4.5

2013-09-11 Thread Erick Erickson

I think Jack's idea of setting a target date N weeks out is a good one.
People commonly have some tidying up they'd like to do and a little
time-frame between now and cutting a release would allow that to
happen. In particular the just-committed SOLR-4816 and associated
would have a bit more time to bake.

Straw man, 1-Oct for cutting the release... Nice end-of-quarter date :)

FWIW,
Erick

But +1 to releasing a 4.5 Real Soon Now.




On Wed, Sep 11, 2013 at 12:01 PM, Jack Krupansky j...@basetechnology.comwrote:

 There seems to be a modest amount of Solr work in progress that probably
 wants to be in Solr 4.5. I would suggest another two weeks for Solr work to
 finish and bake. Maybe cut an RC0 on, say, 9/23. Or maybe things will have
 settled down in a week and it would be worth an RC0 in a week, with the
 risk of an RC1 a few days or a week later.

 Otherwise, +1.

 -- Jack Krupansky

 -Original Message- From: Adrien Grand
 Sent: Wednesday, September 11, 2013 9:40 AM
 To: dev@lucene.apache.org
 Subject: Lucene/Solr 4.5


 Hi,

 I was looking at the changelogs for Lucene and Solr and I think they
 look pretty good. What would you think about realeasing Lucene/Solr
 4.5, are there issues you would like to get in before the release?

 --
 Adrien

 --**--**-
 To unsubscribe, e-mail: 
 dev-unsubscribe@lucene.apache.**orgdev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

 --**--**-
 To unsubscribe, e-mail: 
 dev-unsubscribe@lucene.apache.**orgdev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 4.5

2013-09-11 Thread Robert Muir

I think its terrible.

Don't commit things to the stable branch_4x unless they are ready to
be released: if features need baking do that shit in trunk.

+1 to release 4.5 Right Now.

On Wed, Sep 11, 2013 at 9:20 AM, Erick Erickson erickerick...@gmail.com wrote:
 I think Jack's idea of setting a target date N weeks out is a good one.
 People commonly have some tidying up they'd like to do and a little
 time-frame between now and cutting a release would allow that to
 happen. In particular the just-committed SOLR-4816 and associated
 would have a bit more time to bake.

 Straw man, 1-Oct for cutting the release... Nice end-of-quarter date :)

 FWIW,
 Erick

 But +1 to releasing a 4.5 Real Soon Now.




 On Wed, Sep 11, 2013 at 12:01 PM, Jack Krupansky j...@basetechnology.com
 wrote:

 There seems to be a modest amount of Solr work in progress that probably
 wants to be in Solr 4.5. I would suggest another two weeks for Solr work to
 finish and bake. Maybe cut an RC0 on, say, 9/23. Or maybe things will have
 settled down in a week and it would be worth an RC0 in a week, with the risk
 of an RC1 a few days or a week later.

 Otherwise, +1.

 -- Jack Krupansky

 -Original Message- From: Adrien Grand
 Sent: Wednesday, September 11, 2013 9:40 AM
 To: dev@lucene.apache.org
 Subject: Lucene/Solr 4.5


 Hi,

 I was looking at the changelogs for Lucene and Solr and I think they
 look pretty good. What would you think about realeasing Lucene/Solr
 4.5, are there issues you would like to get in before the release?

 --
 Adrien

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 4.5

2013-09-11 Thread Jack Krupansky

There seems to be a modest amount of Solr work in progress that probably 
wants to be in Solr 4.5. I would suggest another two weeks for Solr work to 
finish and bake. Maybe cut an RC0 on, say, 9/23. Or maybe things will have 
settled down in a week and it would be worth an RC0 in a week, with the risk 
of an RC1 a few days or a week later.


Otherwise, +1.

-- Jack Krupansky

-Original Message- 
From: Adrien Grand

Sent: Wednesday, September 11, 2013 9:40 AM
To: dev@lucene.apache.org
Subject: Lucene/Solr 4.5

Hi,

I was looking at the changelogs for Lucene and Solr and I think they
look pretty good. What would you think about realeasing Lucene/Solr
4.5, are there issues you would like to get in before the release?

--
Adrien

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org 



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4816) Add document routing to CloudSolrServer

2013-09-11 Thread Shawn Heisey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764436#comment-13764436
 ] 

Shawn Heisey commented on SOLR-4816:


As requested via IRC, here is a reminder comment to add sugar methods like 
setRequestWriter to CloudSolrServer.

 Add document routing to CloudSolrServer
 ---

 Key: SOLR-4816
 URL: https://issues.apache.org/jira/browse/SOLR-4816
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Affects Versions: 4.3
Reporter: Joel Bernstein
Assignee: Mark Miller
Priority: Minor
 Fix For: 4.5, 5.0

 Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816-sriesenberg.patch


 This issue adds the following enhancements to CloudSolrServer's update logic:
 1) Document routing: Updates are routed directly to the correct shard leader 
 eliminating document routing at the server.
 2) Optional parallel update execution: Updates for each shard are executed in 
 a separate thread so parallel indexing can occur across the cluster.
 These enhancements should allow for near linear scalability on indexing 
 throughput.
 Usage:
 CloudSolrServer cloudClient = new CloudSolrServer(zkAddress);
 cloudClient.setParallelUpdates(true); 
 SolrInputDocument doc1 = new SolrInputDocument();
 doc1.addField(id, 0);
 doc1.addField(a_t, hello1);
 SolrInputDocument doc2 = new SolrInputDocument();
 doc2.addField(id, 2);
 doc2.addField(a_t, hello2);
 UpdateRequest request = new UpdateRequest();
 request.add(doc1);
 request.add(doc2);
 request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false);
 NamedList response = cloudClient.request(request); // Returns a backwards 
 compatible condensed response.
 //To get more detailed response down cast to RouteResponse:
 CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response;

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 4.5

Unless someone else volunteers to do the release, I'll be busy until
~wednesday next week so there are still a few days to get fixes in
anyway. However I agree with Robert that our branch_4x should always
be stable and releasable.

@Jack, @Erick could you share more details about the issues that need
to be fixed before a 4.5 release?

-- 
Adrien

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5232) SolrCloud should distribute updates via streaming rather than buffering.

2013-09-11 Thread Erick Erickson (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1376#comment-1376
 ] 

Erick Erickson commented on SOLR-5232:
--

bq: batch size != queue size

I wasn't confusing the two, they just happen to be the same numbers. What I'm 
after here is something on record as to whether we expect the anecdotal 
evidence of faster indexing by increasing batch size to be affected by 
streaming, making changing the batch size question less relevant.

I suppose with Joel's/Mark's changes to CloudSolrServer, much of the question 
goes away anyway, and this JIRA will then primarily affect the leader 
forwarding the updates to the followers.

Of course all bets are off if a user directly uses SUSS. Seems like a best 
practice is use CloudSolrServer and if you really insist on NOT using it, then 
this JIRA will keep you from deadlock.

 SolrCloud should distribute updates via streaming rather than buffering.
 

 Key: SOLR-5232
 URL: https://issues.apache.org/jira/browse/SOLR-5232
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.5, 5.0

 Attachments: SOLR-5232.patch, SOLR-5232.patch, SOLR-5232.patch


 The current approach was never the best for SolrCloud - it was designed for a 
 pre SolrCloud Solr - it also uses too many connections and threads - nailing 
 that down is likely wasted effort when we should really move away from 
 explicitly buffering docs and sending small batches per thread as we have 
 been doing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5232) SolrCloud should distribute updates via streaming rather than buffering.

[
https://issues.apache.org/jira/browse/SOLR-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764484#comment-13764484
]

Mark Miller commented on SOLR-5232:
---

All SolrJ clients and non SolrJ clients should fall under 'best practice'
depending on your use case and desires. Using the new routing in
CloudSolrServer is a possible workaround for issues that should be addressed,
not worked around. Also, routing removes forwarding to leaders - you still send
documents to replicas - that is still reliant on the buffer. CloudSolrServer
can run also be used in a manner that does not route as an aside.

bq. making changing the batch size question less relevant.

As the comments on the batch size issue indicate, changing the size is not an
option we are willing to go with.

As far as speed comparisons, someone would have to do some benchmarks to know
how the new strategy holds up - it's just clearly where we have always needed
to get to eventually.

There is no similar buffer size to consider here.

SolrCloud should distribute updates via streaming rather than buffering.

Key: SOLR-5232
URL: https://issues.apache.org/jira/browse/SOLR-5232
Project: Solr
Issue Type: Improvement
Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
Fix For: 4.5, 5.0

Attachments: SOLR-5232.patch, SOLR-5232.patch, SOLR-5232.patch

The current approach was never the best for SolrCloud - it was designed for a
pre SolrCloud Solr - it also uses too many connections and threads - nailing
that down is likely wasted effort when we should really move away from
explicitly buffering docs and sending small batches per thread as we have
been doing.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-5235) Update Log replay does not use the processor chain for commit

2013-09-11 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764381#comment-13764381
 ] 

ludovic Boutros edited comment on SOLR-5235 at 9/11/13 3:45 PM:


The code of UpdateLog.java for reference :

{code:title=UpdateLog.java|borderStyle=solid}
CommitUpdateCommand cmd = new CommitUpdateCommand(req, false);
cmd.setVersion(commitVersion);
cmd.softCommit = false;
cmd.waitSearcher = true;
cmd.setFlags(UpdateCommand.REPLAY);
try {
  if (debug) log.debug(commit  +  cmd);
  uhandler.commit(cmd);  // this should cause a commit to be 
added to the incomplete log and avoid it being replayed again after a restart.
} catch (IOException ex) {
  recoveryInfo.errors++;
  loglog.error(Replay exception: final commit., ex);
}
{code} 

  was (Author: lboutros):
The portion code of UpdateLog.java for reference :

{code:title=UpdateLog.java|borderStyle=solid}
CommitUpdateCommand cmd = new CommitUpdateCommand(req, false);
cmd.setVersion(commitVersion);
cmd.softCommit = false;
cmd.waitSearcher = true;
cmd.setFlags(UpdateCommand.REPLAY);
try {
  if (debug) log.debug(commit  +  cmd);
  uhandler.commit(cmd);  // this should cause a commit to be 
added to the incomplete log and avoid it being replayed again after a restart.
} catch (IOException ex) {
  recoveryInfo.errors++;
  loglog.error(Replay exception: final commit., ex);
}
{code} 
  
 Update Log replay does not use the processor chain for commit
 -

 Key: SOLR-5235
 URL: https://issues.apache.org/jira/browse/SOLR-5235
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.3.1, 4.4
Reporter: ludovic Boutros

 The update log replay process commit commands directly with the Update 
 Handler.
 The update processor chain is not used. I may be wrong but I think this is to 
 prevent to log this commit command again in the LogUpdateProcessor.
 But this commit command is flagged with the flag UpdateCommand.REPLAY. I 
 think this flag should be checked in the LogUpdateProcessor in order to adapt 
 its behavior.
 Currently, commit actions in custom Update Processors are not applied in case 
 of a crash without an explicit commit.
 A workaround can be done with the finish function but this is not ideal.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 4.5

2013-09-11 Thread Mikhail Khludnev

I just want to mention that blockjoin update behaves unexpectedly sometime.
https://issues.apache.org/jira/browse/SOLR-5211
if parent with children is overwritten by parent without children orphan
children stick to the next parent _after_ merge.
I'm not sure about severity, just want to let you know.
Unfortunately have no time even for testproof.



On Wed, Sep 11, 2013 at 8:35 PM, Adrien Grand jpou...@gmail.com wrote:

 Unless someone else volunteers to do the release, I'll be busy until
 ~wednesday next week so there are still a few days to get fixes in
 anyway. However I agree with Robert that our branch_4x should always
 be stable and releasable.

 @Jack, @Erick could you share more details about the issues that need
 to be fixed before a 4.5 release?

 --
 Adrien

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org




-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

http://www.griddynamics.com
 mkhlud...@griddynamics.com

[jira] [Commented] (SOLR-5231) When a boolean field is missing from a doc it is sometimes treated as true by the if function (based on other docs in segment?)


[ 
https://issues.apache.org/jira/browse/SOLR-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764542#comment-13764542
 ] 

ASF subversion and git services commented on SOLR-5231:
---

Commit 1521948 from hoss...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1521948 ]

SOLR-5231: Fixed a bug with the behavior of BoolField that caused documents w/o 
a value for the field to act as if the value were true in functions if no other 
documents in the same index segment had a value of true.

 When a boolean field is missing from a doc it is sometimes treated as true 
 by the if function (based on other docs in segment?)
 ---

 Key: SOLR-5231
 URL: https://issues.apache.org/jira/browse/SOLR-5231
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.4
Reporter: Hoss Man
 Attachments: SOLR-5231.patch


 This issue is hard to explain with out a long example.
 crux of the problem is that the behavior of the if function, wrapped arround 
 a boolean field (ie: {{if(fieldName,x,y)}} ) is not consistent for 
 documents that do not have any value for that functio -- the behavior seems 
 to  depend on whether or not other documents in the same segment have a value 
 for that field.
 for brevity, details will follow in a comment - but i've been able to 
 reproduce on trunk, 4.3, and 4.3 (didn't look back farther then that)
 the work around is to explicitly use the {{exists()}} function in the if 
 condition (ie: {{if(exists(fieldName),x,y)}} )
 (Thanks to Elodie Sannier for reporting the initial symptoms of this on the 
 mailing list)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Setup Atlassian FishEye over Lucene/Solr source repository

2013-09-11 Thread Steve Rowe

Atlassian offers free hosting for its FishEye service over Apache projects' 
source repositories.  Looks like they currently host 60 or so projects: 
https://fisheye6.atlassian.com/browse

FishEye can search file history - content as well as metadata - and I'd like to 
be able to do that against Lucene/Solr sources.

If there are no objections, I'll request Atlassian set this up for us - 
instructions on how to do this are under Adding an Apache Repo on the 
Welcome Message section here https://fisheye6.atlassian.com.

Steve
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5231) When a boolean field is missing from a doc it is sometimes treated as true by the if function (based on other docs in segment?)


 [ 
https://issues.apache.org/jira/browse/SOLR-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-5231:
---

Affects Version/s: 4.2
   4.3
   4.3.1
Fix Version/s: 5.0
   4.5
 Assignee: Hoss Man

 When a boolean field is missing from a doc it is sometimes treated as true 
 by the if function (based on other docs in segment?)
 ---

 Key: SOLR-5231
 URL: https://issues.apache.org/jira/browse/SOLR-5231
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.2, 4.3, 4.3.1, 4.4
Reporter: Hoss Man
Assignee: Hoss Man
 Fix For: 4.5, 5.0

 Attachments: SOLR-5231.patch


 This issue is hard to explain with out a long example.
 crux of the problem is that the behavior of the if function, wrapped arround 
 a boolean field (ie: {{if(fieldName,x,y)}} ) is not consistent for 
 documents that do not have any value for that functio -- the behavior seems 
 to  depend on whether or not other documents in the same segment have a value 
 for that field.
 for brevity, details will follow in a comment - but i've been able to 
 reproduce on trunk, 4.3, and 4.3 (didn't look back farther then that)
 the work around is to explicitly use the {{exists()}} function in the if 
 condition (ie: {{if(exists(fieldName),x,y)}} )
 (Thanks to Elodie Sannier for reporting the initial symptoms of this on the 
 mailing list)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-5231) When a boolean field is missing from a doc it is sometimes treated as true by the if function (based on other docs in segment?)


 [ 
https://issues.apache.org/jira/browse/SOLR-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man resolved SOLR-5231.


Resolution: Fixed

http://svn.apache.org/r1521948
http://svn.apache.org/r1521969

 When a boolean field is missing from a doc it is sometimes treated as true 
 by the if function (based on other docs in segment?)
 ---

 Key: SOLR-5231
 URL: https://issues.apache.org/jira/browse/SOLR-5231
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.2, 4.3, 4.3.1, 4.4
Reporter: Hoss Man
Assignee: Hoss Man
 Fix For: 4.5, 5.0

 Attachments: SOLR-5231.patch


 This issue is hard to explain with out a long example.
 crux of the problem is that the behavior of the if function, wrapped arround 
 a boolean field (ie: {{if(fieldName,x,y)}} ) is not consistent for 
 documents that do not have any value for that functio -- the behavior seems 
 to  depend on whether or not other documents in the same segment have a value 
 for that field.
 for brevity, details will follow in a comment - but i've been able to 
 reproduce on trunk, 4.3, and 4.3 (didn't look back farther then that)
 the work around is to explicitly use the {{exists()}} function in the if 
 condition (ie: {{if(exists(fieldName),x,y)}} )
 (Thanks to Elodie Sannier for reporting the initial symptoms of this on the 
 mailing list)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Setup Atlassian FishEye over Lucene/Solr source repository

2013-09-11 Thread Dawid Weiss

Do they also offer Bamboo services? This would be a nice addition to
Jenkins... :)

Dawid

On Wed, Sep 11, 2013 at 8:23 PM, Steve Rowe sar...@gmail.com wrote:
 Atlassian offers free hosting for its FishEye service over Apache projects' 
 source repositories.  Looks like they currently host 60 or so projects: 
 https://fisheye6.atlassian.com/browse

 FishEye can search file history - content as well as metadata - and I'd like 
 to be able to do that against Lucene/Solr sources.

 If there are no objections, I'll request Atlassian set this up for us - 
 instructions on how to do this are under Adding an Apache Repo on the 
 Welcome Message section here https://fisheye6.atlassian.com.

 Steve
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5235) Update Log replay does not use the processor chain for commit


 [ 
https://issues.apache.org/jira/browse/SOLR-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ludovic Boutros updated SOLR-5235:
--

Description: 
The update log replay sends commit commands directly with the Update Handler.
The update processor chain is not used. I may be wrong but I think this is to 
prevent to log this commit command again in the LogUpdateProcessor.

But this commit command is flagged with the flag UpdateCommand.REPLAY. I think 
this flag should be checked in the LogUpdateProcessor in order to adapt its 
behavior.

Currently, commit actions in custom Update Processors are not applied in case 
of a crash without an explicit commit.
A workaround can be done with the finish function but this is not ideal.

  was:
The update log replay process commit commands directly with the Update Handler.
The update processor chain is not used. I may be wrong but I think this is to 
prevent to log this commit command again in the LogUpdateProcessor.

But this commit command is flagged with the flag UpdateCommand.REPLAY. I think 
this flag should be checked in the LogUpdateProcessor in order to adapt its 
behavior.

Currently, commit actions in custom Update Processors are not applied in case 
of a crash without an explicit commit.
A workaround can be done with the finish function but this is not ideal.


 Update Log replay does not use the processor chain for commit
 -

 Key: SOLR-5235
 URL: https://issues.apache.org/jira/browse/SOLR-5235
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.3.1, 4.4
Reporter: ludovic Boutros

 The update log replay sends commit commands directly with the Update Handler.
 The update processor chain is not used. I may be wrong but I think this is to 
 prevent to log this commit command again in the LogUpdateProcessor.
 But this commit command is flagged with the flag UpdateCommand.REPLAY. I 
 think this flag should be checked in the LogUpdateProcessor in order to adapt 
 its behavior.
 Currently, commit actions in custom Update Processors are not applied in case 
 of a crash without an explicit commit.
 A workaround can be done with the finish function but this is not ideal.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5236) Arguments with spaces show as broken lines in Solr Admin

Cassandra Targett created SOLR-5236:
---

 Summary: Arguments with spaces show as broken lines in Solr Admin 
 Key: SOLR-5236
 URL: https://issues.apache.org/jira/browse/SOLR-5236
 Project: Solr
  Issue Type: Bug
  Components: web gui
 Environment: Mac OS X 10.8.4, Solr 4.x branch build #401
Reporter: Cassandra Targett
Priority: Minor
 Attachments: Solr4.x-startArgsWithSpaces.png

A screenshot will explain it best, but if you pass arguments at startup with 
spaces in them, the Solr Admin dashboard shows the arguments as broken lines.

This is the command I used to start Solr:

java -Dsdfasdf=asdfsdfasdfasd\ asdfsdfasdf\ sdfasdfasdfasdf\ asdfasdfasdf 
-Dfile.encoding=UTF-8 -jar start.jar

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5236) Arguments with spaces show as broken lines in Solr Admin

2013-09-11 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett updated SOLR-5236:


Attachment: Solr4.x-startArgsWithSpaces.png

 Arguments with spaces show as broken lines in Solr Admin 
 -

 Key: SOLR-5236
 URL: https://issues.apache.org/jira/browse/SOLR-5236
 Project: Solr
  Issue Type: Bug
  Components: web gui
 Environment: Mac OS X 10.8.4, Solr 4.x branch build #401
Reporter: Cassandra Targett
Priority: Minor
 Attachments: Solr4.x-startArgsWithSpaces.png


 A screenshot will explain it best, but if you pass arguments at startup with 
 spaces in them, the Solr Admin dashboard shows the arguments as broken lines.
 This is the command I used to start Solr:
 java -Dsdfasdf=asdfsdfasdfasd\ asdfsdfasdf\ sdfasdfasdfasdf\ asdfasdfasdf 
 -Dfile.encoding=UTF-8 -jar start.jar

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-MAVEN] Lucene-Solr-Maven-4.x #444: POMs out of sync

2013-09-11 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-Maven-4.x/444/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch

Error Message:
null

Stack Trace:
java.lang.NullPointerException: null
at 
__randomizedtesting.SeedInfo.seed([A52EF3898E587FD1:24C87D91F9071FED]:0)
at 
org.apache.solr.common.cloud.ZkCoreNodeProps.getBaseUrl(ZkCoreNodeProps.java:40)
at 
org.apache.solr.client.solrj.impl.CloudSolrServer.buildUrlMap(CloudSolrServer.java:404)
at 
org.apache.solr.client.solrj.impl.CloudSolrServer.directUpdate(CloudSolrServer.java:300)
at 
org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:534)
at 
org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:168)
at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:146)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1517)
at 
org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:288)
at 
org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:115)




Build Log:
[...truncated 25054 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5202) Support easier overrides of Carrot2 clustering attributes via XML data sets exported from the Workbench.


[ 
https://issues.apache.org/jira/browse/SOLR-5202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764622#comment-13764622
 ] 

ASF subversion and git services commented on SOLR-5202:
---

Commit 1521981 from [~dawidweiss] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1521981 ]

SOLR-5202: cleaned up comments surrounding clustering component. Left all the 
algorithms in there, but added a pointer to integration-strategies which 
includes more exhaustive configuration examples.

 Support easier overrides of Carrot2 clustering attributes via XML data sets 
 exported from the Workbench.
 

 Key: SOLR-5202
 URL: https://issues.apache.org/jira/browse/SOLR-5202
 Project: Solr
  Issue Type: New Feature
Reporter: Dawid Weiss
Assignee: Dawid Weiss
 Fix For: 4.5, 5.0

 Attachments: SOLR-5202.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 4.5

2013-09-11 Thread Simon Willnauer

I think I stated my opinion on this a couple of times. I agree with
robert +1 to not push last minute fixes and release 4.5 right now

simon

On Wed, Sep 11, 2013 at 6:23 PM, Robert Muir rcm...@gmail.com wrote:
 I think its terrible.

 Don't commit things to the stable branch_4x unless they are ready to
 be released: if features need baking do that shit in trunk.

 +1 to release 4.5 Right Now.

 On Wed, Sep 11, 2013 at 9:20 AM, Erick Erickson erickerick...@gmail.com 
 wrote:
 I think Jack's idea of setting a target date N weeks out is a good one.
 People commonly have some tidying up they'd like to do and a little
 time-frame between now and cutting a release would allow that to
 happen. In particular the just-committed SOLR-4816 and associated
 would have a bit more time to bake.

 Straw man, 1-Oct for cutting the release... Nice end-of-quarter date :)

 FWIW,
 Erick

 But +1 to releasing a 4.5 Real Soon Now.




 On Wed, Sep 11, 2013 at 12:01 PM, Jack Krupansky j...@basetechnology.com
 wrote:

 There seems to be a modest amount of Solr work in progress that probably
 wants to be in Solr 4.5. I would suggest another two weeks for Solr work to
 finish and bake. Maybe cut an RC0 on, say, 9/23. Or maybe things will have
 settled down in a week and it would be worth an RC0 in a week, with the risk
 of an RC1 a few days or a week later.

 Otherwise, +1.

 -- Jack Krupansky

 -Original Message- From: Adrien Grand
 Sent: Wednesday, September 11, 2013 9:40 AM
 To: dev@lucene.apache.org
 Subject: Lucene/Solr 4.5


 Hi,

 I was looking at the changelogs for Lucene and Solr and I think they
 look pretty good. What would you think about realeasing Lucene/Solr
 4.5, are there issues you would like to get in before the release?

 --
 Adrien

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org



 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 4.5

2013-09-11 Thread Mark Miller

I still believe, as I have voted before, that because we are a community of 
developers spread across the world and work space, we should have some warning 
for releases - we are collaborating here - people should have a bit of time to 
tie up loose ends, investigate reports, weigh in about the pending release, 
whatever, to help make a solid release. A week minimum has always seemed 
reasonable to me. It's not enough time to slam in your latest big feature - 
unless you are confident or marking it as experimental or whatever consensus on 
that issue decides (we play by consensus) - but it is enough time to harden, 
investigate, and tie up loose ends that have worried you, but you have not 
tackled, not yet having the pressure of an impending release.

No one should *need* a release right now unless it's a critical bug fix 
release. Otherwise it seems you can always bring it up a week before you need 
it. And then everyone gets to play nice together even though this may not be 
their first job or they may be in university for the weekend, or whatever.

So +1 to next Wednesday, -1 to right now.

- Mark

On Sep 11, 2013, at 12:35 PM, Adrien Grand jpou...@gmail.com wrote:

 Unless someone else volunteers to do the release, I'll be busy until
 ~wednesday next week so there are still a few days to get fixes in
 anyway. However I agree with Robert that our branch_4x should always
 be stable and releasable.
 
 @Jack, @Erick could you share more details about the issues that need
 to be fixed before a 4.5 release?
 
 -- 
 Adrien
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org
 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5237) Add Lucene Index heap usage to LukeRequestHandler

2013-09-11 Thread Robert Muir (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764659#comment-13764659
 ] 

Robert Muir commented on SOLR-5237:
---

If any leaf is not a SegmentReader, i would just return -1 immediately (i think 
this is just adding an else).

This would be consistent with other index statistics that sometimes arent 
supported.

 Add Lucene Index heap usage to LukeRequestHandler 
 --

 Key: SOLR-5237
 URL: https://issues.apache.org/jira/browse/SOLR-5237
 Project: Solr
  Issue Type: Improvement
Reporter: Areek Zillur
 Attachments: SOLR-5237.patch


 It would be useful to see the current index heap usage 
 (https://issues.apache.org/jira/browse/LUCENE-5197) by lucene in the 
 LukeRequestHandler.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5237) Add Lucene Index heap usage to LukeRequestHandler

2013-09-11 Thread Areek Zillur (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Areek Zillur updated SOLR-5237:
---

Attachment: SOLR-5237.patch

Initial patch that adds the attribute indexHeapUsageBytes to LukeRequestHandler

 Add Lucene Index heap usage to LukeRequestHandler 
 --

 Key: SOLR-5237
 URL: https://issues.apache.org/jira/browse/SOLR-5237
 Project: Solr
  Issue Type: Improvement
Reporter: Areek Zillur
 Attachments: SOLR-5237.patch


 It would be useful to see the current index heap usage 
 (https://issues.apache.org/jira/browse/LUCENE-5197) by lucene in the 
 LukeRequestHandler.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-4956) make maxBufferedAddsPerServer configurable

2013-09-11 Thread Otis Gospodnetic (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Otis Gospodnetic resolved SOLR-4956.

Resolution: Fixed

Will be fixed by SOLR-5232.

make maxBufferedAddsPerServer configurable
--

Key: SOLR-4956
URL: https://issues.apache.org/jira/browse/SOLR-4956
Project: Solr
Issue Type: Improvement
Affects Versions: 4.3, 5.0
Reporter: Erick Erickson

Anecdotal user's list evidence indicates that in high-throughput situations,
the default of 10 docs/batch for inter-shard batching can generate
significant CPU load. See the thread titled Sharding and Replication on
June 19th, but the gist is below.
I haven't poked around, but it's a little surprising on the surface that Asif
is seeing this kind of difference. So I'm wondering if this change indicates
some other underlying issue. Regardless, this seems like it would be good to
investigate.
Here's the gist of Asif's experience from the thread:
Its a completely practical problem - we are exploring Solr to build a real
time analytics/data solution for a system handling about 1000 qps. We have
various metrics that are stored as different collections on the cloud,
which means very high amount of writes. The cloud also needs to support
about 300-400 qps.
We initially tested with a single Solr node on a 16 core / 24 GB box for a
single metric. We saw that writes were not a issue at all - Solr was
handling it extremely well. We were also able to achieve about 200 qps from
a single node.
When we set up the cloud ( a ensemble on 6 boxes), we saw very high CPU
usage on the replicas. Up to 10 cores were getting used for writes on the
replicas. Hence my concern with respect to batch updates for the replicas.
BTW, I altered the maxBufferedAddsPerServer to 1000 - and now CPU usage is
very similar to single node installation.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5232) SolrCloud should distribute updates via streaming rather than buffering.

2013-09-11 Thread Otis Gospodnetic (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764634#comment-13764634
 ] 

Otis Gospodnetic commented on SOLR-5232:


[~markrmil...@gmail.com] does that mean we can close SOLR-4956 as Won't Fix?

 SolrCloud should distribute updates via streaming rather than buffering.
 

 Key: SOLR-5232
 URL: https://issues.apache.org/jira/browse/SOLR-5232
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.5, 5.0

 Attachments: SOLR-5232.patch, SOLR-5232.patch, SOLR-5232.patch


 The current approach was never the best for SolrCloud - it was designed for a 
 pre SolrCloud Solr - it also uses too many connections and threads - nailing 
 that down is likely wasted effort when we should really move away from 
 explicitly buffering docs and sending small batches per thread as we have 
 been doing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [jira] [Commented] (LUCENE-26) Wildcard query only accepts 1 or more characters for prefixes (not 0 or more)

2013-09-11 Thread Chris Hostetter


FYI: commit-jira bot pattern matching glitch...

https://issues.apache.org/jira/browse/INFRA-6745



: Date: Wed, 11 Sep 2013 14:01:52 + (UTC)
: From: ASF subversion and git services (JIRA) j...@apache.org
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: [jira] [Commented] (LUCENE-26) Wildcard query only accepts 1 or more
: characters for prefixes (not 0 or more)
: 
: 
: [ 
https://issues.apache.org/jira/browse/LUCENE-26?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764321#comment-13764321
 ] 
: 
: ASF subversion and git services commented on LUCENE-26:
: ---
: 
: Commit 1521851 from [~vajda] in branch 'pylucene/trunk'
: [ https://svn.apache.org/r1521851 ]
: 
: fixed bug PYLUCENE-26 (Martin Anon)
: 
:  Wildcard query only accepts 1 or more characters for prefixes (not 0 or 
more)
:  
-
: 
:  Key: LUCENE-26
:  URL: https://issues.apache.org/jira/browse/LUCENE-26
:  Project: Lucene - Core
:   Issue Type: Bug
:   Components: core/search
:  Environment: Operating System: Linux
:  Platform: PC
: Reporter: Lee Mallabone
: Assignee: Lucene Developers
:  Attachments: ASF.LICENSE.NOT.GRANTED--WildcardTermEnum.java.diff, 
ASF.LICENSE.NOT.GRANTED--WildcardTest.java
: 
: 
:  When using the WildcardQuery class to do a wildcard search with the 
wildcard at
:  the end of the string, the wildcard is treated as 1 or more characters. 
This
:  is inconsistent with other wildcard behaviour, and undesirable. The correct
:  behaviour is for the wildcard to represent 0 or more characters.
:  The error is due to a problem in the wildcard comparison method in
:  org.apache.lucene.search.WildcardTermEnum.
:  Patch to follow.
: 
: --
: This message is automatically generated by JIRA.
: If you think it was sent incorrectly, please contact your JIRA administrators
: For more information on JIRA, see: http://www.atlassian.com/software/jira
: 
: -
: To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: For additional commands, e-mail: dev-h...@lucene.apache.org
: 
: 

-Hoss

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-5219) Rewritten selection of the default search and document clustering algorithms

2013-09-11 Thread Dawid Weiss (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Weiss resolved SOLR-5219.
---

Resolution: Fixed

 Rewritten selection of the default search and document clustering algorithms
 

 Key: SOLR-5219
 URL: https://issues.apache.org/jira/browse/SOLR-5219
 Project: Solr
  Issue Type: Improvement
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor
 Fix For: 4.5, 5.0


 This is currently quite messy: the user needs to explicitly name one 
 algorithm as the default, the code of processing that is a bit hairy.
 The new logic is:
 1) if there is a clustering algorithm named 'default' it becomes the default 
 one.
 2) if there's any algorithm defined at all, the first one in declaration 
 order becomes the default one.
 There is no need to declare clustering.algorithm parameter for the clustering 
 component, unless the default from the above procedure needs to be altered.
 This is a fully backwards compatible improvement that allows one to declare 
 more than one algorithm and give them sensible names.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5219) Rewritten selection of the default search and document clustering algorithms

2013-09-11 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764625#comment-13764625
 ] 

ASF subversion and git services commented on SOLR-5219:
---

Commit 1521982 from [~dawidweiss] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1521982 ]

SOLR-5219: Rewritten selection of the default search and document clustering 
algorithms.

 Rewritten selection of the default search and document clustering algorithms
 

 Key: SOLR-5219
 URL: https://issues.apache.org/jira/browse/SOLR-5219
 Project: Solr
  Issue Type: Improvement
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor
 Fix For: 4.5, 5.0


 This is currently quite messy: the user needs to explicitly name one 
 algorithm as the default, the code of processing that is a bit hairy.
 The new logic is:
 1) if there is a clustering algorithm named 'default' it becomes the default 
 one.
 2) if there's any algorithm defined at all, the first one in declaration 
 order becomes the default one.
 There is no need to declare clustering.algorithm parameter for the clustering 
 component, unless the default from the above procedure needs to be altered.
 This is a fully backwards compatible improvement that allows one to declare 
 more than one algorithm and give them sensible names.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5238) Update the .css for the Ref Guide

Cassandra Targett created SOLR-5238:
---

 Summary: Update the .css for the Ref Guide
 Key: SOLR-5238
 URL: https://issues.apache.org/jira/browse/SOLR-5238
 Project: Solr
  Issue Type: Improvement
  Components: documentation
Reporter: Cassandra Targett
Priority: Minor
 Fix For: 4.5


I put a custom .css into the Ref Guide before it was uploaded. I was going for 
something parallel to the Solr website, but only spent a little time with it. 
In terms of readibility of the text online, it's not great, which is putting it 
nicely. It's also very difficult to differentiate between normal text and 
monospaced text to indicate a command, program name, etc.

I'm attaching a new .css that can simply replace what's already in the Space 
Admin - Stylesheet section. I did several things with this:

* cleaned up the .css generally, consolidated some repetitive sections, and 
added more comments in case future changes are desired.
* changed the font throughout to Helvetica, Arial, or sans-serif and updated 
the color to a slightly less strong black.
* changed the monospace font to match the font used in the code boxes 
(Consolas) and made them the same color as the text (default is a lot lighter).
* added a bit more space between lines.
* removed the negative margin in the header/breadcrumbs to give it a bit more 
space.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5238) Update the .css for the Ref Guide

[
https://issues.apache.org/jira/browse/SOLR-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Cassandra Targett updated SOLR-5238:

Attachment: SolrRefGuide.css

Update the .css for the Ref Guide
-

Key: SOLR-5238
URL: https://issues.apache.org/jira/browse/SOLR-5238
Project: Solr
Issue Type: Improvement
Components: documentation
Reporter: Cassandra Targett
Priority: Minor
Fix For: 4.5

Attachments: SolrRefGuide.css

I put a custom .css into the Ref Guide before it was uploaded. I was going
for something parallel to the Solr website, but only spent a little time with
it. In terms of readibility of the text online, it's not great, which is
putting it nicely. It's also very difficult to differentiate between normal
text and monospaced text to indicate a command, program name, etc.
I'm attaching a new .css that can simply replace what's already in the Space
Admin - Stylesheet section. I did several things with this:
* cleaned up the .css generally, consolidated some repetitive sections, and
added more comments in case future changes are desired.
* changed the font throughout to Helvetica, Arial, or sans-serif and updated
the color to a slightly less strong black.
* changed the monospace font to match the font used in the code boxes
(Consolas) and made them the same color as the text (default is a lot
lighter).
* added a bit more space between lines.
* removed the negative margin in the header/breadcrumbs to give it a bit more
space.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5232) SolrCloud should distribute updates via streaming rather than buffering.


[ 
https://issues.apache.org/jira/browse/SOLR-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764638#comment-13764638
 ] 

Mark Miller commented on SOLR-5232:
---

Or resolve it as fixed by this if you wanted to track the issue as the 
performance problem. I'm fine with either approach.

We still have to look at thread count usage with the this though - and do some 
general checks.

 SolrCloud should distribute updates via streaming rather than buffering.
 

 Key: SOLR-5232
 URL: https://issues.apache.org/jira/browse/SOLR-5232
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.5, 5.0

 Attachments: SOLR-5232.patch, SOLR-5232.patch, SOLR-5232.patch


 The current approach was never the best for SolrCloud - it was designed for a 
 pre SolrCloud Solr - it also uses too many connections and threads - nailing 
 that down is likely wasted effort when we should really move away from 
 explicitly buffering docs and sending small batches per thread as we have 
 been doing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5236) Arguments with spaces show as broken lines in Solr Admin

2013-09-11 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-5236:
---

Attachment: SOLR-5236.linux.widescreen.png
SOLR-5236.linux.narrowscreen.png

As i mentioned to Cassandra about this on IRC: it doesn't seem to happen on 
linux.  

Here are my screenshots from a similar command that don't exibit the problem, 
evne when the browser window is too narrow for the strings to fit.

Also interesting to note that when it does happen, the arg isn't just broken up 
on multiple lines, the *order* of the fragments are odd ... note in Cassandra's 
screenshot that the whitespace seperated values essentially appear in reverse 
order that they were on the command line

 Arguments with spaces show as broken lines in Solr Admin 
 -

 Key: SOLR-5236
 URL: https://issues.apache.org/jira/browse/SOLR-5236
 Project: Solr
  Issue Type: Bug
  Components: web gui
 Environment: Mac OS X 10.8.4, Solr 4.x branch build #401
Reporter: Cassandra Targett
Priority: Minor
 Attachments: Solr4.x-startArgsWithSpaces.png, 
 SOLR-5236.linux.narrowscreen.png, SOLR-5236.linux.widescreen.png


 A screenshot will explain it best, but if you pass arguments at startup with 
 spaces in them, the Solr Admin dashboard shows the arguments as broken lines.
 This is the command I used to start Solr:
 java -Dsdfasdf=asdfsdfasdfasd\ asdfsdfasdf\ sdfasdfasdfasdf\ asdfasdfasdf 
 -Dfile.encoding=UTF-8 -jar start.jar

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5237) Add Lucene Index heap usage to LukeRequestHandler

2013-09-11 Thread Areek Zillur (JIRA)

Areek Zillur created SOLR-5237:
--

 Summary: Add Lucene Index heap usage to LukeRequestHandler 
 Key: SOLR-5237
 URL: https://issues.apache.org/jira/browse/SOLR-5237
 Project: Solr
  Issue Type: Improvement
Reporter: Areek Zillur


It would be useful to see the current index heap usage 
(https://issues.apache.org/jira/browse/LUCENE-5197) by lucene in the 
LukeRequestHandler.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5219) Rewritten selection of the default search and document clustering algorithms


[ 
https://issues.apache.org/jira/browse/SOLR-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764614#comment-13764614
 ] 

ASF subversion and git services commented on SOLR-5219:
---

Commit 1521978 from [~dawidweiss] in branch 'dev/trunk'
[ https://svn.apache.org/r1521978 ]

SOLR-5219: Rewritten selection of the default search and document clustering 
algorithms.

 Rewritten selection of the default search and document clustering algorithms
 

 Key: SOLR-5219
 URL: https://issues.apache.org/jira/browse/SOLR-5219
 Project: Solr
  Issue Type: Improvement
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor
 Fix For: 4.5, 5.0


 This is currently quite messy: the user needs to explicitly name one 
 algorithm as the default, the code of processing that is a bit hairy.
 The new logic is:
 1) if there is a clustering algorithm named 'default' it becomes the default 
 one.
 2) if there's any algorithm defined at all, the first one in declaration 
 order becomes the default one.
 There is no need to declare clustering.algorithm parameter for the clustering 
 component, unless the default from the above procedure needs to be altered.
 This is a fully backwards compatible improvement that allows one to declare 
 more than one algorithm and give them sensible names.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5219) Rewritten selection of the default search and document clustering algorithms

2013-09-11 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764615#comment-13764615
 ] 

ASF subversion and git services commented on SOLR-5219:
---

Commit 1521979 from [~dawidweiss] in branch 'dev/trunk'
[ https://svn.apache.org/r1521979 ]

SOLR-5219: always forget about eol-style...

 Rewritten selection of the default search and document clustering algorithms
 

 Key: SOLR-5219
 URL: https://issues.apache.org/jira/browse/SOLR-5219
 Project: Solr
  Issue Type: Improvement
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor
 Fix For: 4.5, 5.0


 This is currently quite messy: the user needs to explicitly name one 
 algorithm as the default, the code of processing that is a bit hairy.
 The new logic is:
 1) if there is a clustering algorithm named 'default' it becomes the default 
 one.
 2) if there's any algorithm defined at all, the first one in declaration 
 order becomes the default one.
 There is no need to declare clustering.algorithm parameter for the clustering 
 component, unless the default from the above procedure needs to be altered.
 This is a fully backwards compatible improvement that allows one to declare 
 more than one algorithm and give them sensible names.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Setup Atlassian FishEye over Lucene/Solr source repository

2013-09-11 Thread Dawid Weiss

I know about licensing -- we're actually using it for Carrot2. I was
thinking about free iron to run it on :)

D.

On Wed, Sep 11, 2013 at 8:46 PM, Steve Rowe sar...@gmail.com wrote:
 Looks like they offer a free *license*: 
 https://www.atlassian.com/opensource/overview (there's a Bamboo tab under 
 Supported Projects), but I rather doubt they host anybody - I couldn't find 
 any information about it anyway.

 Steve

 On Sep 11, 2013, at 2:26 PM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote:

 Do they also offer Bamboo services? This would be a nice addition to
 Jenkins... :)

 Dawid

 On Wed, Sep 11, 2013 at 8:23 PM, Steve Rowe sar...@gmail.com wrote:
 Atlassian offers free hosting for its FishEye service over Apache projects' 
 source repositories.  Looks like they currently host 60 or so projects: 
 https://fisheye6.atlassian.com/browse

 FishEye can search file history - content as well as metadata - and I'd 
 like to be able to do that against Lucene/Solr sources.

 If there are no objections, I'll request Atlassian set this up for us - 
 instructions on how to do this are under Adding an Apache Repo on the 
 Welcome Message section here https://fisheye6.atlassian.com.

 Steve
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org



 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4816) Add document routing to CloudSolrServer


[ 
https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764687#comment-13764687
 ] 

Joel Bernstein commented on SOLR-4816:
--

Just did a fresh 4x pull and org.apache.solr.cloud.SyncSliceTest ran clean for 
me several times.

 Add document routing to CloudSolrServer
 ---

 Key: SOLR-4816
 URL: https://issues.apache.org/jira/browse/SOLR-4816
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Affects Versions: 4.3
Reporter: Joel Bernstein
Assignee: Mark Miller
Priority: Minor
 Fix For: 4.5, 5.0

 Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816-sriesenberg.patch


 This issue adds the following enhancements to CloudSolrServer's update logic:
 1) Document routing: Updates are routed directly to the correct shard leader 
 eliminating document routing at the server.
 2) Optional parallel update execution: Updates for each shard are executed in 
 a separate thread so parallel indexing can occur across the cluster.
 These enhancements should allow for near linear scalability on indexing 
 throughput.
 Usage:
 CloudSolrServer cloudClient = new CloudSolrServer(zkAddress);
 cloudClient.setParallelUpdates(true); 
 SolrInputDocument doc1 = new SolrInputDocument();
 doc1.addField(id, 0);
 doc1.addField(a_t, hello1);
 SolrInputDocument doc2 = new SolrInputDocument();
 doc2.addField(id, 2);
 doc2.addField(a_t, hello2);
 UpdateRequest request = new UpdateRequest();
 request.add(doc1);
 request.add(doc2);
 request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false);
 NamedList response = cloudClient.request(request); // Returns a backwards 
 compatible condensed response.
 //To get more detailed response down cast to RouteResponse:
 CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response;

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Setup Atlassian FishEye over Lucene/Solr source repository

2013-09-11 Thread Dawid Weiss

Much nicer user interface ;)

And more seriously, it does have a few interesting features and things, but
nothing that would make it a different grade software. I like test vault
which allows marking tests as unstable. But it is very similar overall.

d.

On Wednesday, September 11, 2013, Adrien Grand wrote:

 Out of curiosity, is it just for having another testing environment or
 does Bamboo have useful features for us that Jenkins doesn't have?

 On Wed, Sep 11, 2013 at 8:26 PM, Dawid Weiss
 dawid.we...@cs.put.poznan.pl javascript:; wrote:
  Do they also offer Bamboo services? This would be a nice addition to
  Jenkins... :)
 
  Dawid
 
  On Wed, Sep 11, 2013 at 8:23 PM, Steve Rowe sar...@gmail.comjavascript:;
 wrote:
  Atlassian offers free hosting for its FishEye service over Apache
 projects' source repositories.  Looks like they currently host 60 or so
 projects: https://fisheye6.atlassian.com/browse
 
  FishEye can search file history - content as well as metadata - and I'd
 like to be able to do that against Lucene/Solr sources.
 
  If there are no objections, I'll request Atlassian set this up for us -
 instructions on how to do this are under Adding an Apache Repo on the
 Welcome Message section here https://fisheye6.atlassian.com.
 
  Steve
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.orgjavascript:;
  For additional commands, e-mail: dev-h...@lucene.apache.orgjavascript:;
 
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org javascript:;
  For additional commands, e-mail: dev-h...@lucene.apache.orgjavascript:;
 



 --
 Adrien

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org javascript:;
 For additional commands, e-mail: dev-h...@lucene.apache.org javascript:;

[jira] [Updated] (SOLR-5219) Rewritten selection of the default search and document clustering algorithms

2013-09-11 Thread Dawid Weiss (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Weiss updated SOLR-5219:
--

Summary: Rewritten selection of the default search and document clustering 
algorithms  (was: Refactor selection of the default clustering algorithm)

 Rewritten selection of the default search and document clustering algorithms
 

 Key: SOLR-5219
 URL: https://issues.apache.org/jira/browse/SOLR-5219
 Project: Solr
  Issue Type: Improvement
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor
 Fix For: 4.5, 5.0


 This is currently quite messy: the user needs to explicitly name the 
 'default' algorithm. The logic should be:
 1) if there's only one algorithm, it becomes the default,
 2) if there's more than one algorithm, the first one becomes the default one.
 3) for back-compat, if there is an algorithm called 'default', it does become 
 the default one.
 The code will simplify a great deal too.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Setup Atlassian FishEye over Lucene/Solr source repository

2013-09-11 Thread Steve Rowe

Looks like they offer a free *license*: 
https://www.atlassian.com/opensource/overview (there's a Bamboo tab under 
Supported Projects), but I rather doubt they host anybody - I couldn't find any 
information about it anyway.

Steve

On Sep 11, 2013, at 2:26 PM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote:

 Do they also offer Bamboo services? This would be a nice addition to
 Jenkins... :)
 
 Dawid
 
 On Wed, Sep 11, 2013 at 8:23 PM, Steve Rowe sar...@gmail.com wrote:
 Atlassian offers free hosting for its FishEye service over Apache projects' 
 source repositories.  Looks like they currently host 60 or so projects: 
 https://fisheye6.atlassian.com/browse
 
 FishEye can search file history - content as well as metadata - and I'd like 
 to be able to do that against Lucene/Solr sources.
 
 If there are no objections, I'll request Atlassian set this up for us - 
 instructions on how to do this are under Adding an Apache Repo on the 
 Welcome Message section here https://fisheye6.atlassian.com.
 
 Steve
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org
 
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org
 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5219) Rewritten selection of the default search and document clustering algorithms

2013-09-11 Thread Dawid Weiss (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dawid Weiss updated SOLR-5219:
--

Description:
This is currently quite messy: the user needs to explicitly name one algorithm
as the default, the code of processing that is a bit hairy.

The new logic is:

1) if there is a clustering algorithm named 'default' it becomes the default
one.
2) if there's any algorithm defined at all, the first one in declaration order
becomes the default one.

There is no need to declare clustering.algorithm parameter for the clustering
component, unless the default from the above procedure needs to be altered.

This is a fully backwards compatible improvement that allows one to declare
more than one algorithm and give them sensible names.

was:
This is currently quite messy: the user needs to explicitly name the 'default'
algorithm. The logic should be:

1) if there's only one algorithm, it becomes the default,
2) if there's more than one algorithm, the first one becomes the default one.
3) for back-compat, if there is an algorithm called 'default', it does become
the default one.

The code will simplify a great deal too.

Rewritten selection of the default search and document clustering algorithms

Key: SOLR-5219
URL: https://issues.apache.org/jira/browse/SOLR-5219
Project: Solr
Issue Type: Improvement
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor
Fix For: 4.5, 5.0

This is currently quite messy: the user needs to explicitly name one
algorithm as the default, the code of processing that is a bit hairy.
The new logic is:
1) if there is a clustering algorithm named 'default' it becomes the default
one.
2) if there's any algorithm defined at all, the first one in declaration
order becomes the default one.
There is no need to declare clustering.algorithm parameter for the clustering
component, unless the default from the above procedure needs to be altered.
This is a fully backwards compatible improvement that allows one to declare
more than one algorithm and give them sensible names.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Setup Atlassian FishEye over Lucene/Solr source repository

Out of curiosity, is it just for having another testing environment or
does Bamboo have useful features for us that Jenkins doesn't have?

On Wed, Sep 11, 2013 at 8:26 PM, Dawid Weiss
dawid.we...@cs.put.poznan.pl wrote:
 Do they also offer Bamboo services? This would be a nice addition to
 Jenkins... :)

 Dawid

 On Wed, Sep 11, 2013 at 8:23 PM, Steve Rowe sar...@gmail.com wrote:
 Atlassian offers free hosting for its FishEye service over Apache projects' 
 source repositories.  Looks like they currently host 60 or so projects: 
 https://fisheye6.atlassian.com/browse

 FishEye can search file history - content as well as metadata - and I'd like 
 to be able to do that against Lucene/Solr sources.

 If there are no objections, I'll request Atlassian set this up for us - 
 instructions on how to do this are under Adding an Apache Repo on the 
 Welcome Message section here https://fisheye6.atlassian.com.

 Steve
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org




-- 
Adrien

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5237) Add Lucene Index heap usage to LukeRequestHandler

2013-09-11 Thread Areek Zillur (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Areek Zillur updated SOLR-5237:
---

Attachment: SOLR-5237.patch

Return -1 when any of the readers are not SegmentReader

 Add Lucene Index heap usage to LukeRequestHandler 
 --

 Key: SOLR-5237
 URL: https://issues.apache.org/jira/browse/SOLR-5237
 Project: Solr
  Issue Type: Improvement
Reporter: Areek Zillur
 Attachments: SOLR-5237.patch, SOLR-5237.patch


 It would be useful to see the current index heap usage 
 (https://issues.apache.org/jira/browse/LUCENE-5197) by lucene in the 
 LukeRequestHandler.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 4.5

2013-09-11 Thread Michael McCandless

+1 to release now.

The 4.x branch should be always shippable, and I think it's risky to
rush features in a the last minute.  Better to commit features right
after a release, so it has time to bake before the next release.

Mike McCandless

http://blog.mikemccandless.com


On Wed, Sep 11, 2013 at 3:25 PM, Simon Willnauer
simon.willna...@gmail.com wrote:
 I think I stated my opinion on this a couple of times. I agree with
 robert +1 to not push last minute fixes and release 4.5 right now

 simon

 On Wed, Sep 11, 2013 at 6:23 PM, Robert Muir rcm...@gmail.com wrote:
 I think its terrible.

 Don't commit things to the stable branch_4x unless they are ready to
 be released: if features need baking do that shit in trunk.

 +1 to release 4.5 Right Now.

 On Wed, Sep 11, 2013 at 9:20 AM, Erick Erickson erickerick...@gmail.com 
 wrote:
 I think Jack's idea of setting a target date N weeks out is a good one.
 People commonly have some tidying up they'd like to do and a little
 time-frame between now and cutting a release would allow that to
 happen. In particular the just-committed SOLR-4816 and associated
 would have a bit more time to bake.

 Straw man, 1-Oct for cutting the release... Nice end-of-quarter date :)

 FWIW,
 Erick

 But +1 to releasing a 4.5 Real Soon Now.




 On Wed, Sep 11, 2013 at 12:01 PM, Jack Krupansky j...@basetechnology.com
 wrote:

 There seems to be a modest amount of Solr work in progress that probably
 wants to be in Solr 4.5. I would suggest another two weeks for Solr work to
 finish and bake. Maybe cut an RC0 on, say, 9/23. Or maybe things will have
 settled down in a week and it would be worth an RC0 in a week, with the 
 risk
 of an RC1 a few days or a week later.

 Otherwise, +1.

 -- Jack Krupansky

 -Original Message- From: Adrien Grand
 Sent: Wednesday, September 11, 2013 9:40 AM
 To: dev@lucene.apache.org
 Subject: Lucene/Solr 4.5


 Hi,

 I was looking at the changelogs for Lucene and Solr and I think they
 look pretty good. What would you think about realeasing Lucene/Solr
 4.5, are there issues you would like to get in before the release?

 --
 Adrien

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org



 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-MAVEN] Lucene-Solr-Maven-trunk #967: POMs out of sync

2013-09-11 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-Maven-trunk/967/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch

Error Message:
null

Stack Trace:
java.lang.NullPointerException: null
at 
__randomizedtesting.SeedInfo.seed([E24A476A1649882E:63ACC9726116E812]:0)
at 
org.apache.solr.common.cloud.ZkCoreNodeProps.getBaseUrl(ZkCoreNodeProps.java:40)
at 
org.apache.solr.client.solrj.impl.CloudSolrServer.buildUrlMap(CloudSolrServer.java:404)
at 
org.apache.solr.client.solrj.impl.CloudSolrServer.directUpdate(CloudSolrServer.java:300)
at 
org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:534)
at 
org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:168)
at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:146)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1517)
at 
org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:288)
at 
org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:115)


REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
expected:5 but was:4

Stack Trace:
java.lang.AssertionError: expected:5 but was:4
at 
__randomizedtesting.SeedInfo.seed([9B801116E92CE4ED:1A669F0E9E7384D1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:175)




Build Log:
[...truncated 24592 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5204) Make Directory easier to wrap

2013-09-11 Thread Adrien Grand (JIRA)

Adrien Grand created LUCENE-5204:


 Summary: Make Directory easier to wrap
 Key: LUCENE-5204
 URL: https://issues.apache.org/jira/browse/LUCENE-5204
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor


We have a few DirectoryWrapper implementations such as 
RateLimitedDirectoryWrapper and MockDirectoryWrapper. However, the Directory 
class is not straightforward to wrap since it already has logic for getting and 
setting the lock factory, so wrappers need to decide whether they should 
forward lock handling to the delegate or handle it themselves.

I would like to move the locking logic out of the Directory class and to have a 
base {{FilterDirectory}} that could be extended by other directory wrapper 
impls.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5204) Make Directory easier to wrap

2013-09-11 Thread Adrien Grand (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-5204:
-

Attachment: LUCENE-5204.patch

Here is a patch:
 - the LockFactory-related methods are now abstract in Directory, impls have 
been moved to BaseDirectory, which is the base class for all concrete 
implementations (RAMDirectory, FSDirectories, etc.).
 - FilterDirectory extends Directory and implements all methods that don't have 
a default implementation in Directory.
 - BaseDirectoryWrapper, RateLimitedDirectoryWrapper and 
TrackingDirectoryWrapper now extend FilterDirectory, only overriding what's 
necessary.


 Make Directory easier to wrap
 -

 Key: LUCENE-5204
 URL: https://issues.apache.org/jira/browse/LUCENE-5204
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
 Attachments: LUCENE-5204.patch


 We have a few DirectoryWrapper implementations such as 
 RateLimitedDirectoryWrapper and MockDirectoryWrapper. However, the Directory 
 class is not straightforward to wrap since it already has logic for getting 
 and setting the lock factory, so wrappers need to decide whether they should 
 forward lock handling to the delegate or handle it themselves.
 I would like to move the locking logic out of the Directory class and to have 
 a base {{FilterDirectory}} that could be extended by other directory wrapper 
 impls.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5204) Make Directory easier to wrap

2013-09-11 Thread Robert Muir (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764908#comment-13764908
 ] 

Robert Muir commented on LUCENE-5204:
-

Can we make FilterDirectory package-private?

I know we have done this FilterXXX strategy with other apis, but delegating 
over an abstract class is always broken by definition... Its a little scary to 
expose this as a supported API for directory.

 Make Directory easier to wrap
 -

 Key: LUCENE-5204
 URL: https://issues.apache.org/jira/browse/LUCENE-5204
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
 Attachments: LUCENE-5204.patch


 We have a few DirectoryWrapper implementations such as 
 RateLimitedDirectoryWrapper and MockDirectoryWrapper. However, the Directory 
 class is not straightforward to wrap since it already has logic for getting 
 and setting the lock factory, so wrappers need to decide whether they should 
 forward lock handling to the delegate or handle it themselves.
 I would like to move the locking logic out of the Directory class and to have 
 a base {{FilterDirectory}} that could be extended by other directory wrapper 
 impls.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5204) Make Directory easier to wrap

2013-09-11 Thread Robert Muir (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764910#comment-13764910
 ] 

Robert Muir commented on LUCENE-5204:
-

Or at least mark it internal with a huge huge huge warning, thats ok too. I 
just think all these delegators are going to bite us bigtime.

 Make Directory easier to wrap
 -

 Key: LUCENE-5204
 URL: https://issues.apache.org/jira/browse/LUCENE-5204
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
 Attachments: LUCENE-5204.patch


 We have a few DirectoryWrapper implementations such as 
 RateLimitedDirectoryWrapper and MockDirectoryWrapper. However, the Directory 
 class is not straightforward to wrap since it already has logic for getting 
 and setting the lock factory, so wrappers need to decide whether they should 
 forward lock handling to the delegate or handle it themselves.
 I would like to move the locking logic out of the Directory class and to have 
 a base {{FilterDirectory}} that could be extended by other directory wrapper 
 impls.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4221) Custom sharding


[ 
https://issues.apache.org/jira/browse/SOLR-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764968#comment-13764968
 ] 

Hoss Man commented on SOLR-4221:


Is this feature complete? (all of the subtasks are marked resolved and several 
commits associated with this isue are in branch 4x)


Can someone who understands all of the various changes made in these issues 
please update the ref guide (or post a comment suggestion what additions should 
be made)...

https://cwiki.apache.org/confluence/display/solr/Collections+API




 Custom sharding
 ---

 Key: SOLR-4221
 URL: https://issues.apache.org/jira/browse/SOLR-4221
 Project: Solr
  Issue Type: New Feature
Reporter: Yonik Seeley
Assignee: Noble Paul
 Attachments: SOLR-4221.patch, SOLR-4221.patch, SOLR-4221.patch, 
 SOLR-4221.patch, SOLR-4221.patch


 Features to let users control everything about sharding/routing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 4.5

2013-09-11 Thread Erick Erickson

No one has said anything about rushing in last minute features. It's all
about cleaning up any loose ends that have been nagging with the current
code base.

Come on, people. We all do our level best to only commit patches that are
ready. But I defy anyone to claim they always get it right the very first
time
every time. Having a few days to insure that everyone on the different
schedules can tidy up whatever, get the last test in for the corner case
that
just occurred do them etc. is eminently reasonable.

Erick


On Wed, Sep 11, 2013 at 6:35 PM, Michael McCandless 
luc...@mikemccandless.com wrote:

 +1 to release now.

 The 4.x branch should be always shippable, and I think it's risky to
 rush features in a the last minute.  Better to commit features right
 after a release, so it has time to bake before the next release.

 Mike McCandless

 http://blog.mikemccandless.com


 On Wed, Sep 11, 2013 at 3:25 PM, Simon Willnauer
 simon.willna...@gmail.com wrote:
  I think I stated my opinion on this a couple of times. I agree with
  robert +1 to not push last minute fixes and release 4.5 right now
 
  simon
 
  On Wed, Sep 11, 2013 at 6:23 PM, Robert Muir rcm...@gmail.com wrote:
  I think its terrible.
 
  Don't commit things to the stable branch_4x unless they are ready to
  be released: if features need baking do that shit in trunk.
 
  +1 to release 4.5 Right Now.
 
  On Wed, Sep 11, 2013 at 9:20 AM, Erick Erickson 
 erickerick...@gmail.com wrote:
  I think Jack's idea of setting a target date N weeks out is a good one.
  People commonly have some tidying up they'd like to do and a little
  time-frame between now and cutting a release would allow that to
  happen. In particular the just-committed SOLR-4816 and associated
  would have a bit more time to bake.
 
  Straw man, 1-Oct for cutting the release... Nice end-of-quarter date :)
 
  FWIW,
  Erick
 
  But +1 to releasing a 4.5 Real Soon Now.
 
 
 
 
  On Wed, Sep 11, 2013 at 12:01 PM, Jack Krupansky 
 j...@basetechnology.com
  wrote:
 
  There seems to be a modest amount of Solr work in progress that
 probably
  wants to be in Solr 4.5. I would suggest another two weeks for Solr
 work to
  finish and bake. Maybe cut an RC0 on, say, 9/23. Or maybe things will
 have
  settled down in a week and it would be worth an RC0 in a week, with
 the risk
  of an RC1 a few days or a week later.
 
  Otherwise, +1.
 
  -- Jack Krupansky
 
  -Original Message- From: Adrien Grand
  Sent: Wednesday, September 11, 2013 9:40 AM
  To: dev@lucene.apache.org
  Subject: Lucene/Solr 4.5
 
 
  Hi,
 
  I was looking at the changelogs for Lucene and Solr and I think they
  look pretty good. What would you think about realeasing Lucene/Solr
  4.5, are there issues you would like to get in before the release?
 
  --
  Adrien
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
  For additional commands, e-mail: dev-h...@lucene.apache.org
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
  For additional commands, e-mail: dev-h...@lucene.apache.org
 
 
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
  For additional commands, e-mail: dev-h...@lucene.apache.org
 
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
  For additional commands, e-mail: dev-h...@lucene.apache.org
 

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 4.5

2013-09-11 Thread Yonik Seeley

+1, a week at minimum seems very reasonable.

-Yonik
http://lucidworks.com


On Wed, Sep 11, 2013 at 3:00 PM, Mark Miller markrmil...@gmail.com wrote:
 I still believe, as I have voted before, that because we are a community of 
 developers spread across the world and work space, we should have some 
 warning for releases - we are collaborating here - people should have a bit 
 of time to tie up loose ends, investigate reports, weigh in about the pending 
 release, whatever, to help make a solid release. A week minimum has always 
 seemed reasonable to me. It's not enough time to slam in your latest big 
 feature - unless you are confident or marking it as experimental or whatever 
 consensus on that issue decides (we play by consensus) - but it is enough 
 time to harden, investigate, and tie up loose ends that have worried you, but 
 you have not tackled, not yet having the pressure of an impending release.

 No one should *need* a release right now unless it's a critical bug fix 
 release. Otherwise it seems you can always bring it up a week before you need 
 it. And then everyone gets to play nice together even though this may not be 
 their first job or they may be in university for the weekend, or whatever.

 So +1 to next Wednesday, -1 to right now.

 - Mark

 On Sep 11, 2013, at 12:35 PM, Adrien Grand jpou...@gmail.com wrote:

 Unless someone else volunteers to do the release, I'll be busy until
 ~wednesday next week so there are still a few days to get fixes in
 anyway. However I agree with Robert that our branch_4x should always
 be stable and releasable.

 @Jack, @Erick could you share more details about the issues that need
 to be fixed before a 4.5 release?

 --
 Adrien

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org



 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4937) SolrCloud doesn't distribute null values

2013-09-11 Thread Steve Davids (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765095#comment-13765095
 ] 

Steve Davids commented on SOLR-4937:


The support for null values is only available if you use the update syntax as 
described above. I was plainly using a null value which updated the leader's 
value but during the XML transformation it drops the null value when 
distributing to replicas (if not accompanied with the update command). If 
JavaBin was used to forward the message (SOLR-5223) it would work either way, 
setting just null or {set:null}.

 SolrCloud doesn't distribute null values
 

 Key: SOLR-4937
 URL: https://issues.apache.org/jira/browse/SOLR-4937
 Project: Solr
  Issue Type: Bug
Reporter: Steve Davids
 Fix For: 4.5, 5.0


 When trying to overwrite field values in SolrCloud using 
 doc.setField(fieldName, null) it produces inconsistent behavior depending on 
 the routing of the document to a specific shard. The binary format that is 
 sent in preserves the null, but when the DistributedProcessor forwards the 
 message to replicas it writes the message to XML using 
 ClientUtils.writeVal(..) which drops any null value from the XML 
 representation. This was especially problematic when a custom processor was 
 initially placed after the distributed processor using the previously 
 mentioned setField(null) approach but then moved ahead of the 
 DistributedProcessor which no longer works as expected. It appears that I now 
 need to updated the code to: doc.setField(fieldName, 
 Collections.singletonMap(set, null)) for it to properly distribute 
 throughout the cloud due to the XML restrictions. The fact that the custom 
 processor needs to change depending on it's location in reference to the 
 DistributedProcessor is a drag. I believe there should be a requirement that 
 you can take a SolrInputDocument - toXml - toSolrInputDocument and assert 
 that the two SolrInputDocuments are equivalent, instead of a lossy 
 translation to XML.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4816) Add document routing to CloudSolrServer


 [ 
https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shikhar Bhushan updated SOLR-4816:
--

Attachment: RequestTask-removal.patch

 Add document routing to CloudSolrServer
 ---

 Key: SOLR-4816
 URL: https://issues.apache.org/jira/browse/SOLR-4816
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Affects Versions: 4.3
Reporter: Joel Bernstein
Assignee: Mark Miller
Priority: Minor
 Fix For: 4.5, 5.0

 Attachments: RequestTask-removal.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch


 This issue adds the following enhancements to CloudSolrServer's update logic:
 1) Document routing: Updates are routed directly to the correct shard leader 
 eliminating document routing at the server.
 2) Optional parallel update execution: Updates for each shard are executed in 
 a separate thread so parallel indexing can occur across the cluster.
 These enhancements should allow for near linear scalability on indexing 
 throughput.
 Usage:
 CloudSolrServer cloudClient = new CloudSolrServer(zkAddress);
 cloudClient.setParallelUpdates(true); 
 SolrInputDocument doc1 = new SolrInputDocument();
 doc1.addField(id, 0);
 doc1.addField(a_t, hello1);
 SolrInputDocument doc2 = new SolrInputDocument();
 doc2.addField(id, 2);
 doc2.addField(a_t, hello2);
 UpdateRequest request = new UpdateRequest();
 request.add(doc1);
 request.add(doc2);
 request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false);
 NamedList response = cloudClient.request(request); // Returns a backwards 
 compatible condensed response.
 //To get more detailed response down cast to RouteResponse:
 CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response;

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4816) Add document routing to CloudSolrServer


[ 
https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765114#comment-13765114
 ] 

Shikhar Bhushan commented on SOLR-4816:
---

We've run into some issues with CloudSolrServer leaking loads of 
LBHttpSolrServer's aliveCheckExecutor thread pools with {{parallelUpdates = 
true}}.

The root cause here is that the RequestTask inner class is creating a new 
LBHttpSolrServer for each run() rather than utilizing CloudSolrServer.lbServer 
which is already available to it.

Some detail: LBHttpSolrServer lazily initializes a single-threaded 
ScheduledExecutorService for the aliveCheckExecutor when e.g. there is some 
kind of error talking to a server. So this issue tends to come up when Solr 
nodes are unavailable and exceptions are thrown. There is also no call to 
shutdown() on that LBHttpSolrServer which gets created from RequestTask.run(). 
LBHttpSolrServer does have a finalizer that tries to shutdown the 
aliveCheckExecutor but there's no guarantee of finalizers executing (or maybe 
there is some other memory leak preventing that LBHttpSolrServer from being 
GC'ed at all).

So the one-liner fix that should definitely go in is to simply have RequestTask 
use CloudSolrServer.lbServer.

I have attached a patch that removes RequestTask altogether in favor of simply 
using Callable's and Future's which is much more idiomatic. 
(RequestTask-removal.patch)

 Add document routing to CloudSolrServer
 ---

 Key: SOLR-4816
 URL: https://issues.apache.org/jira/browse/SOLR-4816
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Affects Versions: 4.3
Reporter: Joel Bernstein
Assignee: Mark Miller
Priority: Minor
 Fix For: 4.5, 5.0

 Attachments: RequestTask-removal.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch


 This issue adds the following enhancements to CloudSolrServer's update logic:
 1) Document routing: Updates are routed directly to the correct shard leader 
 eliminating document routing at the server.
 2) Optional parallel update execution: Updates for each shard are executed in 
 a separate thread so parallel indexing can occur across the cluster.
 These enhancements should allow for near linear scalability on indexing 
 throughput.
 Usage:
 CloudSolrServer cloudClient = new CloudSolrServer(zkAddress);
 cloudClient.setParallelUpdates(true); 
 SolrInputDocument doc1 = new SolrInputDocument();
 doc1.addField(id, 0);
 doc1.addField(a_t, hello1);
 SolrInputDocument doc2 = new SolrInputDocument();
 doc2.addField(id, 2);
 doc2.addField(a_t, hello2);
 UpdateRequest request = new UpdateRequest();
 request.add(doc1);
 request.add(doc2);
 request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false);
 NamedList response = cloudClient.request(request); // Returns a backwards 
 compatible condensed response.
 //To get more detailed response down cast to RouteResponse:
 CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response;

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4816) Add document routing to CloudSolrServer


[ 
https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765120#comment-13765120
 ] 

Shikhar Bhushan commented on SOLR-4816:
---

This is a separate issue but worth noting: CloudSolrServer.shutdown() does not 
call lbServer.shutdown()

In case the lbServer is provided as a constructor arg from outside that 
probably make sense.

But in case of the constructors where it is created internally, IMO 
CloudSolrServer should assume ownership and also shut it down.

 Add document routing to CloudSolrServer
 ---

 Key: SOLR-4816
 URL: https://issues.apache.org/jira/browse/SOLR-4816
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Affects Versions: 4.3
Reporter: Joel Bernstein
Assignee: Mark Miller
Priority: Minor
 Fix For: 4.5, 5.0

 Attachments: RequestTask-removal.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch


 This issue adds the following enhancements to CloudSolrServer's update logic:
 1) Document routing: Updates are routed directly to the correct shard leader 
 eliminating document routing at the server.
 2) Optional parallel update execution: Updates for each shard are executed in 
 a separate thread so parallel indexing can occur across the cluster.
 These enhancements should allow for near linear scalability on indexing 
 throughput.
 Usage:
 CloudSolrServer cloudClient = new CloudSolrServer(zkAddress);
 cloudClient.setParallelUpdates(true); 
 SolrInputDocument doc1 = new SolrInputDocument();
 doc1.addField(id, 0);
 doc1.addField(a_t, hello1);
 SolrInputDocument doc2 = new SolrInputDocument();
 doc2.addField(id, 2);
 doc2.addField(a_t, hello2);
 UpdateRequest request = new UpdateRequest();
 request.add(doc1);
 request.add(doc2);
 request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false);
 NamedList response = cloudClient.request(request); // Returns a backwards 
 compatible condensed response.
 //To get more detailed response down cast to RouteResponse:
 CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response;

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4998) be more precise about IOContext for reads