date:20131005

2013-10-05 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-5300:


Attachment: SOLR-5300.patch

A new parameter ranges is introduced in split shard action which accepts 
comma-separated hash ranges in hexadecimal only.

 Split shards with custom hash ranges
 

 Key: SOLR-5300
 URL: https://issues.apache.org/jira/browse/SOLR-5300
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
 Fix For: 5.0, 4.6

 Attachments: SOLR-5300.patch


 Currently, shards can only be split at the mid point of their hash range. 
 This makes it difficult to control the distribution of data in the sub shards.
 We should make it possible to specify ranges to be used for splitting. A 
 ranges parameter can be added which can accept hash ranges in decimal or 
 hexadecimal e.g. ranges=0-0x1f4,0x1f5-0x3e8,0x3e9-0x5dc will split a shard 
 with range 0-1500 into three shards with ranges [0,500], [501-1000] and 
 [1001-1500] respectively.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5304) Typo in exception string in CurrencyField.java


[ 
https://issues.apache.org/jira/browse/SOLR-5304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13786975#comment-13786975
 ] 

ASF subversion and git services commented on SOLR-5304:
---

Commit 1529405 from sha...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1529405 ]

SOLR-5304: Fixed typo in exception message in CurrencyField

 Typo in exception string in CurrencyField.java
 --

 Key: SOLR-5304
 URL: https://issues.apache.org/jira/browse/SOLR-5304
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.4
Reporter: Caleb Burns
Priority: Trivial
  Labels: typo
   Original Estimate: 5m
  Remaining Estimate: 5m

 There is a typo in an exception string in CurrencyField.java. As of today, in 
 https://svn.apache.org/repos/asf/lucene/dev/trunk/solr/core/src/java/org/apache/solr/schema/CurrencyField.java
  on line 149:
 {code:title=CurrencyField.java}
   throw new SolrException(ErrorCode.BAD_REQUEST, Error instantiating 
 exhange rate provider +exchangeRateProviderClass+:  + e.getMessage(), e);
 {code}
 should be:
 {code:title=CurrencyField.java}
 throw new SolrException(ErrorCode.BAD_REQUEST, Error instantiating exchange 
 rate provider +exchangeRateProviderClass+:  + e.getMessage(), e);
 {code}
 exchange is misspelled  as exhange.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5304) Typo in exception string in CurrencyField.java

2013-10-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13786976#comment-13786976
 ] 

ASF subversion and git services commented on SOLR-5304:
---

Commit 1529408 from sha...@apache.org in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1529408 ]

SOLR-5304: Fixed typo in exception message in CurrencyField

 Typo in exception string in CurrencyField.java
 --

 Key: SOLR-5304
 URL: https://issues.apache.org/jira/browse/SOLR-5304
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.4
Reporter: Caleb Burns
Priority: Trivial
  Labels: typo
 Fix For: 5.0, 4.6

   Original Estimate: 5m
  Remaining Estimate: 5m

 There is a typo in an exception string in CurrencyField.java. As of today, in 
 https://svn.apache.org/repos/asf/lucene/dev/trunk/solr/core/src/java/org/apache/solr/schema/CurrencyField.java
  on line 149:
 {code:title=CurrencyField.java}
   throw new SolrException(ErrorCode.BAD_REQUEST, Error instantiating 
 exhange rate provider +exchangeRateProviderClass+:  + e.getMessage(), e);
 {code}
 should be:
 {code:title=CurrencyField.java}
 throw new SolrException(ErrorCode.BAD_REQUEST, Error instantiating exchange 
 rate provider +exchangeRateProviderClass+:  + e.getMessage(), e);
 {code}
 exchange is misspelled  as exhange.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-5304) Typo in exception string in CurrencyField.java


 [ 
https://issues.apache.org/jira/browse/SOLR-5304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar resolved SOLR-5304.
-

   Resolution: Fixed
Fix Version/s: 4.6
   5.0
 Assignee: Shalin Shekhar Mangar

This is fixed.

Thanks Caleb!

 Typo in exception string in CurrencyField.java
 --

 Key: SOLR-5304
 URL: https://issues.apache.org/jira/browse/SOLR-5304
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.4
Reporter: Caleb Burns
Assignee: Shalin Shekhar Mangar
Priority: Trivial
  Labels: typo
 Fix For: 5.0, 4.6

   Original Estimate: 5m
  Remaining Estimate: 5m

 There is a typo in an exception string in CurrencyField.java. As of today, in 
 https://svn.apache.org/repos/asf/lucene/dev/trunk/solr/core/src/java/org/apache/solr/schema/CurrencyField.java
  on line 149:
 {code:title=CurrencyField.java}
   throw new SolrException(ErrorCode.BAD_REQUEST, Error instantiating 
 exhange rate provider +exchangeRateProviderClass+:  + e.getMessage(), e);
 {code}
 should be:
 {code:title=CurrencyField.java}
 throw new SolrException(ErrorCode.BAD_REQUEST, Error instantiating exchange 
 rate provider +exchangeRateProviderClass+:  + e.getMessage(), e);
 {code}
 exchange is misspelled  as exhange.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-5303) numShards property is not properly taken into account

[
https://issues.apache.org/jira/browse/SOLR-5303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shalin Shekhar Mangar resolved SOLR-5303.
-

Resolution: Cannot Reproduce

I can't reproduce this with Solr 4.3.1. I used the Example A from SolrCloud
wiki at
http://wiki.apache.org/solr/SolrCloud#Example_A:_Simple_two_shard_cluster with
-DnumShards=2 and 2 shards are indeed created.

The code in question is correct. From Integer.getInteger javadocs:
{quote}
Determines the integer value of the system property with the specified name.
The first argument is treated as the name of a system property. System
properties are accessible through the System.getProperty(String) method. The
string value of this property is then interpreted as an integer value and an
Integer object representing this value is returned. Details of possible numeric
formats can be found with the definition of getProperty.
If there is no property with the specified name, if the specified name is empty
or null, or if the property does not have the correct numeric format, then null
is returned.
In other words, this method returns an Integer object equal to the value of:
getInteger(nm, null)
{quote}

The maxShardsPerNode is another property which limits the actual number of
physical shards per Solr node. This defaults to 1 unless you're using the
collection API. Perhaps that is confusing you? I'll close this issue and I
request you to discuss this on the solr-user mailing list.

numShards property is not properly taken into account
-

Key: SOLR-5303
URL: https://issues.apache.org/jira/browse/SOLR-5303
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.2, 4.3.1
Environment: SolR on 3 VMs each with an external Zookeeper,
multi-core startups
Reporter: Federico Piai
Labels: shard, solrcloud, zookeeper

It looks like the 'numShards' argument is ignored by SolR. The number of
shards is always defaulted to 1 unless cores are dynamically created with
Collection API.
I had this log :
INFO: numShards not found on descriptor - reading it from system property, I
looked for the piece of code where the log was located
((https://svn.apache.org/repos/asf/lucene/dev/trunk/solr/core/src/java/org/apache/solr/cloud/ZkController.java))
and I found a possible error :
log.info(numShards not found on descriptor - reading it from system
property);
numShards = Integer.getInteger(ZkStateReader.NUM_SHARDS_PROP);

--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[ANNOUNCE] Apache Lucene 4.5 released

2013-10-05 Thread Adrien Grand

October 2013, Apache Lucene™ 4.5 available

The Lucene PMC is pleased to announce the release of Apache Lucene 4.5

Apache Lucene is a high-performance, full-featured text search engine
library written entirely in Java. It is a technology suitable for nearly
any application that requires full-text search, especially cross-platform.

This release contains numerous bug fixes, optimizations, and
improvements, some of which are highlighted below. The release
is available for immediate download at:
  http://lucene.apache.org/core/mirrors-core-latest-redir.html

See the CHANGES.txt file included with the release for a full list of
details.

Lucene 4.5 Release Highlights:

* Added support for missing values to DocValues fields through
AtomicReader.getDocsWithField.

* Lucene 4.5 has a new Lucene45Codec with Lucene45DocValues,
supporting missing values and with most datastructures residing
off-heap.

* New in-memory DocIdSet implementations which are especially better
than FixedBitSet on small sets: WAH8DocIdSet, PFORDeltaDocIdSet and
EliasFanoDocIdSet.

* CachingWrapperFilter now caches filters with WAH8DocIdSet by
default, which has the same memory usage as FixedBitSet in the worst
case but is smaller and faster on small sets.

* TokenStreams now set the position increment in end(), so we can
handle trailing holes.

* IndexWriter no longer clones the given IndexWriterConfig.

* Various bugfixes and optimizations since the 4.4 release.

Please read CHANGES.txt for a full list of new features.

Please report any feedback to the mailing lists
(http://lucene.apache.org/core/discussion.html)

Note: The Apache Software Foundation uses an extensive mirroring network
for distributing releases.  It is possible that the mirror you are using
may not have replicated the release yet.  If that is the case, please
try another mirror.  This also goes for Maven access.

-- 
Adrien Grand

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-5255) Extraneous zookeeper call in ZkStateReader cluster state watcher and setting wrong watcher

2013-10-05 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar reassigned SOLR-5255:
---

Assignee: Shalin Shekhar Mangar

 Extraneous zookeeper call in ZkStateReader cluster state watcher and setting 
 wrong watcher
 --

 Key: SOLR-5255
 URL: https://issues.apache.org/jira/browse/SOLR-5255
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.4, 4.5, 5.0
Reporter: Jessica Cheng
Assignee: Shalin Shekhar Mangar
  Labels: cloud, zookeeper

 In ZkStateReader.createClusterStateWatchersAndUpdate, in the watcher for the 
 call to watch CLUSTER_STATE:
 ListString liveNodes = zkClient.getChildren(
 LIVE_NODES_ZKNODE, this, true);
  
 SetString liveNodesSet = new HashSetString();
 liveNodesSet.addAll(liveNodes);
 The liveNodesSet set is never used in subsequent code, since the new cluster 
 state is made from the live node set from the existing cluster state:
 SetString ln = ZkStateReader.this.clusterState.getLiveNodes();
 ClusterState clusterState = ClusterState.load(stat.getVersion(), data, 
 ln);
 ...which is as it should since this is a watcher for cluster state (if live 
 nodes changes, the watcher for live nodes will be invoked).
 Due to the above extraneous call, however, now the cluster state watcher 
 (this) is also set on the LIVE_NODES_ZKNODE path, which is unnecessary to 
 fire when live nodes changed.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5255) Extraneous zookeeper call in ZkStateReader cluster state watcher and setting wrong watcher


[ 
https://issues.apache.org/jira/browse/SOLR-5255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787176#comment-13787176
 ] 

ASF subversion and git services commented on SOLR-5255:
---

Commit 1529433 from sha...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1529433 ]

SOLR-5255: Remove unnecessary call to fetch and watch live nodes in 
ZkStateReader cluster watcher

 Extraneous zookeeper call in ZkStateReader cluster state watcher and setting 
 wrong watcher
 --

 Key: SOLR-5255
 URL: https://issues.apache.org/jira/browse/SOLR-5255
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.4, 4.5, 5.0
Reporter: Jessica Cheng
Assignee: Shalin Shekhar Mangar
  Labels: cloud, zookeeper

 In ZkStateReader.createClusterStateWatchersAndUpdate, in the watcher for the 
 call to watch CLUSTER_STATE:
 ListString liveNodes = zkClient.getChildren(
 LIVE_NODES_ZKNODE, this, true);
  
 SetString liveNodesSet = new HashSetString();
 liveNodesSet.addAll(liveNodes);
 The liveNodesSet set is never used in subsequent code, since the new cluster 
 state is made from the live node set from the existing cluster state:
 SetString ln = ZkStateReader.this.clusterState.getLiveNodes();
 ClusterState clusterState = ClusterState.load(stat.getVersion(), data, 
 ln);
 ...which is as it should since this is a watcher for cluster state (if live 
 nodes changes, the watcher for live nodes will be invoked).
 Due to the above extraneous call, however, now the cluster state watcher 
 (this) is also set on the LIVE_NODES_ZKNODE path, which is unnecessary to 
 fire when live nodes changed.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5255) Extraneous zookeeper call in ZkStateReader cluster state watcher and setting wrong watcher

2013-10-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787177#comment-13787177
 ] 

ASF subversion and git services commented on SOLR-5255:
---

Commit 1529434 from sha...@apache.org in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1529434 ]

SOLR-5255: Remove unnecessary call to fetch and watch live nodes in 
ZkStateReader cluster watcher

 Extraneous zookeeper call in ZkStateReader cluster state watcher and setting 
 wrong watcher
 --

 Key: SOLR-5255
 URL: https://issues.apache.org/jira/browse/SOLR-5255
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.4, 4.5, 5.0
Reporter: Jessica Cheng
Assignee: Shalin Shekhar Mangar
  Labels: cloud, zookeeper
 Fix For: 5.0, 4.6


 In ZkStateReader.createClusterStateWatchersAndUpdate, in the watcher for the 
 call to watch CLUSTER_STATE:
 ListString liveNodes = zkClient.getChildren(
 LIVE_NODES_ZKNODE, this, true);
  
 SetString liveNodesSet = new HashSetString();
 liveNodesSet.addAll(liveNodes);
 The liveNodesSet set is never used in subsequent code, since the new cluster 
 state is made from the live node set from the existing cluster state:
 SetString ln = ZkStateReader.this.clusterState.getLiveNodes();
 ClusterState clusterState = ClusterState.load(stat.getVersion(), data, 
 ln);
 ...which is as it should since this is a watcher for cluster state (if live 
 nodes changes, the watcher for live nodes will be invoked).
 Due to the above extraneous call, however, now the cluster state watcher 
 (this) is also set on the LIVE_NODES_ZKNODE path, which is unnecessary to 
 fire when live nodes changed.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-5255) Extraneous zookeeper call in ZkStateReader cluster state watcher and setting wrong watcher


 [ 
https://issues.apache.org/jira/browse/SOLR-5255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar resolved SOLR-5255.
-

   Resolution: Fixed
Fix Version/s: 4.6
   5.0

This is fixed. Thanks Jessica!

 Extraneous zookeeper call in ZkStateReader cluster state watcher and setting 
 wrong watcher
 --

 Key: SOLR-5255
 URL: https://issues.apache.org/jira/browse/SOLR-5255
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.4, 4.5, 5.0
Reporter: Jessica Cheng
Assignee: Shalin Shekhar Mangar
  Labels: cloud, zookeeper
 Fix For: 5.0, 4.6


 In ZkStateReader.createClusterStateWatchersAndUpdate, in the watcher for the 
 call to watch CLUSTER_STATE:
 ListString liveNodes = zkClient.getChildren(
 LIVE_NODES_ZKNODE, this, true);
  
 SetString liveNodesSet = new HashSetString();
 liveNodesSet.addAll(liveNodes);
 The liveNodesSet set is never used in subsequent code, since the new cluster 
 state is made from the live node set from the existing cluster state:
 SetString ln = ZkStateReader.this.clusterState.getLiveNodes();
 ClusterState clusterState = ClusterState.load(stat.getVersion(), data, 
 ln);
 ...which is as it should since this is a watcher for cluster state (if live 
 nodes changes, the watcher for live nodes will be invoked).
 Due to the above extraneous call, however, now the cluster state watcher 
 (this) is also set on the LIVE_NODES_ZKNODE path, which is unnecessary to 
 fire when live nodes changed.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-5302) Analytics Component


 [ 
https://issues.apache.org/jira/browse/SOLR-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson reassigned SOLR-5302:


Assignee: Erick Erickson

 Analytics Component
 ---

 Key: SOLR-5302
 URL: https://issues.apache.org/jira/browse/SOLR-5302
 Project: Solr
  Issue Type: New Feature
Reporter: Steven Bower
Assignee: Erick Erickson
 Attachments: Search Analytics Component.pdf, 
 solr_analytics-2013.10.04-2.patch, Statistical Expressions.pdf


 This ticket is to track a replacement for the StatsComponent. The 
 AnalyticsComponent supports the following features:
 * All functionality of StatsComponent (SOLR-4499)
 * Field Faceting (SOLR-3435)
 ** Support for limit
 ** Sorting (bucket name or any stat in the bucket
 ** Support for offset
 * Range Faceting
 ** Supports all options of standard range faceting
 * Query Faceting (SOLR-2925)
 * Ability to use overall/field facet statistics as input to range/query 
 faceting (ie calc min/max date and then facet over that range
 * Support for more complex aggregate/mapping operations (SOLR-1622)
 ** Aggregations: min, max, sum, sum-of-square, count, missing, stddev, mean, 
 median, percentiles
 ** Operations: negation, abs, add, multiply, divide, power, log, date math, 
 string reversal, string concat
 ** Easily pluggable framework to add additional operations
 * New / cleaner output format
 Outstanding Issues:
 * Multi-value field support for stats (supported for faceting)
 * Multi-shard support (may not be possible for some operations, eg median)



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5300) Split shards with custom hash ranges


 [ 
https://issues.apache.org/jira/browse/SOLR-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-5300:


Attachment: SOLR-5300.patch

Added a check to make sure that specified hash range is a subset of parent hash 
range.

 Split shards with custom hash ranges
 

 Key: SOLR-5300
 URL: https://issues.apache.org/jira/browse/SOLR-5300
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
 Fix For: 5.0, 4.6

 Attachments: SOLR-5300.patch, SOLR-5300.patch


 Currently, shards can only be split at the mid point of their hash range. 
 This makes it difficult to control the distribution of data in the sub shards.
 We should make it possible to specify ranges to be used for splitting. A 
 ranges parameter can be added which can accept hash ranges in decimal or 
 hexadecimal e.g. ranges=0-0x1f4,0x1f5-0x3e8,0x3e9-0x5dc will split a shard 
 with range 0-1500 into three shards with ranges [0,500], [501-1000] and 
 [1001-1500] respectively.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5300) Split shards with custom hash ranges

2013-10-05 Thread ASF subversion and git services (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shalin Shekhar Mangar updated SOLR-5300:

Description:
Currently, shards can only be split at the mid point of their hash range. This
makes it difficult to control the distribution of data in the sub shards.

We should make it possible to specify ranges to be used for splitting. A ranges
parameter can be added which can accept hash ranges in decimal or hexadecimal
e.g. ranges=0-1f4,1f5-3e8,3e9-5dc will split a shard with range 0-1500 into
three shards with ranges [0,500], [501-1000] and [1001-1500] respectively.

was:
Currently, shards can only be split at the mid point of their hash range. This
makes it difficult to control the distribution of data in the sub shards.

We should make it possible to specify ranges to be used for splitting. A ranges
parameter can be added which can accept hash ranges in decimal or hexadecimal
e.g. ranges=0-0x1f4,0x1f5-0x3e8,0x3e9-0x5dc will split a shard with range
0-1500 into three shards with ranges [0,500], [501-1000] and [1001-1500]
respectively.

Split shards with custom hash ranges

Key: SOLR-5300
URL: https://issues.apache.org/jira/browse/SOLR-5300
Project: Solr
Issue Type: Improvement
Components: SolrCloud
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
Fix For: 5.0, 4.6

Attachments: SOLR-5300.patch, SOLR-5300.patch

Currently, shards can only be split at the mid point of their hash range.
This makes it difficult to control the distribution of data in the sub shards.
We should make it possible to specify ranges to be used for splitting. A
ranges parameter can be added which can accept hash ranges in decimal or
hexadecimal e.g. ranges=0-1f4,1f5-3e8,3e9-5dc will split a shard with range
0-1500 into three shards with ranges [0,500], [501-1000] and [1001-1500]
respectively.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5300) Split shards with custom hash ranges


[ 
https://issues.apache.org/jira/browse/SOLR-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787201#comment-13787201
 ] 

ASF subversion and git services commented on SOLR-5300:
---

Commit 1529444 from sha...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1529444 ]

SOLR-5300: Shards can be split by specifying arbitrary number of hash ranges 
within the shard's hash range

 Split shards with custom hash ranges
 

 Key: SOLR-5300
 URL: https://issues.apache.org/jira/browse/SOLR-5300
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
 Fix For: 5.0, 4.6

 Attachments: SOLR-5300.patch, SOLR-5300.patch


 Currently, shards can only be split at the mid point of their hash range. 
 This makes it difficult to control the distribution of data in the sub shards.
 We should make it possible to specify ranges to be used for splitting. A 
 ranges parameter can be added which can accept hash ranges in decimal or 
 hexadecimal e.g. ranges=0-1f4,1f5-3e8,3e9-5dc will split a shard with range 
 0-1500 into three shards with ranges [0,500], [501-1000] and [1001-1500] 
 respectively.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-5300) Split shards with custom hash ranges

2013-10-05 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar resolved SOLR-5300.
-

Resolution: Fixed

 Split shards with custom hash ranges
 

 Key: SOLR-5300
 URL: https://issues.apache.org/jira/browse/SOLR-5300
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
 Fix For: 5.0, 4.6

 Attachments: SOLR-5300.patch, SOLR-5300.patch


 Currently, shards can only be split at the mid point of their hash range. 
 This makes it difficult to control the distribution of data in the sub shards.
 We should make it possible to specify ranges to be used for splitting. A 
 ranges parameter can be added which can accept hash ranges in decimal or 
 hexadecimal e.g. ranges=0-1f4,1f5-3e8,3e9-5dc will split a shard with range 
 0-1500 into three shards with ranges [0,500], [501-1000] and [1001-1500] 
 respectively.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5300) Split shards with custom hash ranges


[ 
https://issues.apache.org/jira/browse/SOLR-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787202#comment-13787202
 ] 

ASF subversion and git services commented on SOLR-5300:
---

Commit 1529445 from sha...@apache.org in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1529445 ]

SOLR-5300: Shards can be split by specifying arbitrary number of hash ranges 
within the shard's hash range

 Split shards with custom hash ranges
 

 Key: SOLR-5300
 URL: https://issues.apache.org/jira/browse/SOLR-5300
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
 Fix For: 5.0, 4.6

 Attachments: SOLR-5300.patch, SOLR-5300.patch


 Currently, shards can only be split at the mid point of their hash range. 
 This makes it difficult to control the distribution of data in the sub shards.
 We should make it possible to specify ranges to be used for splitting. A 
 ranges parameter can be added which can accept hash ranges in decimal or 
 hexadecimal e.g. ranges=0-1f4,1f5-3e8,3e9-5dc will split a shard with range 
 0-1500 into three shards with ranges [0,500], [501-1000] and [1001-1500] 
 respectively.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: VOTE: RC1 Release apache-solr-ref-guide-4.5.pdf

2013-10-05 Thread Steve Rowe


On Oct 3, 2013, at 1:45 PM, Chris Hostetter hossman_luc...@fucit.org wrote:
 2) i'll be traveling the next few days, so i've asked sarowe to fill in  
 and handle a few of the post-publish actions for me once the mirrors are 
 updated...
 
 https://cwiki.apache.org/confluence/display/solr/Internal+-+How+To+Publish+This+Documentation#Internal-HowToPublishThisDocumentation-UpdateLinks%26VersionNumbers
 - update version numbers in cwiki, and doc links on the website
 https://cwiki.apache.org/confluence/display/solr/Internal+-+How+To+Publish+This+Documentation#Internal-HowToPublishThisDocumentation-Announce
 - send announcement email

These are done.

 (given the delays Adrien mentioned publishing the 4.5 code artifacts, it 
 will probably make sense to hold off on the doc release announcement -- 
 and possible just send a single release announcement, mentioning both solr
 4.5 the ref guide 4.5 in the same email -- i'll leave that up to sarowe  
 jpountz)

I didn't coordinate with Adrien before he sent out the release announcement, so 
I sent out a separate announcement after his.

Steve


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5302) Analytics Component

2013-10-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787219#comment-13787219
 ] 

Erick Erickson commented on SOLR-5302:
--

I've assigned this to myself to commit  etc. I'll need all the help anyone 
wants to lend as far as the technical details are concerned, this is a lot of 
code in places I'm not all that familiar with, and like everyone else I have 
fr too many things on my plate :)

Steven  co:

A couple of procedural details:

1 There's no need at all to remove old patches when you put new ones up, in 
fact it's preferable to leave the old ones there. Just name them _all_ 
SOLR-5302.patch. The newest version will be in blue and all the older versions 
will be gray and they're listed in date order so it's really easy to know the 
order and look at changes version-to-version should that be necessary.

2 At some point when we're in some agreement (very soon I hope!), I'll commit 
the patch to trunk where we can bang on it a while before merging into 4x. I'll 
try to turn any new patches around in a day or less when we get to that point. 
I'm weaseling here since I'll be traveling for 10 days or so starting this 
weekend, otherwise I should be faster

I applied the patch to trunk and there are two issues:

3a A couple of files have this: import org.apache.mahout.math.Arrays;, and 
as far as I can tell it only is for the toString operation in error messages. 
The code compiles if we just use the java.util.Arrays import. I'd rather not 
introduce a new dependency so how about switching to java.util.Arrays?

3b Trying to run the tests on trunk at least gives this error: dynamicField 
can not have a default value: *_i.  (there are a couple of others). See 
SOLR-5227 which CHANGES.txt claims that setting the default and required 
options was silently ignored anyway as of 4.5 and emits a new init error... 
Removing the default assignments gets us past the initialization error, but 
then several tests fail, stack trace at the end (TRUNK), I haven't pursued it 
yet:

Thanks loads for taking this all no and contributing it back! I'll do my best 
to get it into the code base as fast as possible. And the patch comes with 
documentation too! How cool is that!

Erick

java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([22E0CD041D7B8CF3]:0)
at 
org.apache.solr.analytics.util.valuesource.MissFieldSource.description(MissFieldSource.java:52)
at 
org.apache.lucene.queries.function.ValueSource.toString(ValueSource.java:58)
at 
org.apache.solr.analytics.statistics.StatsCollectorSupplierFactory.create(StatsCollectorSupplierFactory.java:159)
at 
org.apache.solr.analytics.accumulator.BasicAccumulator.init(BasicAccumulator.java:60)
at 
org.apache.solr.analytics.accumulator.BasicAccumulator.create(BasicAccumulator.java:84)
at 
org.apache.solr.analytics.request.AnalyticsStats.execute(AnalyticsStats.java:82)
at 
org.apache.solr.handler.component.AnalyticsComponent.process(AnalyticsComponent.java:44)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:209)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1861)
at org.apache.solr.util.TestHarness.query(TestHarness.java:291)
at org.apache.solr.util.TestHarness.query(TestHarness.java:273)
at 
org.apache.solr.analytics.NoFacetTest.beforeClass(NoFacetTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:677)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at

[jira] [Commented] (SOLR-5302) Analytics Component

2013-10-05 Thread David Arthur (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787221#comment-13787221
 ] 

David Arthur commented on SOLR-5302:


Would this support performing analytics on the score? For example, I'd like to 
roll up results by some fields and sum the scores. 

Awesome patch by the way. 

 Analytics Component
 ---

 Key: SOLR-5302
 URL: https://issues.apache.org/jira/browse/SOLR-5302
 Project: Solr
  Issue Type: New Feature
Reporter: Steven Bower
Assignee: Erick Erickson
 Attachments: Search Analytics Component.pdf, 
 solr_analytics-2013.10.04-2.patch, Statistical Expressions.pdf


 This ticket is to track a replacement for the StatsComponent. The 
 AnalyticsComponent supports the following features:
 * All functionality of StatsComponent (SOLR-4499)
 * Field Faceting (SOLR-3435)
 ** Support for limit
 ** Sorting (bucket name or any stat in the bucket
 ** Support for offset
 * Range Faceting
 ** Supports all options of standard range faceting
 * Query Faceting (SOLR-2925)
 * Ability to use overall/field facet statistics as input to range/query 
 faceting (ie calc min/max date and then facet over that range
 * Support for more complex aggregate/mapping operations (SOLR-1622)
 ** Aggregations: min, max, sum, sum-of-square, count, missing, stddev, mean, 
 median, percentiles
 ** Operations: negation, abs, add, multiply, divide, power, log, date math, 
 string reversal, string concat
 ** Easily pluggable framework to add additional operations
 * New / cleaner output format
 Outstanding Issues:
 * Multi-value field support for stats (supported for faceting)
 * Multi-shard support (may not be possible for some operations, eg median)



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5236) Use broadword bit selection in EliasFanoDecoder

2013-10-05 Thread Paul Elschot (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787245#comment-13787245
 ] 

Paul Elschot commented on LUCENE-5236:
--

LUCENE-5109 also modifies the EliasFanoDecoder.advanceToValue method.

 Use broadword bit selection in EliasFanoDecoder
 ---

 Key: LUCENE-5236
 URL: https://issues.apache.org/jira/browse/LUCENE-5236
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Paul Elschot
Priority: Minor

 Try and speed up decoding



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5240) additional safety in Tokenizer state machine


[ 
https://issues.apache.org/jira/browse/LUCENE-5240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787258#comment-13787258
 ] 

ASF subversion and git services commented on LUCENE-5240:
-

Commit 1529482 from [~rcmuir] in branch 'dev/trunk'
[ https://svn.apache.org/r1529482 ]

LUCENE-5240: additional safety in Tokenizer state machine

 additional safety in Tokenizer state machine
 

 Key: LUCENE-5240
 URL: https://issues.apache.org/jira/browse/LUCENE-5240
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir
 Attachments: LUCENE-5240.patch, LUCENE-5240.patch, LUCENE-5240.patch


 {code}
* bNOTE:/b 
* The default implementation closes the input Reader, so
* be sure to call codesuper.close()/code when overriding this method.
*/
   @Override
   public void close() throws IOException {
 {code}
 We can add a simple check for this easily now in setReader. I found a few 
 bugs, and fixed all except TrieTokenizer in solr (I am lost here... somewhere 
 i have a patch to remove this thing).



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5240) additional safety in Tokenizer state machine

2013-10-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787273#comment-13787273
 ] 

ASF subversion and git services commented on LUCENE-5240:
-

Commit 1529486 from [~rcmuir] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1529486 ]

LUCENE-5240: additional safety in Tokenizer state machine

 additional safety in Tokenizer state machine
 

 Key: LUCENE-5240
 URL: https://issues.apache.org/jira/browse/LUCENE-5240
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir
 Attachments: LUCENE-5240.patch, LUCENE-5240.patch, LUCENE-5240.patch


 {code}
* bNOTE:/b 
* The default implementation closes the input Reader, so
* be sure to call codesuper.close()/code when overriding this method.
*/
   @Override
   public void close() throws IOException {
 {code}
 We can add a simple check for this easily now in setReader. I found a few 
 bugs, and fixed all except TrieTokenizer in solr (I am lost here... somewhere 
 i have a patch to remove this thing).



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5240) additional safety in Tokenizer state machine

2013-10-05 Thread Robert Muir (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir resolved LUCENE-5240.
-

   Resolution: Fixed
Fix Version/s: 4.6
   5.0

 additional safety in Tokenizer state machine
 

 Key: LUCENE-5240
 URL: https://issues.apache.org/jira/browse/LUCENE-5240
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir
 Fix For: 5.0, 4.6

 Attachments: LUCENE-5240.patch, LUCENE-5240.patch, LUCENE-5240.patch


 {code}
* bNOTE:/b 
* The default implementation closes the input Reader, so
* be sure to call codesuper.close()/code when overriding this method.
*/
   @Override
   public void close() throws IOException {
 {code}
 We can add a simple check for this easily now in setReader. I found a few 
 bugs, and fixed all except TrieTokenizer in solr (I am lost here... somewhere 
 i have a patch to remove this thing).



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Roadmap for fixing features broken by core autodiscovery

2013-10-05 Thread Erick Erickson

Right, let's move this discussion to SOLR-4779. There's some history
here. Sharing named config sets got a bit wrapped up in sharing the
underlying solrconfig object. This latter has been taken off the
table, but we should discuss fixing Trey's issues up. Here's what the
thinking was:
There would be a directory like solr_home/configs/configset1,
solr_home/configs/configset2, etc. Then a new parameter for
core.properties or create or whatever like configset=configset1 that
would be smart enough to look in solr_home/configs for an entire
conf directory named configste1.

Trey:
Does that work for your case? If so, please add your comments to 4779
and we can take it from there. FWIW, I don't think this is especially
hard, but time is always at a premium.

Erick

On Fri, Oct 4, 2013 at 6:51 PM, Shawn Heisey s...@elyograg.org wrote:
On 10/4/2013 7:21 PM, Trey Grainger wrote:
There are two use-cases that appear broken with the new core
auto-discovery mechanism:

*1) The Core Admin Handler's CREATE command no longer works to create
brand new cores*
(unless you have logged on the box and created the core's directory
structure manually, which largely defeats the purpose of the CREATE
command). With the old Solr.xml format, we could spin up as many cores
as we wanted to dynamically with the following command:
http://localhost:8983/solr/admin/cores?action=CREATEname=newCore1instanceDir=collection1dataDir=newCore1/data
...
http://localhost:8983/solr/admin/cores?action=CREATEname=newCoreNinstanceDir=collection1dataDir=newCoreN/data

In the new core discovery mode, this exception is now thrown:
Error CREATEing SolrCore 'newCore1': Could not create a new core in
solr/collection1/as another core is already defined there

The CREATE action has *always* required that you have your configuration
on the disk before you call it. You are sharing the instanceDir, which
is the only reason you can skip that step.

If you want completely dynamic creation, use SolrCloud, which keeps the
config in zookeeper and requires ZERO config information to exist on the
disk.

*2) Having a shared configuration directory (instanceDir) across many
cores no longer works*.
Every core has to have it's own conf/ directory, and this doesn't seem
to be overridable any longer. Previously, it was possible to have many
cores share the same instanceDir (and just override their dataDir for
obvious reasons). Now, it is necessary to copy and paste identical
config files for each Solr core.

From what I understand talking to the people that worked on this, the
lack of a shared instanceDir was completely deliberate. It's the only
way that core discovery can work in any kind of predictable and sane
manner. The entire point of it is that every core is self-contained and
solr.xml isn't used to tell Solr about them.

I personally have never tried to share the instanceDir. I do have
shared configs, though - my corename/conf directories have symlinks to a
shared config directory. I also don't dynamically create cores - I have
seven shards, each of which has a live core and a build core. There are
two other cores that serve as frontends, with the shards parameter in
the request handlers.

I don't know if there's already a current roadmap for fixing this. I
saw https://issues.apache.org/jira/browse/SOLR-4478, which suggested
replacing instanceDir with the ability to specify a named configSet.
This solves problem 2, but not problem1 (since you still can't have
multiple core.properties files in the same folder). Based on Erick's
comments in the JIRA ticket, it also sounds like this ticket is also
dead at the moment.

There is definitely a need to have a shared config directory - whether
that is through a configSet or an explicit indexDir doesn't matter to
me. There's also a need to be able to dynamically create Solr cores
from external systems. I currently can't upgrade to core auto discovery
because it doesn't allow dynamic core creation. Does anyone have some
thoughts on how to best get these features working again under core
autodiscovery? Adding instanceDir to core.properties seems like an easy
solution, but there must be a desire not to do that or it would probably
have already been done.

Thankfully, you do not need to upgrade to core discovery anytime soon.
All future 4.x versions will support the old format, and any problems
with that will be considered bugs. It will be mandatory in Solr 5.0,
which currently doesn't have any kind of release roadmap or timeframe.
I suspect that what we currently call SolrCloud will also be mandatory
in 5.0, and that gives you shared configs with zookeeper. Requiring
zookeeper allows completely dynamic core/collection creation, because
the only thing that will be on the disk is the index and transaction log
data.

Thanks,
Shawn

-
To unsubscribe, e-mail:

[jira] [Commented] (SOLR-4478) Allow cores to specify a named config set in non-SolrCloud mode

[
https://issues.apache.org/jira/browse/SOLR-4478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787277#comment-13787277
]

Trey Grainger commented on SOLR-4478:
-

(moving this from my previous e-mail to the solr-dev mailing list)

There are two use-cases that appear broken with the new core auto-discovery
mechanism:

1) *The Core Admin Handler's CREATE command no longer works to create brand new
cores*
(unless you have logged on the box and created the core's directory structure
manually, which largely defeats the purpose of the CREATE command). With the
old Solr.xml format, we could spin up as many cores as we wanted to dynamically
with the following command:
http://localhost:8983/solr/admin/cores?action=CREATEname=newCore1instanceDir=collection1dataDir=newCore1/data
...
http://localhost:8983/solr/admin/cores?action=CREATEname=newCoreNinstanceDir=collection1dataDir=newCoreN/data

In the new core discovery mode, this exception is now thrown:
Error CREATEing SolrCore 'newCore1': Could not create a new core in
solr/collection1/as another core is already defined there

The exception is being intentionally thrown in CorePropertiesLocator.java
because a core.properties file already exists in solr/collection1 (and only one
can exist per directory).

2) *Having a shared configuration directory (instanceDir) across many cores no
longer works.*
Every core has to have it's own conf/ directory, and this doesn't seem to be
overridable any longer. Previously, it was possible to have many cores share
the same instanceDir (and just override their dataDir for obvious reasons).
Now, it is necessary to copy and paste identical config files for each Solr
core.

I don't know if there's already a current roadmap for fixing this. I saw
https://issues.apache.org/jira/browse/SOLR-4478, which suggested replacing
instanceDir with the ability to specify a named configSet. This solves problem
2, but not problem1 (since you still can't have multiple core.properties files
in the same folder). Based on Erick's comments in the JIRA ticket, it also
sounds like this ticket is also dead at the moment.

There is definitely a need to have a shared config directory - whether that is
through a configSet or an explicit indexDir doesn't matter to me. There's also
a need to be able to dynamically create Solr cores from external systems. I
currently can't upgrade to core auto discovery because it doesn't allow dynamic
core creation. Does anyone have some thoughts on how to best get these
features working again under core autodiscovery? Adding instanceDir to
core.properties seems like an easy solution, but there must be a desire not to
do that or it would probably have already been done.

I'm happy to contribute some time to resolving this if there is agreed upon
path forward.

Allow cores to specify a named config set in non-SolrCloud mode
---

Key: SOLR-4478
URL: https://issues.apache.org/jira/browse/SOLR-4478
Project: Solr
Issue Type: Improvement
Affects Versions: 4.2, 5.0
Reporter: Erick Erickson
Attachments: SOLR-4478.patch, SOLR-4478.patch

Part of moving forward to the new way, after SOLR-4196 etc... I propose an
additional parameter specified on the core node in solr.xml or as a
parameter in the discovery mode core.properties file, call it configSet,
where the value provided is a path to a directory, either absolute or
relative. Really, this is as though you copied the conf directory somewhere
to be used by more than one core.
Straw-man: There will be a directory solr_home/configsets which will be the
default. If the configSet parameter is, say, myconf, then I'd expect a
directory named myconf to exist in solr_home/configsets, which would look
something like
solr_home/configsets/myconf/schema.xml
solrconfig.xml
stopwords.txt
velocity
velocity/query.vm
etc.
If multiple cores used the same configSet, schema, solrconfig etc. would all
be shared (i.e. shareSchema=true would be assumed). I don't see a good
use-case for _not_ sharing schemas, so I don't propose to allow this to be
turned off. Hmmm, what if shareSchema is explicitly set to false in the
solr.xml or properties file? I'd guess it should be honored but maybe log a
warning?
Mostly I'm putting this up for comments. I know that there are already
thoughts about how this all should work floating around, so before I start
any work on this I thought I'd at least get an idea of whether this is the
way people are thinking about going.
Configset can be either a relative or absolute path, if relative it's assumed
to be relative to solr_home.
Thoughts?

--
This

[jira] [Commented] (SOLR-4478) Allow cores to specify a named config set in non-SolrCloud mode

[
https://issues.apache.org/jira/browse/SOLR-4478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787278#comment-13787278
]

Trey Grainger commented on SOLR-4478:
-

(Eric's response to my post)

Trey:
Does that work for your case? If so, please add your comments to 4779
and we can take it from there. FWIW, I don't think this is especially
hard, but time is always at a premium.

Allow cores to specify a named config set in non-SolrCloud mode
---

--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5236) Use broadword bit selection in EliasFanoDecoder

2013-10-05 Thread Paul Elschot (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Elschot updated LUCENE-5236:
-

Attachment: LUCENE-5236.patch

The patch of 5 Oct 2013 includes my modifications for LUCENE-5109.
The patch also adds some tests.

This passes TestEliasFanoSequence and TestEliasFanoDocIdSet.

I did not do any performance tests for broadword bit selection yet.
The times taken by some of the longer tests are encouraging, but that does not 
mean much.

 Use broadword bit selection in EliasFanoDecoder
 ---

 Key: LUCENE-5236
 URL: https://issues.apache.org/jira/browse/LUCENE-5236
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Paul Elschot
Priority: Minor
 Attachments: LUCENE-5236.patch


 Try and speed up decoding



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-4478) Allow cores to specify a named config set in non-SolrCloud mode

[
https://issues.apache.org/jira/browse/SOLR-4478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787278#comment-13787278
]

Trey Grainger edited comment on SOLR-4478 at 10/5/13 5:47 PM:
--

(Erick's response to my post)

Trey:
Does that work for your case? If so, please add your comments to 4779
and we can take it from there. FWIW, I don't think this is especially
hard, but time is always at a premium.

was (Author: solrtrey):
(Eric's response to my post)

Trey:
Does that work for your case? If so, please add your comments to 4779
and we can take it from there. FWIW, I don't think this is especially
hard, but time is always at a premium.

Allow cores to specify a named config set in non-SolrCloud mode
---

--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-4478) Allow cores to specify a named config set in non-SolrCloud mode


[ 
https://issues.apache.org/jira/browse/SOLR-4478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787282#comment-13787282
 ] 

Trey Grainger edited comment on SOLR-4478 at 10/5/13 5:50 PM:
--

Hi Erick,

Yes, that resolves the hardest of the two problems.  The other issue is that 
since a dedicated folder is now required per-core (to hold the core.properties 
file), the core _CREATE_ command needs to now also be able to create the folder 
for the new core if it doesn't exist.  Something like:
http://localhost:8983/solr/admin/cores?action=CREATEname=newCore; 
*coreDir=cores/newCore* configset=sharedconfig

Alternatively, _instanceDir_ could continue to serve that function (instead of 
being deprecated):
http://localhost:8983/solr/admin/cores?action=CREATEname=newCore; 
*instanceDir=cores/newCore* configset=sharedconfig

I think the combination of adding configSet and adding the ability for the 
CREATE command to actually create the new folder to hold core.properties should 
handle the use case.


was (Author: solrtrey):
Hi Erick,

Yes, that resolves the hardest of the two problems.  The other issue is that 
since a dedicated folder is now required per-core (to hold the core.properties 
file), the core _CREATE_ command needs to now also be able to create the folder 
for the new core if it doesn't exist.  Something like:
http://localhost:8983/solr/admin/cores?action=CREATEname=newCore*coreDir=cores/newCore*configset=sharedconfig

Alternatively, _instanceDir_ could continue to serve that function (instead of 
being deprecated):
http://localhost:8983/solr/admin/cores?action=CREATEname=newCore*instanceDir=cores/newCore*configset=sharedconfig

I think the combination of adding configSet and adding the ability for the 
CREATE command to actually create the new folder to hold core.properties should 
handle the use case.

 Allow cores to specify a named config set in non-SolrCloud mode
 ---

 Key: SOLR-4478
 URL: https://issues.apache.org/jira/browse/SOLR-4478
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2, 5.0
Reporter: Erick Erickson
 Attachments: SOLR-4478.patch, SOLR-4478.patch


 Part of moving forward to the new way, after SOLR-4196 etc... I propose an 
 additional parameter specified on the core node in solr.xml or as a 
 parameter in the discovery mode core.properties file, call it configSet, 
 where the value provided is a path to a directory, either absolute or 
 relative. Really, this is as though you copied the conf directory somewhere 
 to be used by more than one core.
 Straw-man: There will be a directory solr_home/configsets which will be the 
 default. If the configSet parameter is, say, myconf, then I'd expect a 
 directory named myconf to exist in solr_home/configsets, which would look 
 something like
 solr_home/configsets/myconf/schema.xml
   solrconfig.xml
   stopwords.txt
   velocity
   velocity/query.vm
 etc.
 If multiple cores used the same configSet, schema, solrconfig etc. would all 
 be shared (i.e. shareSchema=true would be assumed). I don't see a good 
 use-case for _not_ sharing schemas, so I don't propose to allow this to be 
 turned off. Hmmm, what if shareSchema is explicitly set to false in the 
 solr.xml or properties file? I'd guess it should be honored but maybe log a 
 warning?
 Mostly I'm putting this up for comments. I know that there are already 
 thoughts about how this all should work floating around, so before I start 
 any work on this I thought I'd at least get an idea of whether this is the 
 way people are thinking about going.
 Configset can be either a relative or absolute path, if relative it's assumed 
 to be relative to solr_home.
 Thoughts?



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4478) Allow cores to specify a named config set in non-SolrCloud mode

2013-10-05 Thread ASF subversion and git services (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787282#comment-13787282
]

Trey Grainger commented on SOLR-4478:
-

Hi Erick,

Yes, that resolves the hardest of the two problems. The other issue is that
since a dedicated folder is now required per-core (to hold the core.properties
file), the core _CREATE_ command needs to now also be able to create the folder
for the new core if it doesn't exist. Something like:
http://localhost:8983/solr/admin/cores?action=CREATEname=newCore*coreDir=cores/newCore*configset=sharedconfig

Alternatively, _instanceDir_ could continue to serve that function (instead of
being deprecated):
http://localhost:8983/solr/admin/cores?action=CREATEname=newCore*instanceDir=cores/newCore*configset=sharedconfig

I think the combination of adding configSet and adding the ability for the
CREATE command to actually create the new folder to hold core.properties should
handle the use case.

Allow cores to specify a named config set in non-SolrCloud mode
---

--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5305) Admin UI - Reloading System-Information on Dashboard does not work anymore

2013-10-05 Thread Stefan Matheis (steffkes) (JIRA)

Stefan Matheis (steffkes) created SOLR-5305:
---

 Summary: Admin UI - Reloading System-Information on Dashboard does 
not work anymore
 Key: SOLR-5305
 URL: https://issues.apache.org/jira/browse/SOLR-5305
 Project: Solr
  Issue Type: Bug
  Components: web gui
Affects Versions: 4.5
Reporter: Stefan Matheis (steffkes)
Assignee: Stefan Matheis (steffkes)
 Fix For: 5.0, 4.6


[~yriveiro] mentioned on #solr that reloading the system information stopped 
working:

{quote}yriveiro the system's button for reload graph on dashboard doesn't 
work for me on 4.5, someone else with the same behaviour?
yriveiro the debugger console of chrome says: Uncaught ReferenceError: 
{{environment_basepath}} is not defined{quote}

{{grep}}'ing the source shows one leftover from our Refactoring for SOLR-3633 
which still (tries) to use the {{environment_basepath}} - which is infact for 
reloading the information on the dashboard.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5305) Admin UI - Reloading System-Information on Dashboard does not work anymore

2013-10-05 Thread Stefan Matheis (steffkes) (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Matheis (steffkes) updated SOLR-5305:


Attachment: SOLR-5305.patch

 Admin UI - Reloading System-Information on Dashboard does not work anymore
 --

 Key: SOLR-5305
 URL: https://issues.apache.org/jira/browse/SOLR-5305
 Project: Solr
  Issue Type: Bug
  Components: web gui
Affects Versions: 4.5
Reporter: Stefan Matheis (steffkes)
Assignee: Stefan Matheis (steffkes)
 Fix For: 5.0, 4.6

 Attachments: SOLR-5305.patch


 [~yriveiro] mentioned on #solr that reloading the system information stopped 
 working:
 {quote}yriveiro the system's button for reload graph on dashboard doesn't 
 work for me on 4.5, someone else with the same behaviour?
 yriveiro the debugger console of chrome says: Uncaught ReferenceError: 
 {{environment_basepath}} is not defined{quote}
 {{grep}}'ing the source shows one leftover from our Refactoring for SOLR-3633 
 which still (tries) to use the {{environment_basepath}} - which is infact for 
 reloading the information on the dashboard.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5305) Admin UI - Reloading System-Information on Dashboard does not work anymore


[ 
https://issues.apache.org/jira/browse/SOLR-5305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787311#comment-13787311
 ] 

ASF subversion and git services commented on SOLR-5305:
---

Commit 1529502 from [~steffkes] in branch 'dev/trunk'
[ https://svn.apache.org/r1529502 ]

SOLR-5305: Admin UI - Reloading System-Information on Dashboard does not work 
anymore

 Admin UI - Reloading System-Information on Dashboard does not work anymore
 --

 Key: SOLR-5305
 URL: https://issues.apache.org/jira/browse/SOLR-5305
 Project: Solr
  Issue Type: Bug
  Components: web gui
Affects Versions: 4.5
Reporter: Stefan Matheis (steffkes)
Assignee: Stefan Matheis (steffkes)
 Fix For: 5.0, 4.6

 Attachments: SOLR-5305.patch


 [~yriveiro] mentioned on #solr that reloading the system information stopped 
 working:
 {quote}yriveiro the system's button for reload graph on dashboard doesn't 
 work for me on 4.5, someone else with the same behaviour?
 yriveiro the debugger console of chrome says: Uncaught ReferenceError: 
 {{environment_basepath}} is not defined{quote}
 {{grep}}'ing the source shows one leftover from our Refactoring for SOLR-3633 
 which still (tries) to use the {{environment_basepath}} - which is infact for 
 reloading the information on the dashboard.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5305) Admin UI - Reloading System-Information on Dashboard does not work anymore

2013-10-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787312#comment-13787312
 ] 

ASF subversion and git services commented on SOLR-5305:
---

Commit 1529503 from [~steffkes] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1529503 ]

SOLR-5305: Admin UI - Reloading System-Information on Dashboard does not work 
anymore (merge r1529502)

 Admin UI - Reloading System-Information on Dashboard does not work anymore
 --

 Key: SOLR-5305
 URL: https://issues.apache.org/jira/browse/SOLR-5305
 Project: Solr
  Issue Type: Bug
  Components: web gui
Affects Versions: 4.5
Reporter: Stefan Matheis (steffkes)
Assignee: Stefan Matheis (steffkes)
 Fix For: 5.0, 4.6

 Attachments: SOLR-5305.patch


 [~yriveiro] mentioned on #solr that reloading the system information stopped 
 working:
 {quote}yriveiro the system's button for reload graph on dashboard doesn't 
 work for me on 4.5, someone else with the same behaviour?
 yriveiro the debugger console of chrome says: Uncaught ReferenceError: 
 {{environment_basepath}} is not defined{quote}
 {{grep}}'ing the source shows one leftover from our Refactoring for SOLR-3633 
 which still (tries) to use the {{environment_basepath}} - which is infact for 
 reloading the information on the dashboard.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-5305) Admin UI - Reloading System-Information on Dashboard does not work anymore

2013-10-05 Thread Stefan Matheis (steffkes) (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Matheis (steffkes) resolved SOLR-5305.
-

Resolution: Fixed

 Admin UI - Reloading System-Information on Dashboard does not work anymore
 --

 Key: SOLR-5305
 URL: https://issues.apache.org/jira/browse/SOLR-5305
 Project: Solr
  Issue Type: Bug
  Components: web gui
Affects Versions: 4.5
Reporter: Stefan Matheis (steffkes)
Assignee: Stefan Matheis (steffkes)
 Fix For: 5.0, 4.6

 Attachments: SOLR-5305.patch


 [~yriveiro] mentioned on #solr that reloading the system information stopped 
 working:
 {quote}yriveiro the system's button for reload graph on dashboard doesn't 
 work for me on 4.5, someone else with the same behaviour?
 yriveiro the debugger console of chrome says: Uncaught ReferenceError: 
 {{environment_basepath}} is not defined{quote}
 {{grep}}'ing the source shows one leftover from our Refactoring for SOLR-3633 
 which still (tries) to use the {{environment_basepath}} - which is infact for 
 reloading the information on the dashboard.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5302) Analytics Component

2013-10-05 Thread Houston Putman (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787335#comment-13787335
 ] 

Houston Putman commented on SOLR-5302:
--

Not currently, but I think it would be very easy to add that functionality. We 
tried to make adding new features as painless as possible.

 Analytics Component
 ---

 Key: SOLR-5302
 URL: https://issues.apache.org/jira/browse/SOLR-5302
 Project: Solr
  Issue Type: New Feature
Reporter: Steven Bower
Assignee: Erick Erickson
 Attachments: Search Analytics Component.pdf, 
 solr_analytics-2013.10.04-2.patch, Statistical Expressions.pdf


 This ticket is to track a replacement for the StatsComponent. The 
 AnalyticsComponent supports the following features:
 * All functionality of StatsComponent (SOLR-4499)
 * Field Faceting (SOLR-3435)
 ** Support for limit
 ** Sorting (bucket name or any stat in the bucket
 ** Support for offset
 * Range Faceting
 ** Supports all options of standard range faceting
 * Query Faceting (SOLR-2925)
 * Ability to use overall/field facet statistics as input to range/query 
 faceting (ie calc min/max date and then facet over that range
 * Support for more complex aggregate/mapping operations (SOLR-1622)
 ** Aggregations: min, max, sum, sum-of-square, count, missing, stddev, mean, 
 median, percentiles
 ** Operations: negation, abs, add, multiply, divide, power, log, date math, 
 string reversal, string concat
 ** Easily pluggable framework to add additional operations
 * New / cleaner output format
 Outstanding Issues:
 * Multi-value field support for stats (supported for faceting)
 * Multi-shard support (may not be possible for some operations, eg median)



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4816) Add document routing to CloudSolrServer


[ 
https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787341#comment-13787341
 ] 

Erick Erickson commented on SOLR-4816:
--

Hey Joel! Now that you have karma, you can assign this to yourself if you want 
and Mark doesn't object!

 Add document routing to CloudSolrServer
 ---

 Key: SOLR-4816
 URL: https://issues.apache.org/jira/browse/SOLR-4816
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Affects Versions: 4.3
Reporter: Joel Bernstein
Assignee: Mark Miller
Priority: Minor
 Fix For: 4.6

 Attachments: RequestTask-removal.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch


 This issue adds the following enhancements to CloudSolrServer's update logic:
 1) Document routing: Updates are routed directly to the correct shard leader 
 eliminating document routing at the server.
 2) Optional parallel update execution: Updates for each shard are executed in 
 a separate thread so parallel indexing can occur across the cluster.
 These enhancements should allow for near linear scalability on indexing 
 throughput.
 Usage:
 CloudSolrServer cloudClient = new CloudSolrServer(zkAddress);
 cloudClient.setParallelUpdates(true); 
 SolrInputDocument doc1 = new SolrInputDocument();
 doc1.addField(id, 0);
 doc1.addField(a_t, hello1);
 SolrInputDocument doc2 = new SolrInputDocument();
 doc2.addField(id, 2);
 doc2.addField(a_t, hello2);
 UpdateRequest request = new UpdateRequest();
 request.add(doc1);
 request.add(doc2);
 request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false);
 NamedList response = cloudClient.request(request); // Returns a backwards 
 compatible condensed response.
 //To get more detailed response down cast to RouteResponse:
 CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response;



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5236) Use broadword bit selection in EliasFanoDecoder

2013-10-05 Thread Paul Elschot (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787360#comment-13787360
 ] 

Paul Elschot commented on LUCENE-5236:
--

See http://people.apache.org/~jpountz/doc_id_sets.html for the 
EliasFanoDocIdSet without index and without broadword bit selection.

With index and broadword bit selection on my 32 bit machine, worst case 
performance for load factor -1 (1/10) for any advance(), relative to 
FixedBitSet, is  -3.42 (2 log scale)  for  advance(3571).
The other advance() cases have better worst cases, so broadword bit selection 
really helps there, too.

 Use broadword bit selection in EliasFanoDecoder
 ---

 Key: LUCENE-5236
 URL: https://issues.apache.org/jira/browse/LUCENE-5236
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Paul Elschot
Priority: Minor
 Attachments: LUCENE-5236.patch


 Try and speed up decoding



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4816) Add document routing to CloudSolrServer

2013-10-05 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787368#comment-13787368
 ] 

Mark Miller commented on SOLR-4816:
---

This should be resolved as fixed for 4.5. Any further work should be done in 
new issues. I'd do it now but I'm on mobile. 

 Add document routing to CloudSolrServer
 ---

 Key: SOLR-4816
 URL: https://issues.apache.org/jira/browse/SOLR-4816
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Affects Versions: 4.3
Reporter: Joel Bernstein
Assignee: Mark Miller
Priority: Minor
 Fix For: 4.6

 Attachments: RequestTask-removal.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, 
 SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch


 This issue adds the following enhancements to CloudSolrServer's update logic:
 1) Document routing: Updates are routed directly to the correct shard leader 
 eliminating document routing at the server.
 2) Optional parallel update execution: Updates for each shard are executed in 
 a separate thread so parallel indexing can occur across the cluster.
 These enhancements should allow for near linear scalability on indexing 
 throughput.
 Usage:
 CloudSolrServer cloudClient = new CloudSolrServer(zkAddress);
 cloudClient.setParallelUpdates(true); 
 SolrInputDocument doc1 = new SolrInputDocument();
 doc1.addField(id, 0);
 doc1.addField(a_t, hello1);
 SolrInputDocument doc2 = new SolrInputDocument();
 doc2.addField(id, 2);
 doc2.addField(a_t, hello2);
 UpdateRequest request = new UpdateRequest();
 request.add(doc1);
 request.add(doc2);
 request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false);
 NamedList response = cloudClient.request(request); // Returns a backwards 
 compatible condensed response.
 //To get more detailed response down cast to RouteResponse:
 CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response;



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5302) Analytics Component

2013-10-05 Thread Steven Bower (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787371#comment-13787371
 ] 

Steven Bower commented on SOLR-5302:


[~erickerickson] Will check out trunk tonight and apply/test.. also will start 
creating linked sub-tickets for requests here but also the laundry list of 
things we plan on adding over time.

 Analytics Component
 ---

 Key: SOLR-5302
 URL: https://issues.apache.org/jira/browse/SOLR-5302
 Project: Solr
  Issue Type: New Feature
Reporter: Steven Bower
Assignee: Erick Erickson
 Attachments: Search Analytics Component.pdf, 
 solr_analytics-2013.10.04-2.patch, Statistical Expressions.pdf


 This ticket is to track a replacement for the StatsComponent. The 
 AnalyticsComponent supports the following features:
 * All functionality of StatsComponent (SOLR-4499)
 * Field Faceting (SOLR-3435)
 ** Support for limit
 ** Sorting (bucket name or any stat in the bucket
 ** Support for offset
 * Range Faceting
 ** Supports all options of standard range faceting
 * Query Faceting (SOLR-2925)
 * Ability to use overall/field facet statistics as input to range/query 
 faceting (ie calc min/max date and then facet over that range
 * Support for more complex aggregate/mapping operations (SOLR-1622)
 ** Aggregations: min, max, sum, sum-of-square, count, missing, stddev, mean, 
 median, percentiles
 ** Operations: negation, abs, add, multiply, divide, power, log, date math, 
 string reversal, string concat
 ** Easily pluggable framework to add additional operations
 * New / cleaner output format
 Outstanding Issues:
 * Multi-value field support for stats (supported for faceting)
 * Multi-shard support (may not be possible for some operations, eg median)



--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4470) Support for basic http auth in internal solr requests

2013-10-05 Thread JIRA

[
https://issues.apache.org/jira/browse/SOLR-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13787389#comment-13787389
]

Jan Høydahl commented on SOLR-4470:
---

I think the only way it's going to get into the code base is to first commit to
trunk for baking and then backport. I screwed up somewhere along the way doing
too many changes at once. If you'd get green light to spend some time on this,
could you perhaps help out with a trunk port? I'm planning to look into this
again myself soonish.

Support for basic http auth in internal solr requests
-

Key: SOLR-4470
URL: https://issues.apache.org/jira/browse/SOLR-4470
Project: Solr
Issue Type: New Feature
Components: clients - java, multicore, replication (java), SolrCloud
Affects Versions: 4.0
Reporter: Per Steffensen
Assignee: Jan Høydahl
Labels: authentication, https, solrclient, solrcloud, ssl
Fix For: 4.6

Attachments: SOLR-4470_branch_4x_r1452629.patch,
SOLR-4470_branch_4x_r1452629.patch, SOLR-4470_branch_4x_r145.patch,
SOLR-4470.patch, SOLR-4470.patch

We want to protect any HTTP-resource (url). We want to require credentials no
matter what kind of HTTP-request you make to a Solr-node.
It can faily easy be acheived as described on
http://wiki.apache.org/solr/SolrSecurity. This problem is that Solr-nodes
also make internal request to other Solr-nodes, and for it to work
credentials need to be provided here also.
Ideally we would like to forward credentials from a particular request to
all the internal sub-requests it triggers. E.g. for search and update
request.
But there are also internal requests
* that only indirectly/asynchronously triggered from outside requests (e.g.
shard creation/deletion/etc based on calls to the Collection API)
* that do not in any way have relation to an outside super-request (e.g.
replica synching stuff)
We would like to aim at a solution where original credentials are
forwarded when a request directly/synchronously trigger a subrequest, and
fallback to a configured internal credentials for the
asynchronous/non-rooted requests.
In our solution we would aim at only supporting basic http auth, but we would
like to make a framework around it, so that not to much refactoring is
needed if you later want to make support for other kinds of auth (e.g. digest)
We will work at a solution but create this JIRA issue early in order to get
input/comments from the community as early as possible.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-4816) Add document routing to CloudSolrServer