date:20140623


 [ 
https://issues.apache.org/jira/browse/SOLR-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler reopened SOLR-6178:
-

  Assignee: Uwe Schindler

 Deprecate Jaspell suggester
 ---

 Key: SOLR-6178
 URL: https://issues.apache.org/jira/browse/SOLR-6178
 Project: Solr
  Issue Type: Improvement
  Components: spellchecker
Reporter: Michael McCandless
Assignee: Uwe Schindler
 Fix For: 5.0, 4.10

 Attachments: SOLR-6178.patch


 Right now Solr defaults to Jaspell, but we've deprecated it in LUCENE-5775 
 ... and in trunk I'd like to remove it.  But first we need to fix Solr to not 
 default to it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6178) Deprecate Jaspell suggester


 [ 
https://issues.apache.org/jira/browse/SOLR-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated SOLR-6178:


Fix Version/s: (was: 4.9)
   4.10

 Deprecate Jaspell suggester
 ---

 Key: SOLR-6178
 URL: https://issues.apache.org/jira/browse/SOLR-6178
 Project: Solr
  Issue Type: Improvement
  Components: spellchecker
Reporter: Michael McCandless
Assignee: Uwe Schindler
 Fix For: 5.0, 4.10

 Attachments: SOLR-6178.patch


 Right now Solr defaults to Jaspell, but we've deprecated it in LUCENE-5775 
 ... and in trunk I'd like to remove it.  But first we need to fix Solr to not 
 default to it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6178) Deprecate Jaspell suggester


[ 
https://issues.apache.org/jira/browse/SOLR-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14040505#comment-14040505
 ] 

Uwe Schindler commented on SOLR-6178:
-

I reverted and reopened for more discussion. The Change of defaults was moved 
to separate issue: SOLR-6185

 Deprecate Jaspell suggester
 ---

 Key: SOLR-6178
 URL: https://issues.apache.org/jira/browse/SOLR-6178
 Project: Solr
  Issue Type: Improvement
  Components: spellchecker
Reporter: Michael McCandless
Assignee: Uwe Schindler
 Fix For: 5.0, 4.10

 Attachments: SOLR-6178.patch


 Right now Solr defaults to Jaspell, but we've deprecated it in LUCENE-5775 
 ... and in trunk I'd like to remove it.  But first we need to fix Solr to not 
 default to it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: SSLMigration test hangs on FreeBSD blackhole

2014-06-23 Thread Dawid Weiss

Wait, let me see what's going on.
D.


On Mon, Jun 23, 2014 at 9:03 AM, Uwe Schindler u...@thetaphi.de wrote:

 Same happened again:
 https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4733/console



 Should I kill -3?



 -

 Uwe Schindler

 H.-H.-Meier-Allee 63, D-28213 Bremen

 http://www.thetaphi.de

 eMail: u...@thetaphi.de



 *From:* dawid.we...@gmail.com [mailto:dawid.we...@gmail.com] *On Behalf
 Of *Dawid Weiss
 *Sent:* Sunday, June 22, 2014 10:03 PM
 *To:* dev@lucene.apache.org
 *Subject:* Re: SSLMigration test hangs on FreeBSD blackhole





 This looks very weird because there's no actual test thread running. And
 the runner's thread is hung on readBytes (?)... This is very suspicious.



 Dawid



 On Sun, Jun 22, 2014 at 4:46 PM, Uwe Schindler u...@thetaphi.de wrote:

 I requested a stack dump:
 https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/565/console

[junit4] JVM J0: stdout was not empty, see:
 /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/temp/junit4-J0-20140621_125257_468.sysout
[junit4]  JVM J0: stdout (verbatim) 
[junit4] 2014-06-22 14:44:03
[junit4] Full thread dump OpenJDK 64-Bit Server VM (24.60-b09 mixed
 mode):
[junit4]
[junit4] RMI TCP Accept-0 daemon prio=5 tid=0x0008033e6800
 nid=0x846b95c00 runnable [0x76e7]
[junit4]java.lang.Thread.State: RUNNABLE
[junit4] at java.net.PlainSocketImpl.socketAccept(Native Method)
[junit4] at
 java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
[junit4] at java.net.ServerSocket.implAccept(ServerSocket.java:530)
[junit4] at java.net.ServerSocket.accept(ServerSocket.java:498)
[junit4] at
 sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:388)
[junit4] at
 sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:360)
[junit4] at java.lang.Thread.run(Thread.java:745)
[junit4]
[junit4] RMI RenewClean-[127.0.0.1:40915] daemon prio=5
 tid=0x0008517df000 nid=0x83c049800 in Object.wait() [0x72426000]
[junit4]java.lang.Thread.State: TIMED_WAITING (on object monitor)
[junit4] at java.lang.Object.wait(Native Method)
[junit4] at
 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
[junit4] - locked 0x00080de40760 (a
 java.lang.ref.ReferenceQueue$Lock)
[junit4] at
 sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:535)
[junit4] at java.lang.Thread.run(Thread.java:745)
[junit4]
[junit4] RMI Scheduler(0) daemon prio=5 tid=0x0008033e2800
 nid=0x84abf0c00 waiting on condition [0x7fffec5c8000]
[junit4]java.lang.Thread.State: TIMED_WAITING (parking)
[junit4] at sun.misc.Unsafe.park(Native Method)
[junit4] - parking to wait for  0x00080de3e008 (a
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
[junit4] at
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
[junit4] at
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
[junit4] at
 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
[junit4] at
 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
[junit4] at
 java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] at
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] at java.lang.Thread.run(Thread.java:745)
[junit4]
[junit4] GC Daemon daemon prio=5 tid=0x0008033e4000
 nid=0x837d98400 in Object.wait() [0x7090b000]
[junit4]java.lang.Thread.State: TIMED_WAITING (on object monitor)
[junit4] at java.lang.Object.wait(Native Method)
[junit4] at sun.misc.GC$Daemon.run(GC.java:117)
[junit4] - locked 0x00080de434a8 (a sun.misc.GC$LatencyLock)
[junit4]
[junit4] RMI Reaper prio=5 tid=0x0008033e3800 nid=0x83c050400 in
 Object.wait() [0x73638000]
[junit4]java.lang.Thread.State: WAITING (on object monitor)
[junit4] at java.lang.Object.wait(Native Method)
[junit4] at
 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
[junit4] - locked 0x00080de36d38 (a
 java.lang.ref.ReferenceQueue$Lock)
[junit4] at
 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
[junit4] at
 sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:351)
[junit4] at java.lang.Thread.run(Thread.java:745)
[junit4]
[junit4] RMI TCP Accept-0 daemon prio=5 tid=0x0008033e2000
 nid=0x850ca1800 runnable

Re: SSLMigration test hangs on FreeBSD blackhole

2014-06-23 Thread Dawid Weiss

I am looking. This looks very weird and seems to be something with the test
framework, but I don't know what this is yet.

Dawid


On Mon, Jun 23, 2014 at 10:11 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl
wrote:


 Wait, let me see what's going on.
 D.


 On Mon, Jun 23, 2014 at 9:03 AM, Uwe Schindler u...@thetaphi.de wrote:

 Same happened again:
 https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4733/console



 Should I kill -3?



 -

 Uwe Schindler

 H.-H.-Meier-Allee 63, D-28213 Bremen

 http://www.thetaphi.de

 eMail: u...@thetaphi.de



 *From:* dawid.we...@gmail.com [mailto:dawid.we...@gmail.com] *On Behalf
 Of *Dawid Weiss
 *Sent:* Sunday, June 22, 2014 10:03 PM
 *To:* dev@lucene.apache.org
 *Subject:* Re: SSLMigration test hangs on FreeBSD blackhole





 This looks very weird because there's no actual test thread running.
 And the runner's thread is hung on readBytes (?)... This is very suspicious.



 Dawid



 On Sun, Jun 22, 2014 at 4:46 PM, Uwe Schindler u...@thetaphi.de wrote:

 I requested a stack dump:
 https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/565/console

[junit4] JVM J0: stdout was not empty, see:
 /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/temp/junit4-J0-20140621_125257_468.sysout
[junit4]  JVM J0: stdout (verbatim) 
[junit4] 2014-06-22 14:44:03
[junit4] Full thread dump OpenJDK 64-Bit Server VM (24.60-b09 mixed
 mode):
[junit4]
[junit4] RMI TCP Accept-0 daemon prio=5 tid=0x0008033e6800
 nid=0x846b95c00 runnable [0x76e7]
[junit4]java.lang.Thread.State: RUNNABLE
[junit4] at java.net.PlainSocketImpl.socketAccept(Native Method)
[junit4] at
 java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
[junit4] at java.net.ServerSocket.implAccept(ServerSocket.java:530)
[junit4] at java.net.ServerSocket.accept(ServerSocket.java:498)
[junit4] at
 sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:388)
[junit4] at
 sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:360)
[junit4] at java.lang.Thread.run(Thread.java:745)
[junit4]
[junit4] RMI RenewClean-[127.0.0.1:40915] daemon prio=5
 tid=0x0008517df000 nid=0x83c049800 in Object.wait() [0x72426000]
[junit4]java.lang.Thread.State: TIMED_WAITING (on object monitor)
[junit4] at java.lang.Object.wait(Native Method)
[junit4] at
 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
[junit4] - locked 0x00080de40760 (a
 java.lang.ref.ReferenceQueue$Lock)
[junit4] at
 sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:535)
[junit4] at java.lang.Thread.run(Thread.java:745)
[junit4]
[junit4] RMI Scheduler(0) daemon prio=5 tid=0x0008033e2800
 nid=0x84abf0c00 waiting on condition [0x7fffec5c8000]
[junit4]java.lang.Thread.State: TIMED_WAITING (parking)
[junit4] at sun.misc.Unsafe.park(Native Method)
[junit4] - parking to wait for  0x00080de3e008 (a
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
[junit4] at
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
[junit4] at
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
[junit4] at
 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
[junit4] at
 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
[junit4] at
 java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] at
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] at java.lang.Thread.run(Thread.java:745)
[junit4]
[junit4] GC Daemon daemon prio=5 tid=0x0008033e4000
 nid=0x837d98400 in Object.wait() [0x7090b000]
[junit4]java.lang.Thread.State: TIMED_WAITING (on object monitor)
[junit4] at java.lang.Object.wait(Native Method)
[junit4] at sun.misc.GC$Daemon.run(GC.java:117)
[junit4] - locked 0x00080de434a8 (a sun.misc.GC$LatencyLock)
[junit4]
[junit4] RMI Reaper prio=5 tid=0x0008033e3800 nid=0x83c050400 in
 Object.wait() [0x73638000]
[junit4]java.lang.Thread.State: WAITING (on object monitor)
[junit4] at java.lang.Object.wait(Native Method)
[junit4] at
 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
[junit4] - locked 0x00080de36d38 (a
 java.lang.ref.ReferenceQueue$Lock)
[junit4] at
 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
[junit4] at

[jira] [Created] (LUCENE-5786) Unflushed/ truncated events file (hung testing subprocess)

Dawid Weiss created LUCENE-5786:
---

 Summary: Unflushed/ truncated events file (hung testing subprocess)
 Key: LUCENE-5786
 URL: https://issues.apache.org/jira/browse/LUCENE-5786
 Project: Lucene - Core
  Issue Type: Bug
  Components: general/test
Reporter: Dawid Weiss
Assignee: Dawid Weiss


This has happened several times on Jenkins, typically on 
SSLMigrationTest.testDistribSearch, but probably on other tests as well.

The symptom is: the test framework never terminates, it also reports an 
incorrect (?) hung test.

The problem is that the actual forked JVM is hung on reading stdin, waiting for 
the next test suite (no test thread is present); the master process is hung on 
receiving data from the forked jvm (both the events file and stdout spill is 
truncated in the middle of a test). The last output is:

{code}
[
  APPEND_STDERR,
  {
chunk: 612639 T30203 oasu.DefaultSolrCoreState.doRecovery Running 
recovery - first canceling any ongoing recovery%0A
  }
]

[
  APPEND_STDERR
{code}

Overall, it looks insane -- there are flushes after each test completes 
(normally or not), there are tests *following* the one that last reported 
output and before dynamic suites on stdin. 

I have no idea. The best explanation is insane -- looks like the test thread 
just died in the middle of executing Java code...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5786) Unflushed/ truncated events file (hung testing subprocess)

2014-06-23 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14040655#comment-14040655
 ] 

Dawid Weiss commented on LUCENE-5786:
-

I think this may actually be a bug in RR code. Here's why -- writing is done 
via RandomAccessFile and it used to have flush() method but at some point it 
was commented out:
{code}
   @Override
   public void flush() throws IOException {
-raf.getChannel().force(true);
+// This was causing intermittent channel invalidations on Windows for
+// no apparent reason. Also, it shouldn't be a problem if we don't sync
+// with the disk (and use OS cache only)?
+// raf.getChannel().force(true);
   }
{code}

So it either randomly fails on Windows or it will not sync properly on FreeBSD. 
Nice. I'll revert to simple FileOutputStream and see if this improves things.

 Unflushed/ truncated events file (hung testing subprocess)
 --

 Key: LUCENE-5786
 URL: https://issues.apache.org/jira/browse/LUCENE-5786
 Project: Lucene - Core
  Issue Type: Bug
  Components: general/test
Reporter: Dawid Weiss
Assignee: Dawid Weiss

 This has happened several times on Jenkins, typically on 
 SSLMigrationTest.testDistribSearch, but probably on other tests as well.
 The symptom is: the test framework never terminates, it also reports an 
 incorrect (?) hung test.
 The problem is that the actual forked JVM is hung on reading stdin, waiting 
 for the next test suite (no test thread is present); the master process is 
 hung on receiving data from the forked jvm (both the events file and stdout 
 spill is truncated in the middle of a test). The last output is:
 {code}
 [
   APPEND_STDERR,
   {
 chunk: 612639 T30203 oasu.DefaultSolrCoreState.doRecovery Running 
 recovery - first canceling any ongoing recovery%0A
   }
 ]
 [
   APPEND_STDERR
 {code}
 Overall, it looks insane -- there are flushes after each test completes 
 (normally or not), there are tests *following* the one that last reported 
 output and before dynamic suites on stdin. 
 I have no idea. The best explanation is insane -- looks like the test thread 
 just died in the middle of executing Java code...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-6076) Using Solr with Windchill 10.0 and there is a need to index one single document

2014-06-23 Thread Priya (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Priya closed SOLR-6076.
---


 Using  Solr with Windchill 10.0 and there is a need to index one single 
 document
 

 Key: SOLR-6076
 URL: https://issues.apache.org/jira/browse/SOLR-6076
 Project: Solr
  Issue Type: Wish
Reporter: Priya
Priority: Minor
  Labels: performance

 Is there any functionality to index a single document



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6184) Replication fetchLatestIndex always failed, that will occur the recovery error.

2014-06-23 Thread Raintung Li (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-6184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14040673#comment-14040673
]

Raintung Li commented on SOLR-6184:
---

How to estimate the duration? You will keep the update in the memory, and need
think to avoid the OOM case.

Replication fetchLatestIndex always failed, that will occur the recovery
error.
---

Key: SOLR-6184
URL: https://issues.apache.org/jira/browse/SOLR-6184
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.6, 4.6.1
Environment: the index file size is more than 70G
Reporter: Raintung Li
Attachments: Solr-6184.txt

Usually the copy full index 70G need 20 minutes at least, 100M read/write
network or disk r/w. If in the 20 minutes happen one hard commit, that means
the copy full index snap pull will be failed, the temp folder will be removed
because it is failed pull task.
In the production, update index will happen in every minute, redo pull task
always failed because index always change.
And also always redo the pull it will occur the network and disk usage keep
the high level.
For my suggestion, the fetchLatestIndex can be do again in some frequency.
Don't need remove the tmp folder, and copy the largest index at first. Redo
the fetchLatestIndex don't download the same biggest file again, only will
copy the commit index just now, at last the task will be easy success.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6095) SolrCloud cluster can end up without an overseer with overseer roles


[ 
https://issues.apache.org/jira/browse/SOLR-6095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14040728#comment-14040728
 ] 

ASF subversion and git services commented on SOLR-6095:
---

Commit 1604791 from [~noble.paul] in branch 'dev/trunk'
[ https://svn.apache.org/r1604791 ]

SOLR-6095 wait for http responses

 SolrCloud cluster can end up without an overseer with overseer roles 
 -

 Key: SOLR-6095
 URL: https://issues.apache.org/jira/browse/SOLR-6095
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.8
Reporter: Shalin Shekhar Mangar
Assignee: Noble Paul
 Fix For: 5.0, 4.10

 Attachments: SOLR-6095.patch, SOLR-6095.patch, SOLR-6095.patch, 
 SOLR-6095.patch


 We have a large cluster running on ec2 which occasionally ends up without an 
 overseer after a rolling restart. We always restart our overseer nodes at the 
 very last otherwise we end up with a large number of shards that can't 
 recover properly.
 This cluster is running a custom branch forked from 4.8 and has SOLR-5473, 
 SOLR-5495 and SOLR-5468 applied. We have a large number of small collections 
 (120 collections each with approx 5M docs) on 16 Solr nodes. We are also 
 using the overseer roles feature to designate two specified nodes as 
 overseers. However, I think the problem that we're seeing is not specific to 
 the overseer roles feature.
 As soon as the overseer was shutdown, we saw the following on the node which 
 was next in line to become the overseer:
 {code}
 2014-05-20 09:55:39,261 [main-EventThread] INFO  solr.cloud.ElectionContext  
 - I am going to be the leader ec2-xx.compute-1.amazonaws.com:8987_solr
 2014-05-20 09:55:39,265 [main-EventThread] WARN  solr.cloud.LeaderElector  - 
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /overseer_elect/leader
   at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
   at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
   at 
 org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:432)
   at 
 org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   at 
 org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:429)
   at 
 org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:386)
   at 
 org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:373)
   at 
 org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:551)
   at 
 org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
   at 
 org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
   at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   at 
 org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
   at 
 org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
 {code}
 When the overseer leader node is gracefully shutdown, we get the following in 
 the logs:
 {code}
 2014-05-20 09:55:39,254 [Thread-63] ERROR solr.cloud.Overseer  - Exception in 
 Overseer main queue loop
 org.apache.solr.common.SolrException: Could not load collection from ZK:sm12
   at 
 org.apache.solr.common.cloud.ZkStateReader.getExternCollectionFresh(ZkStateReader.java:778)
   at 
 org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:553)
   at 
 org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:246)
   at 
 org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:237)
   at java.lang.Thread.run(Thread.java:745)
 Caused by: java.lang.InterruptedException
   at java.lang.Object.wait(Native Method)
   at java.lang.Object.wait(Object.java:503)
   at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342)
   at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1040)
   at 
 org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:226)
   at 
 org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:223)
   at 
 org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   at 
 org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:223)
   at 
 org.apache.solr.common.cloud.ZkStateReader.getExternCollectionFresh(ZkStateReader.java:767)
   ... 4 more
 2014-05-20 09:55:39,254 [Thread-63] INFO  solr.cloud.Overseer  - Overseer 
 Loop exiting :

[jira] [Commented] (SOLR-6095) SolrCloud cluster can end up without an overseer with overseer roles

2014-06-23 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14040730#comment-14040730
 ] 

ASF subversion and git services commented on SOLR-6095:
---

Commit 1604792 from [~noble.paul] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1604792 ]

SOLR-6095 wait for http responses

 SolrCloud cluster can end up without an overseer with overseer roles 
 -

 Key: SOLR-6095
 URL: https://issues.apache.org/jira/browse/SOLR-6095
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.8
Reporter: Shalin Shekhar Mangar
Assignee: Noble Paul
 Fix For: 5.0, 4.10

 Attachments: SOLR-6095.patch, SOLR-6095.patch, SOLR-6095.patch, 
 SOLR-6095.patch


 We have a large cluster running on ec2 which occasionally ends up without an 
 overseer after a rolling restart. We always restart our overseer nodes at the 
 very last otherwise we end up with a large number of shards that can't 
 recover properly.
 This cluster is running a custom branch forked from 4.8 and has SOLR-5473, 
 SOLR-5495 and SOLR-5468 applied. We have a large number of small collections 
 (120 collections each with approx 5M docs) on 16 Solr nodes. We are also 
 using the overseer roles feature to designate two specified nodes as 
 overseers. However, I think the problem that we're seeing is not specific to 
 the overseer roles feature.
 As soon as the overseer was shutdown, we saw the following on the node which 
 was next in line to become the overseer:
 {code}
 2014-05-20 09:55:39,261 [main-EventThread] INFO  solr.cloud.ElectionContext  
 - I am going to be the leader ec2-xx.compute-1.amazonaws.com:8987_solr
 2014-05-20 09:55:39,265 [main-EventThread] WARN  solr.cloud.LeaderElector  - 
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /overseer_elect/leader
   at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
   at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
   at 
 org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:432)
   at 
 org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   at 
 org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:429)
   at 
 org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:386)
   at 
 org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:373)
   at 
 org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:551)
   at 
 org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
   at 
 org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
   at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   at 
 org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
   at 
 org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
 {code}
 When the overseer leader node is gracefully shutdown, we get the following in 
 the logs:
 {code}
 2014-05-20 09:55:39,254 [Thread-63] ERROR solr.cloud.Overseer  - Exception in 
 Overseer main queue loop
 org.apache.solr.common.SolrException: Could not load collection from ZK:sm12
   at 
 org.apache.solr.common.cloud.ZkStateReader.getExternCollectionFresh(ZkStateReader.java:778)
   at 
 org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:553)
   at 
 org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:246)
   at 
 org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:237)
   at java.lang.Thread.run(Thread.java:745)
 Caused by: java.lang.InterruptedException
   at java.lang.Object.wait(Native Method)
   at java.lang.Object.wait(Object.java:503)
   at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342)
   at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1040)
   at 
 org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:226)
   at 
 org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:223)
   at 
 org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   at 
 org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:223)
   at 
 org.apache.solr.common.cloud.ZkStateReader.getExternCollectionFresh(ZkStateReader.java:767)
   ... 4 more
 2014-05-20 09:55:39,254 [Thread-63] INFO  solr.cloud.Overseer  - Overseer 
 Loop exiting

[jira] [Resolved] (SOLR-6091) Race condition in prioritizeOverseerNodes can trigger extra QUIT operations

2014-06-23 Thread Noble Paul (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul resolved SOLR-6091.
--

Resolution: Duplicate

 Race condition in prioritizeOverseerNodes can trigger extra QUIT operations
 ---

 Key: SOLR-6091
 URL: https://issues.apache.org/jira/browse/SOLR-6091
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.7, 4.8
Reporter: Shalin Shekhar Mangar
Assignee: Noble Paul
 Fix For: 4.9, 5.0

 Attachments: SOLR-6091.patch


 When using the overseer roles feature, there is a possibility of more than 
 one thread executing the prioritizeOverseerNodes method and extra QUIT 
 commands being inserted into the overseer queue.
 At a minimum, the prioritizeOverseerNodes should be synchronized to avoid a 
 race condition.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6091) Race condition in prioritizeOverseerNodes can trigger extra QUIT operations

2014-06-23 Thread Noble Paul (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-6091:
-

Fix Version/s: (was: 4.9)
   4.10

 Race condition in prioritizeOverseerNodes can trigger extra QUIT operations
 ---

 Key: SOLR-6091
 URL: https://issues.apache.org/jira/browse/SOLR-6091
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.7, 4.8
Reporter: Shalin Shekhar Mangar
Assignee: Noble Paul
 Fix For: 5.0, 4.10

 Attachments: SOLR-6091.patch


 When using the overseer roles feature, there is a possibility of more than 
 one thread executing the prioritizeOverseerNodes method and extra QUIT 
 commands being inserted into the overseer queue.
 At a minimum, the prioritizeOverseerNodes should be synchronized to avoid a 
 race condition.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0_05) - Build # 10519 - Failure!

2014-06-23 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/10519/
Java: 64bit/jdk1.8.0_05 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
REGRESSION:  org.apache.solr.response.TestChildDocTransformer.testParentFilter

Error Message:
mismatch: '3'!='6' @ response/docs/[0]/_childDocuments_/[0]/id

Stack Trace:
java.lang.RuntimeException: mismatch: '3'!='6' @ 
response/docs/[0]/_childDocuments_/[0]/id
at 
__randomizedtesting.SeedInfo.seed([E5AF40AE6700D67E:B1F548E5AB8C1A12]:0)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:792)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:739)
at 
org.apache.solr.response.TestChildDocTransformer.testParentFilterJSON(TestChildDocTransformer.java:205)
at 
org.apache.solr.response.TestChildDocTransformer.testParentFilter(TestChildDocTransformer.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at

[jira] [Created] (SOLR-6189) If the node hosting a replica is not live, then leader-initiated recovery may publish the down state more than once.

2014-06-23 Thread Timothy Potter (JIRA)

Timothy Potter created SOLR-6189:


 Summary: If the node hosting a replica is not live, then 
leader-initiated recovery may publish the down state more than once.
 Key: SOLR-6189
 URL: https://issues.apache.org/jira/browse/SOLR-6189
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Reporter: Timothy Potter
Assignee: Timothy Potter
Priority: Minor


In ZkController#ensureReplicaInLeaderInitiatedRecovery, the leader will publish 
the down state for a replica more than once if the node is not live. This is 
inefficient and can cause too many state change events to occur, one is 
sufficient.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Reopened] (SOLR-6157) ReplicationFactorTest hangs

2014-06-23 Thread Timothy Potter (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Timothy Potter reopened SOLR-6157:
--


This occurred again:

http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10613/consoleFull

 ReplicationFactorTest hangs
 ---

 Key: SOLR-6157
 URL: https://issues.apache.org/jira/browse/SOLR-6157
 Project: Solr
  Issue Type: Bug
  Components: replication (java)
Reporter: Uwe Schindler
Assignee: Timothy Potter
 Fix For: 4.10


 See: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10517/
 You can download all logs from there.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5480) Make MoreLikeThisHandler distributable

2014-06-23 Thread ASF subversion and git services (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-5480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vitaliy Zhovtyuk updated SOLR-5480:
---

Attachment: SOLR-5480.patch

Updated patch sources to latest trunk. There are two approaches to get
distributed MLT:
1. Added mlt qparser, works in single and cloud mode. Added support for numeric
id.
Result of mlt written as query result - not in MoreLikeThis. Added tests to
call in single and cloud modes.

2. SingleMLT component, mlt per shards distribution, added
org.apache.solr.handler.DistributedMoreLikeThisHandlerTest

Make MoreLikeThisHandler distributable
--

Key: SOLR-5480
URL: https://issues.apache.org/jira/browse/SOLR-5480
Project: Solr
Issue Type: Improvement
Reporter: Steve Molloy
Assignee: Noble Paul
Attachments: SOLR-5480.patch, SOLR-5480.patch, SOLR-5480.patch,
SOLR-5480.patch, SOLR-5480.patch, SOLR-5480.patch, SOLR-5480.patch

The MoreLikeThis component, when used in the standard search handler supports
distributed searches. But the MoreLikeThisHandler itself doesn't, which
prevents from say, passing in text to perform the query. I'll start looking
into adapting the SearchHandler logic to the MoreLikeThisHandler. If anyone
has some work done already and want to share, or want to contribute, any help
will be welcomed.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6157) ReplicationFactorTest hangs

2014-06-23 Thread Shalin Shekhar Mangar (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14040925#comment-14040925
 ] 

Shalin Shekhar Mangar commented on SOLR-6157:
-

This might be related to LUCENE-5786

 ReplicationFactorTest hangs
 ---

 Key: SOLR-6157
 URL: https://issues.apache.org/jira/browse/SOLR-6157
 Project: Solr
  Issue Type: Bug
  Components: replication (java)
Reporter: Uwe Schindler
Assignee: Timothy Potter
 Fix For: 4.10


 See: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10517/
 You can download all logs from there.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6157) ReplicationFactorTest hangs


[ 
https://issues.apache.org/jira/browse/SOLR-6157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14040926#comment-14040926
 ] 

ASF subversion and git services commented on SOLR-6157:
---

Commit 1604868 from [~thelabdude] in branch 'dev/trunk'
[ https://svn.apache.org/r1604868 ]

SOLR-6157: Hang occurred again, somewhere in tearDown ... changing the code to 
close the socket proxies after super.tearDown, if that doesn't work, I'll add 
the AwaitsFix back to the code to ignore this test.

 ReplicationFactorTest hangs
 ---

 Key: SOLR-6157
 URL: https://issues.apache.org/jira/browse/SOLR-6157
 Project: Solr
  Issue Type: Bug
  Components: replication (java)
Reporter: Uwe Schindler
Assignee: Timothy Potter
 Fix For: 4.10


 See: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10517/
 You can download all logs from there.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6190) Self Describing SearchComponents, RequestHandlers, params. etc.

Vitaliy Zhovtyuk created SOLR-6190:
--

 Summary: Self Describing SearchComponents, RequestHandlers, 
params. etc.
 Key: SOLR-6190
 URL: https://issues.apache.org/jira/browse/SOLR-6190
 Project: Solr
  Issue Type: Bug
Reporter: Vitaliy Zhovtyuk


We should have self describing parameters for search components, etc.
I think we should support UNIX style short and long names and that you should 
also be able to get a short description of what a parameter does if you ask for 
INFO on it.

For instance, fl could also be fieldList, etc.
Also, we should put this into the base classes so that new components can add 
to it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6191) Self Describing SearchComponents, RequestHandlers, params. etc.

Vitaliy Zhovtyuk created SOLR-6191:
--

 Summary: Self Describing SearchComponents, RequestHandlers, 
params. etc.
 Key: SOLR-6191
 URL: https://issues.apache.org/jira/browse/SOLR-6191
 Project: Solr
  Issue Type: Bug
Reporter: Vitaliy Zhovtyuk


We should have self describing parameters for search components, etc.
I think we should support UNIX style short and long names and that you should 
also be able to get a short description of what a parameter does if you ask for 
INFO on it.

For instance, fl could also be fieldList, etc.
Also, we should put this into the base classes so that new components can add 
to it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0_05) - Build # 10519 - Failure!

2014-06-23 Thread Chris Hostetter


I'm looking into this now...

: Date: Mon, 23 Jun 2014 13:32:54 + (UTC)
: From: Policeman Jenkins Server jenk...@thetaphi.de
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: [JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0_05) - Build # 10519 -
:  Failure!
: 
: Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/10519/
: Java: 64bit/jdk1.8.0_05 -XX:+UseCompressedOops -XX:+UseParallelGC
: 
: 1 tests failed.
: REGRESSION:  org.apache.solr.response.TestChildDocTransformer.testParentFilter
: 
: Error Message:
: mismatch: '3'!='6' @ response/docs/[0]/_childDocuments_/[0]/id
: 
: Stack Trace:
: java.lang.RuntimeException: mismatch: '3'!='6' @ 
response/docs/[0]/_childDocuments_/[0]/id
:   at 
__randomizedtesting.SeedInfo.seed([E5AF40AE6700D67E:B1F548E5AB8C1A12]:0)
:   at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:792)
:   at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:739)
:   at 
org.apache.solr.response.TestChildDocTransformer.testParentFilterJSON(TestChildDocTransformer.java:205)
:   at 
org.apache.solr.response.TestChildDocTransformer.testParentFilter(TestChildDocTransformer.java:49)
:   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
:   at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
:   at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
:   at java.lang.reflect.Method.invoke(Method.java:483)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
:   at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
:   at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
:   at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
:   at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
:   at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
:   at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
:   at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
:   at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
:   at

[jira] [Closed] (SOLR-6190) Self Describing SearchComponents, RequestHandlers, params. etc.


 [ 
https://issues.apache.org/jira/browse/SOLR-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitaliy Zhovtyuk closed SOLR-6190.
--

Resolution: Invalid

 Self Describing SearchComponents, RequestHandlers, params. etc.
 ---

 Key: SOLR-6190
 URL: https://issues.apache.org/jira/browse/SOLR-6190
 Project: Solr
  Issue Type: Bug
Reporter: Vitaliy Zhovtyuk

 We should have self describing parameters for search components, etc.
 I think we should support UNIX style short and long names and that you should 
 also be able to get a short description of what a parameter does if you ask 
 for INFO on it.
 For instance, fl could also be fieldList, etc.
 Also, we should put this into the base classes so that new components can add 
 to it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6191) Self Describing SearchComponents, RequestHandlers, params. etc.


 [ 
https://issues.apache.org/jira/browse/SOLR-6191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitaliy Zhovtyuk updated SOLR-6191:
---

Attachment: SOLR-6191.patch

All methods in SolrParams are migrated to enum approach, old methods with 
string field name marked as deprecated.
Migrated MLT params to enum containing params description.
Component/Handler that has self described parameters should implement
SelfDescribableParametersT where T is enum type describing each parameter. 
This enum type should implement ParameterDescription interface in order to 
provide same contact for all parameter description enums.

 Self Describing SearchComponents, RequestHandlers, params. etc.
 ---

 Key: SOLR-6191
 URL: https://issues.apache.org/jira/browse/SOLR-6191
 Project: Solr
  Issue Type: Bug
Reporter: Vitaliy Zhovtyuk
  Labels: features
 Attachments: SOLR-6191.patch


 We should have self describing parameters for search components, etc.
 I think we should support UNIX style short and long names and that you should 
 also be able to get a short description of what a parameter does if you ask 
 for INFO on it.
 For instance, fl could also be fieldList, etc.
 Also, we should put this into the base classes so that new components can add 
 to it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Solr Ref Guide edits for 4.9

2014-06-23 Thread Cassandra Targett

The vote for Lucene/Solr 4.9 is underway and we need to get the Solr Ref
Guide updated for the new release.

I went through the CHANGES.txt for Solr on Friday and updated the TODO list
found at
https://cwiki.apache.org/confluence/display/solr/Internal+-+TODO+List. I
notice that many of the new features or changes in 4.9 haven't been
documented in the Ref Guide yet. We need some volunteers to work on
updating the guide before we can make a Ref Guide release candidate for 4.9.

If you look at the TODO list, you'll see lots of stuff from earlier
releases that haven't made it into the guide yet either. If we want users
to be able to use the work we've done for each release, and if we want the
Ref Guide to be a definitive source of Solr information, we have to add
content on new features and keep content up to date as things change.

Please take a few moments and see if there is anything you can add or
change. If you need help with wording or don't know where something should
go, speak up and I'll try to help.

Cassandra

[jira] [Created] (SOLR-6192) Adding docs from the Admin UI may not send the doc to the right shard in Cloud mode

2014-06-23 Thread Erick Erickson (JIRA)

Erick Erickson created SOLR-6192:


 Summary: Adding docs from the Admin UI may not send the doc to the 
right shard in Cloud mode
 Key: SOLR-6192
 URL: https://issues.apache.org/jira/browse/SOLR-6192
 Project: Solr
  Issue Type: Bug
  Components: web gui
Affects Versions: 4.9
Reporter: Erick Erickson
Priority: Minor


Speculating from an exchange on the user's list. User reported that adding a 
document via the admin UI resulted in the document appearing twice in the 
results lists.

Insuring that documents added via the admin UI are cloud-aware seems like a 
Good Thing. That said, I'm not entirely sure it's worth the effort. If we're 
going to make the admin screens cloud-aware in general, there are a number of 
changes necessary.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6193) Distributed search with multiselect faceting ignores the facet.offset local parameter

John Gibson created SOLR-6193:
-

 Summary: Distributed search with multiselect faceting ignores the 
facet.offset local parameter
 Key: SOLR-6193
 URL: https://issues.apache.org/jira/browse/SOLR-6193
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.8.1
 Environment: OS X 10.9.3 Apache Tomcat 7.0.41
Debian Apache Tomcat 7
Reporter: John Gibson


When a distributed search contains multiselect faceting the per-field faceting 
options are not honored for alternate selections of the field. For example with 
a query like:
{noformat}
facet.field=blahfacet.field={!key myblah 
facet.offset=10}blahf.blah.facet.offset=20
{noformat}
The returned facet results for both blah and myblah will use an offset of 20 as 
opposed to a standard search returning myblah with an offset of 10.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6193) Distributed search with multiselect faceting ignores the facet.offset local parameter


 [ 
https://issues.apache.org/jira/browse/SOLR-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Gibson updated SOLR-6193:
--

Attachment: bad_facet_offset_test_4_8_x.patch

Here's a patch for an existing unit test that illustrates the issue.

Also note that while working on this I noticed that the facet.limit local 
parameter is ignored for both the regular and distributed versions of the 
search. Is that intentional? Or a bug?

 Distributed search with multiselect faceting ignores the facet.offset local 
 parameter
 -

 Key: SOLR-6193
 URL: https://issues.apache.org/jira/browse/SOLR-6193
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.8.1
 Environment: OS X 10.9.3 Apache Tomcat 7.0.41
 Debian Apache Tomcat 7
Reporter: John Gibson
 Attachments: bad_facet_offset_test_4_8_x.patch


 When a distributed search contains multiselect faceting the per-field 
 faceting options are not honored for alternate selections of the field. For 
 example with a query like:
 {noformat}
 facet.field=blahfacet.field={!key myblah 
 facet.offset=10}blahf.blah.facet.offset=20
 {noformat}
 The returned facet results for both blah and myblah will use an offset of 20 
 as opposed to a standard search returning myblah with an offset of 10.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr Ref Guide edits for 4.9

2014-06-23 Thread Timothy Potter

Hi Cassandra,

Thanks for putting together this list. I don't think the solution for
SOLR-5495 needs any mention in the Ref Guide as it's internal
hardening of replication / recovery scenarios. However, SOLR-5468
should be mentioned as it is a new parameter that can be passed in
update requests. I'm not sure of a good place to put it though?
Basically, it is an optional feature that can be activated by passing
the min_rf=# in an UpdateRequest to request Solr to return the
achieved replication factor for the request. For instance, if a client
passed min_rf=2 and a shard has 3 replicas, but two of the replicas
are down, then the server will return rf=1. This indicates to the
client application that the collection health is degraded and gives
the client an option of taking some additional action, such as saving
a list of documents that were added during the degraded state. Thus,
this feature is related to replication in SolrCloud.

Also, I'm happy to help out with any of the ManagedResource
refactoring work if we take that on in this release.

Tim

On Mon, Jun 23, 2014 at 10:23 AM, Cassandra Targett
casstarg...@gmail.com wrote:
 The vote for Lucene/Solr 4.9 is underway and we need to get the Solr Ref
 Guide updated for the new release.

 I went through the CHANGES.txt for Solr on Friday and updated the TODO list
 found at
 https://cwiki.apache.org/confluence/display/solr/Internal+-+TODO+List. I
 notice that many of the new features or changes in 4.9 haven't been
 documented in the Ref Guide yet. We need some volunteers to work on updating
 the guide before we can make a Ref Guide release candidate for 4.9.

 If you look at the TODO list, you'll see lots of stuff from earlier releases
 that haven't made it into the guide yet either. If we want users to be able
 to use the work we've done for each release, and if we want the Ref Guide to
 be a definitive source of Solr information, we have to add content on new
 features and keep content up to date as things change.

 Please take a few moments and see if there is anything you can add or
 change. If you need help with wording or don't know where something should
 go, speak up and I'll try to help.

 Cassandra



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [VOTE] 4.9.0

2014-06-23 Thread Martijn v Groningen

+1 all test pass. Lets close the vote.
Op 23 jun. 2014 09:19 schreef Adrien Grand jpou...@gmail.com:

 +1

 SUCCESS! [0:43:52.973702]

 On Fri, Jun 20, 2014 at 2:13 PM, Robert Muir rcm...@gmail.com wrote:
  Artifacts here:
  http://people.apache.org/~rmuir/staging_area/lucene_solr_4_9_0_r1604085/
 
  Here's my +1
 
  SUCCESS! [0:35:36.654925]
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
  For additional commands, e-mail: dev-h...@lucene.apache.org
 



 --
 Adrien

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests

John Gibson created LUCENE-5787:
---

 Summary: LuceneTestCase static leak checker interferes with Groovy 
unit tests
 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
 Environment: Maven 3.0.5
JUnit 4.11
Reporter: John Gibson


{{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
Specifically, Groovy classes have a synthetic static member variable of type 
{{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too large 
then LTC will fail the test. Because the variable is added by the Groovy 
runtime instead of by the developer there is no way for the developer to clear 
the field themselves.

Also note that the static leak checker does not ignore memory held by soft or 
weak references. These should be ignored because the memory retained by such 
fields will be reclaimed instead of triggering OutOfMemoryErrors.

Note that because LTC is a base class for Solr's testing support classes this 
also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-6193) Distributed search with multiselect faceting ignores the facet.offset local parameter


[ 
https://issues.apache.org/jira/browse/SOLR-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041050#comment-14041050
 ] 

John Gibson edited comment on SOLR-6193 at 6/23/14 6:23 PM:


Here's a patch for an existing unit test on the lucene_solr_4_8 branch that 
illustrates the issue.

Also note that while working on this I noticed that the facet.limit local 
parameter is ignored for both the regular and distributed versions of the 
search. Is that intentional? Or a bug?


was (Author: jgibson):
Here's a patch for an existing unit test that illustrates the issue.

Also note that while working on this I noticed that the facet.limit local 
parameter is ignored for both the regular and distributed versions of the 
search. Is that intentional? Or a bug?

 Distributed search with multiselect faceting ignores the facet.offset local 
 parameter
 -

 Key: SOLR-6193
 URL: https://issues.apache.org/jira/browse/SOLR-6193
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.8.1
 Environment: OS X 10.9.3 Apache Tomcat 7.0.41
 Debian Apache Tomcat 7
Reporter: John Gibson
 Attachments: bad_facet_offset_test_4_8_x.patch


 When a distributed search contains multiselect faceting the per-field 
 faceting options are not honored for alternate selections of the field. For 
 example with a query like:
 {noformat}
 facet.field=blahfacet.field={!key myblah 
 facet.offset=10}blahf.blah.facet.offset=20
 {noformat}
 The returned facet results for both blah and myblah will use an offset of 20 
 as opposed to a standard search returning myblah with an offset of 10.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0_05) - Build # 10519 - Failure!

2014-06-23 Thread Chris Hostetter


text bug ... fixed in r1604898  r1604901.

: Date: Mon, 23 Jun 2014 09:44:18 -0700 (MST)
: From: Chris Hostetter hossman_luc...@fucit.org
: To: dev@lucene.apache.org
: Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0_05) - Build #
: 10519 - Failure!
: 
: 
: I'm looking into this now...
: 
: : Date: Mon, 23 Jun 2014 13:32:54 + (UTC)
: : From: Policeman Jenkins Server jenk...@thetaphi.de
: : Reply-To: dev@lucene.apache.org
: : To: dev@lucene.apache.org
: : Subject: [JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0_05) - Build # 
10519 -
: :  Failure!
: : 
: : Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/10519/
: : Java: 64bit/jdk1.8.0_05 -XX:+UseCompressedOops -XX:+UseParallelGC
: : 
: : 1 tests failed.
: : REGRESSION:  
org.apache.solr.response.TestChildDocTransformer.testParentFilter
: : 
: : Error Message:
: : mismatch: '3'!='6' @ response/docs/[0]/_childDocuments_/[0]/id
: : 
: : Stack Trace:
: : java.lang.RuntimeException: mismatch: '3'!='6' @ 
response/docs/[0]/_childDocuments_/[0]/id
: : at 
__randomizedtesting.SeedInfo.seed([E5AF40AE6700D67E:B1F548E5AB8C1A12]:0)
: : at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:792)
: : at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:739)
: : at 
org.apache.solr.response.TestChildDocTransformer.testParentFilterJSON(TestChildDocTransformer.java:205)
: : at 
org.apache.solr.response.TestChildDocTransformer.testParentFilter(TestChildDocTransformer.java:49)
: : at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: : at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
: : at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: : at java.lang.reflect.Method.invoke(Method.java:483)
: : at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
: : at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
: : at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
: : at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
: : at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
: : at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
: : at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
: : at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
: : at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
: : at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
: : at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
: : at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
: : at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: : at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
: : at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
: : at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
: : at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
: : at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
: : at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
: : at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
: : at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: : at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
: : at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
: : at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
: : at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
: : at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
: : at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
: : at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: : at

[jira] [Updated] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests


 [ 
https://issues.apache.org/jira/browse/LUCENE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Gibson updated LUCENE-5787:


Affects Version/s: 4.7
   4.8.1

 LuceneTestCase static leak checker interferes with Groovy unit tests
 

 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.7, 4.8.1
 Environment: Maven 3.0.5
 JUnit 4.11
Reporter: John Gibson

 {{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
 Specifically, Groovy classes have a synthetic static member variable of type 
 {{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too 
 large then LTC will fail the test. Because the variable is added by the 
 Groovy runtime instead of by the developer there is no way for the developer 
 to clear the field themselves.
 Also note that the static leak checker does not ignore memory held by soft or 
 weak references. These should be ignored because the memory retained by such 
 fields will be reclaimed instead of triggering OutOfMemoryErrors.
 Note that because LTC is a base class for Solr's testing support classes this 
 also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests


[ 
https://issues.apache.org/jira/browse/LUCENE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041145#comment-14041145
 ] 

Uwe Schindler commented on LUCENE-5787:
---

This means you want to subclass LuceneTestCase with Groovy as basis for your 
own Solr/Lucene tests?

 LuceneTestCase static leak checker interferes with Groovy unit tests
 

 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.7, 4.8.1
 Environment: Maven 3.0.5
 JUnit 4.11
Reporter: John Gibson

 {{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
 Specifically, Groovy classes have a synthetic static member variable of type 
 {{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too 
 large then LTC will fail the test. Because the variable is added by the 
 Groovy runtime instead of by the developer there is no way for the developer 
 to clear the field themselves.
 Also note that the static leak checker does not ignore memory held by soft or 
 weak references. These should be ignored because the memory retained by such 
 fields will be reclaimed instead of triggering OutOfMemoryErrors.
 Note that because LTC is a base class for Solr's testing support classes this 
 also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests


[ 
https://issues.apache.org/jira/browse/LUCENE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041150#comment-14041150
 ] 

Dawid Weiss commented on LUCENE-5787:
-

We could exclude this particular reference by field class's name. As for weak/ 
soft refs -- I agree, these should also be excluded from the count. What do you 
think, Uwe?

 LuceneTestCase static leak checker interferes with Groovy unit tests
 

 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.7, 4.8.1
 Environment: Maven 3.0.5
 JUnit 4.11
Reporter: John Gibson

 {{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
 Specifically, Groovy classes have a synthetic static member variable of type 
 {{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too 
 large then LTC will fail the test. Because the variable is added by the 
 Groovy runtime instead of by the developer there is no way for the developer 
 to clear the field themselves.
 Also note that the static leak checker does not ignore memory held by soft or 
 weak references. These should be ignored because the memory retained by such 
 fields will be reclaimed instead of triggering OutOfMemoryErrors.
 Note that because LTC is a base class for Solr's testing support classes this 
 also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests


[ 
https://issues.apache.org/jira/browse/LUCENE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041167#comment-14041167
 ] 

Uwe Schindler commented on LUCENE-5787:
---

I agree with all subclasses of Reference. The main usecase of WeakRefs is in 
static fields :-) - see for example the pointers to classes for static caches.

The particular reference to the groovy class could be added a String to the 
leak checker. On the other hand, if you subclass LTC for Groovy, you can add 
the annotation to prevent the leakchecker from running. This bug does not 
affect Lucene, it is a problem of 3rd party infrastructures using our test 
framework. So the big question: What does other scripting laguages do? Scala? 
JRuby? The list is endless. Maybe add an annotation like 
{{@SuppressLeakChecks([fieldname, fieldname, fieldname])}}.

 LuceneTestCase static leak checker interferes with Groovy unit tests
 

 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.7, 4.8.1
 Environment: Maven 3.0.5
 JUnit 4.11
Reporter: John Gibson

 {{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
 Specifically, Groovy classes have a synthetic static member variable of type 
 {{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too 
 large then LTC will fail the test. Because the variable is added by the 
 Groovy runtime instead of by the developer there is no way for the developer 
 to clear the field themselves.
 Also note that the static leak checker does not ignore memory held by soft or 
 weak references. These should be ignored because the memory retained by such 
 fields will be reclaimed instead of triggering OutOfMemoryErrors.
 Note that because LTC is a base class for Solr's testing support classes this 
 also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6194) Allow access to DataImporter and DIHConfiguration

2014-06-23 Thread Aaron LaBella (JIRA)

Aaron LaBella created SOLR-6194:
---

 Summary: Allow access to DataImporter and DIHConfiguration
 Key: SOLR-6194
 URL: https://issues.apache.org/jira/browse/SOLR-6194
 Project: Solr
  Issue Type: Improvement
  Components: contrib - DataImportHandler
Affects Versions: 4.10
Reporter: Aaron LaBella
 Fix For: 4.10


I'd like to change the visibility and access to a couple of the internal 
classes of DataImportHandler, specifically DataImporter and DIHConfiguration.  
My reasoning is that I've added the ability for a new data import handler 
command called *getquery* that will return the exact queries (fully resolved) 
that are executed for an entity within the data import configuration.  This 
makes it much easier to debug the dih, rather than turning on debug/verbose 
flags and digging through the raw response.  Additionally, it gives me a 
service that I can then go take the queries from and run them.

Here's a snippet of Java code that I can now execute now that I have access to 
the DIHConfiguration:

  /**
   * @return a map of all the queries for each entity in the given config
   */
  protected MapString,String getEntityQueries(DIHConfiguration config, 
MapString,Object params)
  {
MapString,String queries = new LinkedHashMap();
if (config != null  config.getEntities() != null)
{
  //make a new variable resolve
  VariableResolver vr = new VariableResolver();
  vr.addNamespace(dataimporter.request,params);

  //for each entity
  for (Entity e : config.getEntities())
  {
//get the query and resolve it
if (e.getAllAttributes().containsKey(SqlEntityProcessor.QUERY))
{
  String query = e.getAllAttributes().get(SqlEntityProcessor.QUERY);
  query = query.replaceAll(\\s+,  ).trim();
  String resolved = vr.replaceTokens(query);
  resolved = resolved.replaceAll(\\s+,  ).trim();
  queries.put(e.getName(),resolved);
  queries.put(e.getName()+_raw,query);
}
  }
}
return queries;
  }

I'm attaching a patch that I would appreciate someone have a look for 
consideration.  It's fully tested -- please let me know if there is something 
else I need to do/test.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6194) Allow access to DataImporter and DIHConfiguration

2014-06-23 Thread Aaron LaBella (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron LaBella updated SOLR-6194:


Attachment: SOLR-6194.patch

 Allow access to DataImporter and DIHConfiguration
 -

 Key: SOLR-6194
 URL: https://issues.apache.org/jira/browse/SOLR-6194
 Project: Solr
  Issue Type: Improvement
  Components: contrib - DataImportHandler
Affects Versions: 4.10
Reporter: Aaron LaBella
 Fix For: 4.10

 Attachments: SOLR-6194.patch

   Original Estimate: 2h
  Remaining Estimate: 2h

 I'd like to change the visibility and access to a couple of the internal 
 classes of DataImportHandler, specifically DataImporter and DIHConfiguration. 
  My reasoning is that I've added the ability for a new data import handler 
 command called *getquery* that will return the exact queries (fully 
 resolved) that are executed for an entity within the data import 
 configuration.  This makes it much easier to debug the dih, rather than 
 turning on debug/verbose flags and digging through the raw response.  
 Additionally, it gives me a service that I can then go take the queries 
 from and run them.
 Here's a snippet of Java code that I can now execute now that I have access 
 to the DIHConfiguration:
 {code:title=Snippet.java|borderStyle=solid}
   /**
* @return a map of all the queries for each entity in the given config
*/
   protected MapString,String getEntityQueries(DIHConfiguration config, 
 MapString,Object params)
   {
 MapString,String queries = new LinkedHashMap();
 if (config != null  config.getEntities() != null)
 {
   //make a new variable resolve
   VariableResolver vr = new VariableResolver();
   vr.addNamespace(dataimporter.request,params);
   //for each entity
   for (Entity e : config.getEntities())
   {
 //get the query and resolve it
 if (e.getAllAttributes().containsKey(SqlEntityProcessor.QUERY))
 {
   String query = e.getAllAttributes().get(SqlEntityProcessor.QUERY);
   query = query.replaceAll(\\s+,  ).trim();
   String resolved = vr.replaceTokens(query);
   resolved = resolved.replaceAll(\\s+,  ).trim();
   queries.put(e.getName(),resolved);
   queries.put(e.getName()+_raw,query);
 }
   }
 }
 return queries;
   }
 {code}
 I'm attaching a patch that I would appreciate someone have a look for 
 consideration.  It's fully tested -- please let me know if there is something 
 else I need to do/test.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6194) Allow access to DataImporter and DIHConfiguration

2014-06-23 Thread Aaron LaBella (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron LaBella updated SOLR-6194:


Description: 
I'd like to change the visibility and access to a couple of the internal 
classes of DataImportHandler, specifically DataImporter and DIHConfiguration.  
My reasoning is that I've added the ability for a new data import handler 
command called *getquery* that will return the exact queries (fully resolved) 
that are executed for an entity within the data import configuration.  This 
makes it much easier to debug the dih, rather than turning on debug/verbose 
flags and digging through the raw response.  Additionally, it gives me a 
service that I can then go take the queries from and run them.

Here's a snippet of Java code that I can now execute now that I have access to 
the DIHConfiguration:

{code:title=Snippet.java|borderStyle=solid}
  /**
   * @return a map of all the queries for each entity in the given config
   */
  protected MapString,String getEntityQueries(DIHConfiguration config, 
MapString,Object params)
  {
MapString,String queries = new LinkedHashMap();
if (config != null  config.getEntities() != null)
{
  //make a new variable resolve
  VariableResolver vr = new VariableResolver();
  vr.addNamespace(dataimporter.request,params);

  //for each entity
  for (Entity e : config.getEntities())
  {
//get the query and resolve it
if (e.getAllAttributes().containsKey(SqlEntityProcessor.QUERY))
{
  String query = e.getAllAttributes().get(SqlEntityProcessor.QUERY);
  query = query.replaceAll(\\s+,  ).trim();
  String resolved = vr.replaceTokens(query);
  resolved = resolved.replaceAll(\\s+,  ).trim();
  queries.put(e.getName(),resolved);
  queries.put(e.getName()+_raw,query);
}
  }
}
return queries;
  }
{code}

I'm attaching a patch that I would appreciate someone have a look for 
consideration.  It's fully tested -- please let me know if there is something 
else I need to do/test.

  was:
I'd like to change the visibility and access to a couple of the internal 
classes of DataImportHandler, specifically DataImporter and DIHConfiguration.  
My reasoning is that I've added the ability for a new data import handler 
command called *getquery* that will return the exact queries (fully resolved) 
that are executed for an entity within the data import configuration.  This 
makes it much easier to debug the dih, rather than turning on debug/verbose 
flags and digging through the raw response.  Additionally, it gives me a 
service that I can then go take the queries from and run them.

Here's a snippet of Java code that I can now execute now that I have access to 
the DIHConfiguration:

  /**
   * @return a map of all the queries for each entity in the given config
   */
  protected MapString,String getEntityQueries(DIHConfiguration config, 
MapString,Object params)
  {
MapString,String queries = new LinkedHashMap();
if (config != null  config.getEntities() != null)
{
  //make a new variable resolve
  VariableResolver vr = new VariableResolver();
  vr.addNamespace(dataimporter.request,params);

  //for each entity
  for (Entity e : config.getEntities())
  {
//get the query and resolve it
if (e.getAllAttributes().containsKey(SqlEntityProcessor.QUERY))
{
  String query = e.getAllAttributes().get(SqlEntityProcessor.QUERY);
  query = query.replaceAll(\\s+,  ).trim();
  String resolved = vr.replaceTokens(query);
  resolved = resolved.replaceAll(\\s+,  ).trim();
  queries.put(e.getName(),resolved);
  queries.put(e.getName()+_raw,query);
}
  }
}
return queries;
  }

I'm attaching a patch that I would appreciate someone have a look for 
consideration.  It's fully tested -- please let me know if there is something 
else I need to do/test.


 Allow access to DataImporter and DIHConfiguration
 -

 Key: SOLR-6194
 URL: https://issues.apache.org/jira/browse/SOLR-6194
 Project: Solr
  Issue Type: Improvement
  Components: contrib - DataImportHandler
Affects Versions: 4.10
Reporter: Aaron LaBella
 Fix For: 4.10

 Attachments: SOLR-6194.patch

   Original Estimate: 2h
  Remaining Estimate: 2h

 I'd like to change the visibility and access to a couple of the internal 
 classes of DataImportHandler, specifically DataImporter and DIHConfiguration. 
  My reasoning is that I've added the ability for a new data import handler 
 command called *getquery* that will return the exact queries (fully 
 resolved) that are executed for an entity within the data import 
 configuration.  This makes it much easier to debug the dih, rather

[jira] [Commented] (LUCENE-5785) White space tokenizer has undocumented limit of 256 characters per token

2014-06-23 Thread Jack Krupansky (JIRA)

[
https://issues.apache.org/jira/browse/LUCENE-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041194#comment-14041194
]

Jack Krupansky commented on LUCENE-5785:

The pattern tokenizer can be used as a workaround for the white space tokenizer
since it doesn't have that hard-wired token length limit.

White space tokenizer has undocumented limit of 256 characters per token

Key: LUCENE-5785
URL: https://issues.apache.org/jira/browse/LUCENE-5785
Project: Lucene - Core
Issue Type: Improvement
Components: modules/analysis
Affects Versions: 4.8.1
Reporter: Jack Krupansky
Priority: Minor

The white space tokenizer breaks tokens at 256 characters, which is a
hard-wired limit of the character tokenizer abstract class.
The limit of 256 is obviously fine for normal, natural language text, but
excessively restrictive for semi-structured data.
1. Document the current limit in the Javadoc for the character tokenizer. Add
a note to any derived tokenizers (such as the white space tokenizer) that
token size is limited as per the character tokenizer.
2. Added the setMaxTokenLength method to the character tokenizer ala the
standard tokenizer so that an application can control the limit. This should
probably be added to the character tokenizer abstract class, and then other
derived tokenizer classes can inherit it.
3. Disallow a token size limit of 0.
4. A limit of -1 would mean no limit.
5. Add a token limit mode method - skip (what the standard tokenizer
does), break (current behavior of the white space tokenizer and its derived
tokenizers), and trim (what I think a lot of people might expect.)
6. Not sure whether to change the current behavior of the character tokenizer
(break mode) to fix it to match the standard tokenizer, or to be trim mode,
which is my choice and likely to be what people might expect.
7. Add matching attributes to the tokenizer factories for Solr, including
Solr XML javadoc.
At a minimum, this issue should address the documentation problem.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests


[ 
https://issues.apache.org/jira/browse/LUCENE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041202#comment-14041202
 ] 

Dawid Weiss commented on LUCENE-5787:
-

I think (didn't check) the leak checker is a test rule, so it cannot be easily 
replaced/ modified (you'd have to shadow the rule chain field and replace all 
the logic in there). Yet another JUnit dead-end corner case I guess.

 LuceneTestCase static leak checker interferes with Groovy unit tests
 

 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.7, 4.8.1
 Environment: Maven 3.0.5
 JUnit 4.11
Reporter: John Gibson

 {{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
 Specifically, Groovy classes have a synthetic static member variable of type 
 {{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too 
 large then LTC will fail the test. Because the variable is added by the 
 Groovy runtime instead of by the developer there is no way for the developer 
 to clear the field themselves.
 Also note that the static leak checker does not ignore memory held by soft or 
 weak references. These should be ignored because the memory retained by such 
 fields will be reclaimed instead of triggering OutOfMemoryErrors.
 Note that because LTC is a base class for Solr's testing support classes this 
 also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests


[ 
https://issues.apache.org/jira/browse/LUCENE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041224#comment-14041224
 ] 

John Gibson commented on LUCENE-5787:
-

{quote}
This means you want to subclass LuceneTestCase with Groovy as basis for your 
own Solr/Lucene tests?
{quote}
Yes, exactly. Although in my case it was subclassing AbstractSolrTestCase 
(which is a descendent of LuceneTestCase).

{quote}
I think (didn't check) the leak checker is a test rule, so it cannot be easily 
replaced/ modified (you'd have to shadow the rule chain field and replace all 
the logic in there). Yet another JUnit dead-end corner case I guess.
{quote}
Dawid, I believe that you're correct. When I first looked into this I couldn't 
figure out how to disable the check, so I ended up copying LTC into my project 
and whitelisting the necessary fields manually.

The most annoying part about this bug was that initially the test worked fine, 
then as I added more fields and code to the suite the ClassInfo reference 
passed the 10 MiB mark and suddenly it stopped working.

{quote}
This bug does not affect Lucene, it is a problem of 3rd party infrastructures 
using our test framework. So the big question: What does other scripting 
laguages do? Scala? JRuby? The list is endless. Maybe add an annotation like 
{noformat}@SuppressLeakChecks([fieldname, fieldname, fieldname]){noformat}
{quote}

This is an excellent observation that I hadn't thought of. An annotation may be 
the correct route. Another option would be to ignore all [synthetic 
fields|http://docs.oracle.com/javase/7/docs/api/java/lang/reflect/Field.html#isSynthetic%28%29].
 (Note that I'm just assuming that Groovy's ClassInfo field is synthetic, given 
that it's generated by the groovy compiler/runtime.)

 LuceneTestCase static leak checker interferes with Groovy unit tests
 

 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.7, 4.8.1
 Environment: Maven 3.0.5
 JUnit 4.11
Reporter: John Gibson

 {{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
 Specifically, Groovy classes have a synthetic static member variable of type 
 {{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too 
 large then LTC will fail the test. Because the variable is added by the 
 Groovy runtime instead of by the developer there is no way for the developer 
 to clear the field themselves.
 Also note that the static leak checker does not ignore memory held by soft or 
 weak references. These should be ignored because the memory retained by such 
 fields will be reclaimed instead of triggering OutOfMemoryErrors.
 Note that because LTC is a base class for Solr's testing support classes this 
 also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests


[ 
https://issues.apache.org/jira/browse/LUCENE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041256#comment-14041256
 ] 

Dawid Weiss commented on LUCENE-5787:
-

Can you check if this is indeed the case (synthetic field), John? It makes 
sense to exclude these I think.

 LuceneTestCase static leak checker interferes with Groovy unit tests
 

 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.7, 4.8.1
 Environment: Maven 3.0.5
 JUnit 4.11
Reporter: John Gibson

 {{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
 Specifically, Groovy classes have a synthetic static member variable of type 
 {{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too 
 large then LTC will fail the test. Because the variable is added by the 
 Groovy runtime instead of by the developer there is no way for the developer 
 to clear the field themselves.
 Also note that the static leak checker does not ignore memory held by soft or 
 weak references. These should be ignored because the memory retained by such 
 fields will be reclaimed instead of triggering OutOfMemoryErrors.
 Note that because LTC is a base class for Solr's testing support classes this 
 also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests


[ 
https://issues.apache.org/jira/browse/LUCENE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041264#comment-14041264
 ] 

Dawid Weiss commented on LUCENE-5787:
-

I changed my mind; synthetic fields may be actually useful to capture (in case 
of some awkward compiler generated stuff we do want to include these, unless 
explicitly excluded).

I think Uwe's idea is better -- an annotation to exclude whatever needs to be 
excluded. I'd make it more flexible by specifying a list of filter classes 
(much like we do with leak thread exclusions); this allows people to write code 
to exclude whatever they need.

 LuceneTestCase static leak checker interferes with Groovy unit tests
 

 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.7, 4.8.1
 Environment: Maven 3.0.5
 JUnit 4.11
Reporter: John Gibson

 {{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
 Specifically, Groovy classes have a synthetic static member variable of type 
 {{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too 
 large then LTC will fail the test. Because the variable is added by the 
 Groovy runtime instead of by the developer there is no way for the developer 
 to clear the field themselves.
 Also note that the static leak checker does not ignore memory held by soft or 
 weak references. These should be ignored because the memory retained by such 
 fields will be reclaimed instead of triggering OutOfMemoryErrors.
 Note that because LTC is a base class for Solr's testing support classes this 
 also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests


 [ 
https://issues.apache.org/jira/browse/LUCENE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Weiss reassigned LUCENE-5787:
---

Assignee: Dawid Weiss

 LuceneTestCase static leak checker interferes with Groovy unit tests
 

 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.7, 4.8.1
 Environment: Maven 3.0.5
 JUnit 4.11
Reporter: John Gibson
Assignee: Dawid Weiss

 {{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
 Specifically, Groovy classes have a synthetic static member variable of type 
 {{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too 
 large then LTC will fail the test. Because the variable is added by the 
 Groovy runtime instead of by the developer there is no way for the developer 
 to clear the field themselves.
 Also note that the static leak checker does not ignore memory held by soft or 
 weak references. These should be ignored because the memory retained by such 
 fields will be reclaimed instead of triggering OutOfMemoryErrors.
 Note that because LTC is a base class for Solr's testing support classes this 
 also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests


[ 
https://issues.apache.org/jira/browse/LUCENE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041305#comment-14041305
 ] 

John Gibson commented on LUCENE-5787:
-

I just checked and Groovy's ClassInfo field is synthetic. However, I agree that 
Uwe's annotation idea is more flexible and will save you from the case where 
the compiler inexplicably generates gigantic static fields which later cause 
OOMs.

 LuceneTestCase static leak checker interferes with Groovy unit tests
 

 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.7, 4.8.1
 Environment: Maven 3.0.5
 JUnit 4.11
Reporter: John Gibson
Assignee: Dawid Weiss

 {{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
 Specifically, Groovy classes have a synthetic static member variable of type 
 {{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too 
 large then LTC will fail the test. Because the variable is added by the 
 Groovy runtime instead of by the developer there is no way for the developer 
 to clear the field themselves.
 Also note that the static leak checker does not ignore memory held by soft or 
 weak references. These should be ignored because the memory retained by such 
 fields will be reclaimed instead of triggering OutOfMemoryErrors.
 Note that because LTC is a base class for Solr's testing support classes this 
 also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [CONF] Apache Solr Reference Guide DocValues

2014-06-23 Thread Chris Hostetter


This edit actaully goes a bit too far...

 In Solr 4.9, the docValuesFormat option was removed and is no longer 
 supported. Storing docValues in memory is the only option allowed.

the 'docValuesFormat=Disk' option no longer works, but specifying a 
'docValuesFormat' attribute does still work, and there are at least 2
significant formats supported that i know of that folks might want to 
consider: the default (currently Lucene49) and Memory.



: Date: Mon, 23 Jun 2014 20:40:00 + (UTC)
: From: Cassandra Targett (Confluence) conflue...@apache.org
: Reply-To: dev@lucene.apache.org
: To: comm...@lucene.apache.org
: Subject: [CONF] Apache Solr Reference Guide  DocValues
: 
: [IMAGE]
: Cassandra Targett edited the page:
: 
: [IMAGE] DOCVALUES
: 
: Comment: update for LUCENE-5761
: 
: View Online · Like · View Changes · Add Comment
: Stop watching space · Manage Notifications
: This message was sent by Atlassian Confluence 5.0.3, Team Collaboration 
Software
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests


[ 
https://issues.apache.org/jira/browse/LUCENE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041341#comment-14041341
 ] 

Uwe Schindler commented on LUCENE-5787:
---

I would allow to use the annotation, but fields explicitely marked as 
synthetic should always be excluded from the leak checker. Those 
fields/methods are defined to be _any constructs introduced by the compiler 
that do not have a corresponding construct in the source code must be marked as 
synthetic, except for default constructors and the class initialization method_ 
(like Java 8 lambda methods and fields, access$-methods, the magic assertions 
enabled static final,...)

 LuceneTestCase static leak checker interferes with Groovy unit tests
 

 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.7, 4.8.1
 Environment: Maven 3.0.5
 JUnit 4.11
Reporter: John Gibson
Assignee: Dawid Weiss

 {{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
 Specifically, Groovy classes have a synthetic static member variable of type 
 {{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too 
 large then LTC will fail the test. Because the variable is added by the 
 Groovy runtime instead of by the developer there is no way for the developer 
 to clear the field themselves.
 Also note that the static leak checker does not ignore memory held by soft or 
 weak references. These should be ignored because the memory retained by such 
 fields will be reclaimed instead of triggering OutOfMemoryErrors.
 Note that because LTC is a base class for Solr's testing support classes this 
 also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests


[ 
https://issues.apache.org/jira/browse/LUCENE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041344#comment-14041344
 ] 

Uwe Schindler commented on LUCENE-5787:
---

bq. here the compiler inexplicably generates gigantic static fields which later 
cause OOMs

The leak checker is meant for stuff in our tests that leak in static fields 
after test class has finished execution. Stuff the compiler adds is nt part of 
that.

 LuceneTestCase static leak checker interferes with Groovy unit tests
 

 Key: LUCENE-5787
 URL: https://issues.apache.org/jira/browse/LUCENE-5787
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.7, 4.8.1
 Environment: Maven 3.0.5
 JUnit 4.11
Reporter: John Gibson
Assignee: Dawid Weiss

 {{LuceneTestCase}}'s static memory leak checker can break Groovy subclasses. 
 Specifically, Groovy classes have a synthetic static member variable of type 
 {{org.codehaus.groovy.reflection.ClassInfo}}. If this variable grows too 
 large then LTC will fail the test. Because the variable is added by the 
 Groovy runtime instead of by the developer there is no way for the developer 
 to clear the field themselves.
 Also note that the static leak checker does not ignore memory held by soft or 
 weak references. These should be ignored because the memory retained by such 
 fields will be reclaimed instead of triggering OutOfMemoryErrors.
 Note that because LTC is a base class for Solr's testing support classes this 
 also affects {{SolrTestCaseJ4}} and {{AbstractSolrTestCase}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6125) add SolrConfig/updateHandler/indexWriter/closeWaitsForMerges=FALSE support


[ 
https://issues.apache.org/jira/browse/SOLR-6125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041394#comment-14041394
 ] 

Hoss Man commented on SOLR-6125:


Why is there no test for this?

And why was the xpath updateHandler/indexWriter/closeWaitsForMerges chosen 
for this? ... there has never been a updateHandler/indexWriter section 
before, and now there is but it only has a single setting under it? that seems 
weird.

 add SolrConfig/updateHandler/indexWriter/closeWaitsForMerges=FALSE support
 --

 Key: SOLR-6125
 URL: https://issues.apache.org/jira/browse/SOLR-6125
 Project: Solr
  Issue Type: Improvement
Reporter: Christine Poerschke
Assignee: Alan Woodward
Priority: Minor

 The problem we saw was slow stopping of the overseer solr instance because it 
 was in the middle of a big merge.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6195) replace updateHandler/indexWriter/closeWaitsForMerges with indexConfig/closeWaitsForMerges

Hoss Man created SOLR-6195:
--

 Summary: replace updateHandler/indexWriter/closeWaitsForMerges 
with indexConfig/closeWaitsForMerges
 Key: SOLR-6195
 URL: https://issues.apache.org/jira/browse/SOLR-6195
 Project: Solr
  Issue Type: Improvement
Reporter: Hoss Man


SOLR-6125 added a new closeWaitsForMerges config option, but in my opinion 
this was done too hastily w/o enough consideration to how/shere the option is 
configured.

* it's currently updateHandler/indexWriter/closeWaitsForMerges
** there has never been an updateHandler/indexWriter section in 
solrconfig.xml until now
** this is the only setting that currently exists in this section
* this setting, although used by DirectUpdateHandler2, does not (from a user 
perspective really fit with the other existing settings in {{updateHandler}}
** i think from a user perspective, it would make much more sense in 
{{indexConfig}} along side the other merge related settings.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6195) replace updateHandler/indexWriter/closeWaitsForMerges with indexConfig/closeWaitsForMerges


[ 
https://issues.apache.org/jira/browse/SOLR-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041412#comment-14041412
 ] 

Hoss Man commented on SOLR-6195:


suggestion for moving forward:

* leave most of the code SOLR-6125 added alone, but mark it deprecated
* add new logic for a {{indexConfig/closeWriterWaitsForMerges}} Boolean in 
SolrIndexConfig which defaults to null
* add new logic in DirectUpdateHandler2 to check 
{{core.getSolrConfig().indexConfig.closeWriterWaitsForMerges}} first, and only 
if null does it look at {{updateHandlerInfo.indexWriterCloseWaitsForMerges;}}
* add tests for both config options

 replace updateHandler/indexWriter/closeWaitsForMerges with 
 indexConfig/closeWaitsForMerges
 --

 Key: SOLR-6195
 URL: https://issues.apache.org/jira/browse/SOLR-6195
 Project: Solr
  Issue Type: Improvement
Reporter: Hoss Man

 SOLR-6125 added a new closeWaitsForMerges config option, but in my opinion 
 this was done too hastily w/o enough consideration to how/shere the option is 
 configured.
 * it's currently updateHandler/indexWriter/closeWaitsForMerges
 ** there has never been an updateHandler/indexWriter section in 
 solrconfig.xml until now
 ** this is the only setting that currently exists in this section
 * this setting, although used by DirectUpdateHandler2, does not (from a user 
 perspective really fit with the other existing settings in {{updateHandler}}
 ** i think from a user perspective, it would make much more sense in 
 {{indexConfig}} along side the other merge related settings.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6125) add SolrConfig/updateHandler/indexWriter/closeWaitsForMerges=FALSE support


[ 
https://issues.apache.org/jira/browse/SOLR-6125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041414#comment-14041414
 ] 

Hoss Man commented on SOLR-6125:


This is already in 4.9, but i really think we should try to make it more 
consistent with the other configs options before 4.10.

filed SOLR-6195 to try and do that.

 add SolrConfig/updateHandler/indexWriter/closeWaitsForMerges=FALSE support
 --

 Key: SOLR-6125
 URL: https://issues.apache.org/jira/browse/SOLR-6125
 Project: Solr
  Issue Type: Improvement
Reporter: Christine Poerschke
Assignee: Alan Woodward
Priority: Minor

 The problem we saw was slow stopping of the overseer solr instance because it 
 was in the middle of a big merge.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6195) replace updateHandler/indexWriter/closeWaitsForMerges with indexConfig/closeWriterWaitsForMerges


 [ 
https://issues.apache.org/jira/browse/SOLR-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-6195:
---

Summary: replace updateHandler/indexWriter/closeWaitsForMerges with 
indexConfig/closeWriterWaitsForMerges  (was: replace 
updateHandler/indexWriter/closeWaitsForMerges with 
indexConfig/closeWaitsForMerges)

 replace updateHandler/indexWriter/closeWaitsForMerges with 
 indexConfig/closeWriterWaitsForMerges
 

 Key: SOLR-6195
 URL: https://issues.apache.org/jira/browse/SOLR-6195
 Project: Solr
  Issue Type: Improvement
Reporter: Hoss Man

 SOLR-6125 added a new closeWaitsForMerges config option, but in my opinion 
 this was done too hastily w/o enough consideration to how/shere the option is 
 configured.
 * it's currently updateHandler/indexWriter/closeWaitsForMerges
 ** there has never been an updateHandler/indexWriter section in 
 solrconfig.xml until now
 ** this is the only setting that currently exists in this section
 * this setting, although used by DirectUpdateHandler2, does not (from a user 
 perspective really fit with the other existing settings in {{updateHandler}}
 ** i think from a user perspective, it would make much more sense in 
 {{indexConfig}} along side the other merge related settings.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-Windows (64bit/jdk1.7.0_60) - Build # 4047 - Failure!

2014-06-23 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Windows/4047/
Java: 64bit/jdk1.7.0_60 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
REGRESSION:  
org.apache.lucene.index.TestConcurrentMergeScheduler.testTotalBytesSize

Error Message:


Stack Trace:
java.lang.AssertionError
at 
__randomizedtesting.SeedInfo.seed([E514CC01D224ED97:F0F79C983D8CCA4E]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at 
org.apache.lucene.index.TestConcurrentMergeScheduler.testTotalBytesSize(TestConcurrentMergeScheduler.java:366)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 555 lines...]
   [junit4] Suite: org.apache.lucene.index.TestConcurrentMergeScheduler
   [junit4]   2 NOTE: reproduce with: ant test  
-Dtestcase=TestConcurrentMergeScheduler -Dtests.method=testTotalBytesSize 
-Dtests.seed=E514CC01D224ED97 -Dtests.slow=true -Dtests.locale=th_TH 
-Dtests.timezone=America/Argentina/La_Rioja

[jira] [Commented] (LUCENE-5755) Explore alternative build systems

2014-06-23 Thread Matt Davis (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041557#comment-14041557
 ] 

Matt Davis commented on LUCENE-5755:


I get errors due to the System.out checks compiling Lucene tests cases due to 
Gradle stealing System.out in my project:

java.lang.AssertionError: Something has changed System.out to: 
org.gradle.util.LinePerThreadBufferingOutputStream


 Explore alternative build systems
 -

 Key: LUCENE-5755
 URL: https://issues.apache.org/jira/browse/LUCENE-5755
 Project: Lucene - Core
  Issue Type: Task
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor

 I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr. 
 It's not even the tool's fault; it seems Lucene builds just hit the borders 
 of what it can do, especially in terms of submodule dependencies etc.
 I don't think Maven will help much too, given certain things I'd like to have 
 in the build (for example collect all tests globally for a single execution 
 phase at the end of the build, to support better load-balancing).
 I'd like to explore Gradle as an alternative. This task is a notepad for 
 thoughts and experiments.
 An example of a complex (?) gradle build is javafx, for example.
 http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-5755) Explore alternative build systems

2014-06-23 Thread Matt Davis (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041557#comment-14041557
 ] 

Matt Davis edited comment on LUCENE-5755 at 6/24/14 1:07 AM:
-

I get errors due to the System.out checks compiling Lucene tests cases due to 
Gradle stealing System.out in my project:

java.lang.AssertionError: Something has changed System.out to: 
org.gradle.util.LinePerThreadBufferingOutputStream

http://forums.gradle.org/gradle/topics/capturing_output_in_unit_test_and_verifying_it


was (Author: mdavis95):
I get errors due to the System.out checks compiling Lucene tests cases due to 
Gradle stealing System.out in my project:

java.lang.AssertionError: Something has changed System.out to: 
org.gradle.util.LinePerThreadBufferingOutputStream


 Explore alternative build systems
 -

 Key: LUCENE-5755
 URL: https://issues.apache.org/jira/browse/LUCENE-5755
 Project: Lucene - Core
  Issue Type: Task
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor

 I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr. 
 It's not even the tool's fault; it seems Lucene builds just hit the borders 
 of what it can do, especially in terms of submodule dependencies etc.
 I don't think Maven will help much too, given certain things I'd like to have 
 in the build (for example collect all tests globally for a single execution 
 phase at the end of the build, to support better load-balancing).
 I'd like to explore Gradle as an alternative. This task is a notepad for 
 thoughts and experiments.
 An example of a complex (?) gradle build is javafx, for example.
 http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-4.x-Windows (64bit/jdk1.7.0_60) - Build # 4047 - Failure!

2014-06-23 Thread Michael McCandless

I committed a fix.

Mike McCandless

http://blog.mikemccandless.com


On Mon, Jun 23, 2014 at 11:39 PM, Policeman Jenkins Server
jenk...@thetaphi.de wrote:
 Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Windows/4047/
 Java: 64bit/jdk1.7.0_60 -XX:-UseCompressedOops -XX:+UseG1GC

 1 tests failed.
 REGRESSION:  
 org.apache.lucene.index.TestConcurrentMergeScheduler.testTotalBytesSize

 Error Message:


 Stack Trace:
 java.lang.AssertionError
 at 
 __randomizedtesting.SeedInfo.seed([E514CC01D224ED97:F0F79C983D8CCA4E]:0)
 at org.junit.Assert.fail(Assert.java:92)
 at org.junit.Assert.assertTrue(Assert.java:43)
 at org.junit.Assert.assertTrue(Assert.java:54)
 at 
 org.apache.lucene.index.TestConcurrentMergeScheduler.testTotalBytesSize(TestConcurrentMergeScheduler.java:366)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at 
 com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
 at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
 at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
 at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
 at 
 org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
 at 
 org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
 at 
 org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
 at 
 com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
 at 
 org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
 at 
 org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
 at 
 org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
 at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
 at 
 com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
 at 
 com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
 at 
 com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
 at 
 com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
 at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
 at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
 at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
 at 
 org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
 at 
 org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
 at 
 com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
 at 
 com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
 at 
 com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
 at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
 at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
 at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
 at 
 org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
 at 
 org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
 at 
 org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
 at 
 org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
 at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
 at 
 com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
 at java.lang.Thread.run(Thread.java:745)




 Build Log:
 [...truncated 555 lines...]
[junit4] Suite: org.apache.lucene.index.TestConcurrentMergeScheduler

Re: [CONF] Apache Solr Reference Guide DocValues

2014-06-23 Thread Shalin Shekhar Mangar

There's a direct format too.


On Tue, Jun 24, 2014 at 2:47 AM, Chris Hostetter hossman_luc...@fucit.org
wrote:


 This edit actaully goes a bit too far...

  In Solr 4.9, the docValuesFormat option was removed and is no longer
  supported. Storing docValues in memory is the only option allowed.

 the 'docValuesFormat=Disk' option no longer works, but specifying a
 'docValuesFormat' attribute does still work, and there are at least 2
 significant formats supported that i know of that folks might want to
 consider: the default (currently Lucene49) and Memory.



 : Date: Mon, 23 Jun 2014 20:40:00 + (UTC)
 : From: Cassandra Targett (Confluence) conflue...@apache.org
 : Reply-To: dev@lucene.apache.org
 : To: comm...@lucene.apache.org
 : Subject: [CONF] Apache Solr Reference Guide  DocValues
 :
 : [IMAGE]
 : Cassandra Targett edited the page:
 :
 : [IMAGE] DOCVALUES
 :
 : Comment: update for LUCENE-5761
 :
 : View Online · Like · View Changes · Add Comment
 : Stop watching space · Manage Notifications
 : This message was sent by Atlassian Confluence 5.0.3, Team Collaboration
 Software
 :

 -Hoss
 http://www.lucidworks.com/


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org




-- 
Regards,
Shalin Shekhar Mangar.

[jira] [Created] (SOLR-6196) The overseerstatus collection API should instrument amILeader and ZK update calls as well

2014-06-23 Thread Shalin Shekhar Mangar (JIRA)

Shalin Shekhar Mangar created SOLR-6196:
---

 Summary: The overseerstatus collection API should instrument 
amILeader and ZK update calls as well
 Key: SOLR-6196
 URL: https://issues.apache.org/jira/browse/SOLR-6196
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
 Fix For: 5.0, 4.10


We don't instrument amILeader calls right now. We have a large cluster where we 
find that the overseer isn't processing operations as fast as it should. Since 
all operations are pretty fast even at the 99th percentile, the slowdown might 
be due to amILeader and the ZK update calls.

Let's start measuring those as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5755) Explore alternative build systems


[ 
https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041717#comment-14041717
 ] 

Dawid Weiss commented on LUCENE-5755:
-

You will have to revert the sysout to what it was before the test suite 
started. This isn't a bug, it's a requirement to cleanup to pristine state 
after a suite is complete.

 Explore alternative build systems
 -

 Key: LUCENE-5755
 URL: https://issues.apache.org/jira/browse/LUCENE-5755
 Project: Lucene - Core
  Issue Type: Task
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor

 I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr. 
 It's not even the tool's fault; it seems Lucene builds just hit the borders 
 of what it can do, especially in terms of submodule dependencies etc.
 I don't think Maven will help much too, given certain things I'd like to have 
 in the build (for example collect all tests globally for a single execution 
 phase at the end of the build, to support better load-balancing).
 I'd like to explore Gradle as an alternative. This task is a notepad for 
 thoughts and experiments.
 An example of a complex (?) gradle build is javafx, for example.
 http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5755) Explore alternative build systems


[ 
https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041719#comment-14041719
 ] 

Dawid Weiss commented on LUCENE-5755:
-

Also, do you have a gradle build for Lucene (even partial)? I'd be nice if you 
could share!

 Explore alternative build systems
 -

 Key: LUCENE-5755
 URL: https://issues.apache.org/jira/browse/LUCENE-5755
 Project: Lucene - Core
  Issue Type: Task
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor

 I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr. 
 It's not even the tool's fault; it seems Lucene builds just hit the borders 
 of what it can do, especially in terms of submodule dependencies etc.
 I don't think Maven will help much too, given certain things I'd like to have 
 in the build (for example collect all tests globally for a single execution 
 phase at the end of the build, to support better load-balancing).
 I'd like to explore Gradle as an alternative. This task is a notepad for 
 thoughts and experiments.
 An example of a complex (?) gradle build is javafx, for example.
 http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5787) LuceneTestCase static leak checker interferes with Groovy unit tests