Good morning
Ah I see. I don't get those warnings at all. Can you maybe try to see if
the same behaviour insists when you go back to the previous version of ES
and Lucene locally as we had until April 3rd:
Lucene 4.10.3 => 4.10.2
ES 1.4.4 => 1.4.2
Do you then still see the same warnings?
Babak
Am 04.04.15 07:58 schrieb "Claus Ibsen" unter <claus.ib...@gmail.com>:
>Hi
>
>I noticed that if you are low on free disk space, then you get WARNs -
>well I got 20gb free.
>
>2015-04-04 07:59:06,391 [pdateTask][T#1]] WARN local
> - [Enigma] timed out waiting for all nodes to process
>published state [15] (timeout [30s], pending nodes: [[Night
>Rider][P7IzfV6oQkWsmz5JJzM-Ug][davsclaus.air][local[5]]{local=true}])
>2015-04-04 07:59:16,621 [anagement][T#2]] WARN decider
> - [Tempo] high disk watermark [10%] exceeded on
>[g56rnNQ8Qt-ckLr3_uCg_g][Tempo] free: 18.5gb[7.9%], shards will be
>relocated away from this node
>
>
>So I think the unit test needs to be adjusted so it wont require a lot
>of free GB to run. I haven't seen this on the older version, the test
>passed in reasonable time. Now it runs forever and I have to kill it.
>
>
>On Fri, Apr 3, 2015 at 8:33 PM, Babak Vahdat
><babak.vah...@swissonline.ch> wrote:
>> Hi
>>
>> Yes I noticed that as well however the test will succeed & pass if you
>>would
>> wait a bit longer, however with some thrown exceptions like:
>>
>> Running
>>org.apache.camel.component.elasticsearch.ElasticsearchComponentTest
>> Exception in thread
>> "elasticsearch[Goldeneye][clusterService#updateTask][T#1]"
>> org.elasticsearch.common.util.concurrent.EsRejectedExecutionException:
>> rejected execution (shutting down) on
>>
>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@25b9
>>9556
>> at
>>
>>org.elasticsearch.common.util.concurrent.EsAbortPolicy.rejectedExecution(
>>EsAbortPolicy.java:62)
>> at
>>
>>java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:82
>>1)
>> at
>>
>>java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(Scheduled
>>ThreadPoolExecutor.java:325)
>> at
>>
>>java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThread
>>PoolExecutor.java:530)
>> at
>>org.elasticsearch.threadpool.ThreadPool.schedule(ThreadPool.java:245)
>> at
>>
>>org.elasticsearch.cluster.service.InternalClusterService$1.run(InternalCl
>>usterService.java:230)
>> at
>>
>>org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$
>>TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:1
>>53)
>> at
>>
>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
>>:1145)
>> at
>>
>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
>>a:615)
>> at java.lang.Thread.run(Thread.java:745)
>> Exception in thread
>> "elasticsearch[Goldeneye][clusterService#updateTask][T#2]"
>> org.elasticsearch.common.util.concurrent.EsRejectedExecutionException:
>> rejected execution (shutting down) on
>>
>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@47a0
>>b02b
>> at
>>
>>org.elasticsearch.common.util.concurrent.EsAbortPolicy.rejectedExecution(
>>EsAbortPolicy.java:62)
>> at
>>
>>java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:82
>>1)
>> at
>>
>>java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(Scheduled
>>ThreadPoolExecutor.java:325)
>> at
>>
>>java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThread
>>PoolExecutor.java:530)
>> at
>>org.elasticsearch.threadpool.ThreadPool.schedule(ThreadPool.java:245)
>> at
>>
>>org.elasticsearch.cluster.service.InternalClusterService$1.run(InternalCl
>>usterService.java:230)
>> at
>>
>>org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$
>>TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:1
>>53)
>> at
>>
>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
>>:1145)
>> at
>>
>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
>>a:615)
>> at java.lang.Thread.run(Thread.java:745)
>> Tests run: 16, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 80.196
>>sec
>> - in org.apache.camel.component.elastic
>>
>> On the other hand this has nothing to do with the ES upgrade and if you
>> would revert to the commit BEFORE the Lucene and the Elasticsearch
>>upgrade
>> of yesterday and today then the behaviour would be exactly the same.
>>Again,
>> it's not nice, but the test passes!
>>
>> That all said, maybe something else very important I've noticed which we
>> should be aware of: in general the upgrade of Lucene and ES inside the
>> parent POM should be always in sync, as ES internally depends on
>>Lucene, if
>> you once do only upgrade Lucene version, then for camel-elasticsearch
>>Maven
>> pulls in the Lucene version we've from the parent POM:
>>
>> https://github.com/apache/camel/blob/master/parent/pom.xml#L2161
>>
>> As an example see how we've got a mix of Lucene versions 4.10.3 and
>>4.10.2
>> below. This was the case yesterday as I only upgraded Lucene but not ES.
>>
>> ~/dev/workspace/camel/components/camel-elasticsearch>git reset --hard
>> d91b0c708e6dce761aef12868802e0d57d45d256
>> HEAD is now at d91b0c7 Removed unnecessary method calls
>> ~/dev/workspace/camel/components/camel-elasticsearch>mvn dependency:tree
>> -Dverbose=true
>> [INFO] Scanning for projects...
>> [INFO]
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Building Camel :: ElasticSearch 2.16-SNAPSHOT
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO]
>> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @
>> camel-elasticsearch ---
>> [INFO] org.apache.camel:camel-elasticsearch:bundle:2.16-SNAPSHOT
>> [INFO] +- org.apache.camel:camel-core:jar:2.16-SNAPSHOT:compile
>> [INFO] | +- org.slf4j:slf4j-api:jar:1.6.6:compile
>> [INFO] | +- (com.sun.xml.bind:jaxb-core:jar:2.2.11:compile - omitted
>>for
>> duplicate)
>> [INFO] | \- (com.sun.xml.bind:jaxb-impl:jar:2.2.11:compile - omitted
>>for
>> duplicate)
>> [INFO] +- org.elasticsearch:elasticsearch:jar:1.4.2:compile
>> [INFO] | +- org.apache.lucene:lucene-core:jar:4.10.3:compile (version
>> managed from 4.10.2)
>> [INFO] | +-
>>org.apache.lucene:lucene-analyzers-common:jar:4.10.3:compile
>> (version managed from 4.10.2)
>> [INFO] | | \- (org.apache.lucene:lucene-core:jar:4.10.3:compile -
>>version
>> managed from 4.10.2; omitted for duplicate)
>> [INFO] | +- org.apache.lucene:lucene-queries:jar:4.10.2:compile
>> [INFO] | | \- (org.apache.lucene:lucene-core:jar:4.10.3:compile -
>>version
>> managed from 4.10.2; omitted for duplicate)
>> [INFO] | +- org.apache.lucene:lucene-memory:jar:4.10.2:compile
>> [INFO] | | \- (org.apache.lucene:lucene-core:jar:4.10.3:compile -
>>version
>> managed from 4.10.2; omitted for duplicate)
>> [INFO] | +- org.apache.lucene:lucene-highlighter:jar:4.10.2:compile
>> [INFO] | | +- (org.apache.lucene:lucene-core:jar:4.10.3:compile -
>>version
>> managed from 4.10.2; omitted for duplicate)
>> [INFO] | | +- (org.apache.lucene:lucene-memory:jar:4.10.2:compile -
>> omitted for duplicate)
>> [INFO] | | \- (org.apache.lucene:lucene-queries:jar:4.10.2:compile -
>> omitted for duplicate)
>> [INFO] | +- org.apache.lucene:lucene-queryparser:jar:4.10.3:compile
>> (version managed from 4.10.2)
>> [INFO] | | +- (org.apache.lucene:lucene-core:jar:4.10.3:compile -
>>version
>> managed from 4.10.2; omitted for duplicate)
>> [INFO] | | +- (org.apache.lucene:lucene-queries:jar:4.10.3:compile -
>> omitted for conflict with 4.10.2)
>> [INFO] | | \- (org.apache.lucene:lucene-sandbox:jar:4.10.3:compile -
>> omitted for conflict with 4.10.2)
>> [INFO] | +- org.apache.lucene:lucene-sandbox:jar:4.10.2:compile
>> [INFO] | | \- (org.apache.lucene:lucene-core:jar:4.10.3:compile -
>>version
>> managed from 4.10.2; omitted for duplicate)
>> [INFO] | +- org.apache.lucene:lucene-suggest:jar:4.10.2:compile
>> [INFO] | | +-
>> (org.apache.lucene:lucene-analyzers-common:jar:4.10.3:compile - version
>> managed from 4.10.2; omitted for duplicate)
>> [INFO] | | +- (org.apache.lucene:lucene-core:jar:4.10.3:compile -
>>version
>> managed from 4.10.2; omitted for duplicate)
>> [INFO] | | +- (org.apache.lucene:lucene-misc:jar:4.10.2:compile -
>>omitted
>> for duplicate)
>> [INFO] | | \- (org.apache.lucene:lucene-queries:jar:4.10.2:compile -
>> omitted for duplicate)
>> [INFO] | +- org.apache.lucene:lucene-misc:jar:4.10.2:compile
>> [INFO] | | \- (org.apache.lucene:lucene-core:jar:4.10.3:compile -
>>version
>> managed from 4.10.2; omitted for duplicate)
>> [INFO] | +- org.apache.lucene:lucene-join:jar:4.10.2:compile
>> [INFO] | | +- (org.apache.lucene:lucene-core:jar:4.10.3:compile -
>>version
>> managed from 4.10.2; omitted for duplicate)
>> [INFO] | | \- (org.apache.lucene:lucene-grouping:jar:4.10.2:compile -
>> omitted for duplicate)
>> [INFO] | +- org.apache.lucene:lucene-grouping:jar:4.10.2:compile
>> [INFO] | | +- (org.apache.lucene:lucene-core:jar:4.10.3:compile -
>>version
>> managed from 4.10.2; omitted for duplicate)
>> [INFO] | | \- (org.apache.lucene:lucene-queries:jar:4.10.2:compile -
>> omitted for duplicate)
>> [INFO] | \- org.apache.lucene:lucene-spatial:jar:4.10.2:compile
>> [INFO] | +- (org.apache.lucene:lucene-core:jar:4.10.3:compile -
>>version
>> managed from 4.10.2; omitted for duplicate)
>> [INFO] | +- (org.apache.lucene:lucene-queries:jar:4.10.2:compile -
>> omitted for duplicate)
>> [INFO] | \- com.spatial4j:spatial4j:jar:0.4.1:compile
>> [INFO] +- org.apache.camel:camel-test:jar:2.16-SNAPSHOT:test
>> [INFO] | +- (org.apache.camel:camel-core:jar:2.16-SNAPSHOT:test -
>>omitted
>> for duplicate)
>> [INFO] | +- junit:junit:jar:4.11:test
>> [INFO] | | \- org.hamcrest:hamcrest-core:jar:1.3:test
>> [INFO] | +- (com.sun.xml.bind:jaxb-core:jar:2.2.11:test - omitted for
>> duplicate)
>> [INFO] | \- (com.sun.xml.bind:jaxb-impl:jar:2.2.11:test - omitted for
>> duplicate)
>> [INFO] +- org.apache.camel:camel-test-spring:jar:2.16-SNAPSHOT:test
>> [INFO] | +- (org.apache.camel:camel-test:jar:2.16-SNAPSHOT:test -
>>omitted
>> for duplicate)
>> [INFO] | +- org.apache.camel:camel-spring:jar:2.16-SNAPSHOT:test
>> [INFO] | | +- (org.apache.camel:camel-core:jar:2.16-SNAPSHOT:test -
>> omitted for duplicate)
>> [INFO] | | +-
>>(org.springframework:spring-context:jar:4.1.6.RELEASE:test -
>> omitted for duplicate)
>> [INFO] | | +- (org.springframework:spring-aop:jar:4.1.6.RELEASE:test -
>> omitted for duplicate)
>> [INFO] | | +- (org.springframework:spring-tx:jar:4.1.6.RELEASE:test -
>> omitted for duplicate)
>> [INFO] | | +- (com.sun.xml.bind:jaxb-core:jar:2.2.11:test - omitted
>>for
>> duplicate)
>> [INFO] | | \- (com.sun.xml.bind:jaxb-impl:jar:2.2.11:test - omitted
>>for
>> duplicate)
>> [INFO] | +- (org.springframework:spring-test:jar:4.1.6.RELEASE:test -
>> omitted for duplicate)
>> [INFO] | +- org.springframework:spring-context:jar:4.1.6.RELEASE:test
>> [INFO] | | +- (org.springframework:spring-aop:jar:4.1.6.RELEASE:test -
>> omitted for duplicate)
>> [INFO] | | +-
>>(org.springframework:spring-beans:jar:4.1.6.RELEASE:test -
>> omitted for duplicate)
>> [INFO] | | +- (org.springframework:spring-core:jar:4.1.6.RELEASE:test
>>-
>> omitted for duplicate)
>> [INFO] | | \-
>> (org.springframework:spring-expression:jar:4.1.6.RELEASE:test - omitted
>>for
>> duplicate)
>> [INFO] | +- org.springframework:spring-beans:jar:4.1.6.RELEASE:test
>> [INFO] | | \- (org.springframework:spring-core:jar:4.1.6.RELEASE:test
>>-
>> omitted for duplicate)
>> [INFO] | +-
>>org.springframework:spring-expression:jar:4.1.6.RELEASE:test
>> [INFO] | | \- (org.springframework:spring-core:jar:4.1.6.RELEASE:test
>>-
>> omitted for duplicate)
>> [INFO] | +- org.springframework:spring-aop:jar:4.1.6.RELEASE:test
>> [INFO] | | +- aopalliance:aopalliance:jar:1.0:test
>> [INFO] | | +-
>>(org.springframework:spring-beans:jar:4.1.6.RELEASE:test -
>> omitted for duplicate)
>> [INFO] | | \- (org.springframework:spring-core:jar:4.1.6.RELEASE:test
>>-
>> omitted for duplicate)
>> [INFO] | +- org.springframework:spring-tx:jar:4.1.6.RELEASE:test
>> [INFO] | | +-
>>(org.springframework:spring-beans:jar:4.1.6.RELEASE:test -
>> omitted for duplicate)
>> [INFO] | | \- (org.springframework:spring-core:jar:4.1.6.RELEASE:test
>>-
>> omitted for duplicate)
>> [INFO] | +- org.springframework:spring-core:jar:4.1.6.RELEASE:test
>> [INFO] | | \- commons-logging:commons-logging:jar:1.1.3:test (version
>> managed from 1.2)
>> [INFO] | +- (com.sun.xml.bind:jaxb-core:jar:2.2.11:test - omitted for
>> duplicate)
>> [INFO] | \- (com.sun.xml.bind:jaxb-impl:jar:2.2.11:test - omitted for
>> duplicate)
>> [INFO] +- org.springframework:spring-test:jar:4.1.6.RELEASE:test
>> [INFO] | \- (org.springframework:spring-core:jar:4.1.6.RELEASE:test -
>> omitted for duplicate)
>> [INFO] +- org.slf4j:slf4j-log4j12:jar:1.7.10:test
>> [INFO] | +- (org.slf4j:slf4j-api:jar:1.6.6:test - version managed from
>> 1.7.10; omitted for duplicate)
>> [INFO] | \- log4j:log4j:jar:1.2.17:test
>> [INFO] +- org.apache.camel:apt:jar:2.16-SNAPSHOT:provided
>> [INFO] | +- (org.slf4j:slf4j-api:jar:1.6.6:provided - version managed
>>from
>> 1.7.10; omitted for duplicate)
>> [INFO] | +- org.apache.camel:spi-annotations:jar:2.16-SNAPSHOT:provided
>> [INFO] | | +- (com.sun.xml.bind:jaxb-core:jar:2.2.11:provided -
>>omitted
>> for duplicate)
>> [INFO] | | \- (com.sun.xml.bind:jaxb-impl:jar:2.2.11:provided -
>>omitted
>> for duplicate)
>> [INFO] | +- (com.sun.xml.bind:jaxb-core:jar:2.2.11:provided - omitted
>>for
>> duplicate)
>> [INFO] | \- (com.sun.xml.bind:jaxb-impl:jar:2.2.11:provided - omitted
>>for
>> duplicate)
>> [INFO] +- com.sun.xml.bind:jaxb-core:jar:2.2.11:compile
>> [INFO] \- com.sun.xml.bind:jaxb-impl:jar:2.2.11:compile
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] BUILD SUCCESS
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Total time: 4.112 s
>> [INFO] Finished at: 2015-04-03T20:15:12+02:00
>> [INFO] Final Memory: 29M/1963M
>> [INFO]
>> ------------------------------------------------------------------------
>>
>> Then as a result of this the camel-elasticsearch tests failed today on
>>CI:
>>
>> https://builds.apache.org/view/All/job/Camel.trunk.fulltest/2249/
>>
>> Babak
>>
>>
>> Claus Ibsen-2 wrote
>>> Hi
>>>
>>> Does the unit test of camel-elastichsearch hang for you? I tried
>>> merging in a PR about another ES fix (a small one) and then it hangs
>>> for me.
>>>
>>> I didnt look into it further, but just wander of the 1.4.2 -> 1.4.4
>>> worked fine for you / others who have build latest code?
>>>
>>> On Fri, Apr 3, 2015 at 3:29 PM, <
>>
>>> bvahdat@
>>
>>> > wrote:
>>>> Repository: camel
>>>> Updated Branches:
>>>> refs/heads/master d91b0c708 -> 9ff05dc66
>>>>
>>>>
>>>> Upgrade to Elasticsearch 1.4.4
>>>>
>>>> Project: http://git-wip-us.apache.org/repos/asf/camel/repo
>>>> Commit: http://git-wip-us.apache.org/repos/asf/camel/commit/9ff05dc6
>>>> Tree: http://git-wip-us.apache.org/repos/asf/camel/tree/9ff05dc6
>>>> Diff: http://git-wip-us.apache.org/repos/asf/camel/diff/9ff05dc6
>>>>
>>>> Branch: refs/heads/master
>>>> Commit: 9ff05dc6619335bec4c0ea6972a9d3da6f1b04c8
>>>> Parents: d91b0c7
>>>> Author: Babak Vahdat <
>>
>>> bvahdat@
>>
>>> >
>>>> Authored: Fri Apr 3 15:29:00 2015 +0200
>>>> Committer: Babak Vahdat <
>>
>>> bvahdat@
>>
>>> >
>>>> Committed: Fri Apr 3 15:29:00 2015 +0200
>>>>
>>>> ----------------------------------------------------------------------
>>>> .../component/elasticsearch/ElasticsearchComponentTest.java | 2
>>>>+-
>>>> parent/pom.xml | 4
>>>> ++--
>>>> 2 files changed, 3 insertions(+), 3 deletions(-)
>>>> ----------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>>http://git-wip-us.apache.org/repos/asf/camel/blob/9ff05dc6/components/c
>>>>amel-elasticsearch/src/test/java/org/apache/camel/component/elasticsear
>>>>ch/ElasticsearchComponentTest.java
>>>> ----------------------------------------------------------------------
>>>> diff --git
>>>>
>>>>a/components/camel-elasticsearch/src/test/java/org/apache/camel/compone
>>>>nt/elasticsearch/ElasticsearchComponentTest.java
>>>>
>>>>b/components/camel-elasticsearch/src/test/java/org/apache/camel/compone
>>>>nt/elasticsearch/ElasticsearchComponentTest.java
>>>> index 55d8998..acfa21e 100644
>>>> ---
>>>>
>>>>a/components/camel-elasticsearch/src/test/java/org/apache/camel/compone
>>>>nt/elasticsearch/ElasticsearchComponentTest.java
>>>> +++
>>>>
>>>>b/components/camel-elasticsearch/src/test/java/org/apache/camel/compone
>>>>nt/elasticsearch/ElasticsearchComponentTest.java
>>>> @@ -68,7 +68,7 @@ public class ElasticsearchComponentTest extends
>>>> CamelTestSupport {
>>>> documents.add(document1);
>>>> documents.add(document2);
>>>>
>>>> - List indexIds = template.requestBody("direct:bulk_index",
>>>> documents, List.class);
>>>> + List<?> indexIds = template.requestBody("direct:bulk_index",
>>>> documents, List.class);
>>>> assertNotNull("indexIds should be set", indexIds);
>>>> assertCollectionSize("Indexed documents should match the
>>>>size of
>>>> documents", indexIds, documents.size());
>>>> }
>>>>
>>>>
>>>>http://git-wip-us.apache.org/repos/asf/camel/blob/9ff05dc6/parent/pom.x
>>>>ml
>>>> ----------------------------------------------------------------------
>>>> diff --git a/parent/pom.xml b/parent/pom.xml
>>>> index 8956c67..5bc200b2 100644
>>>> --- a/parent/pom.xml
>>>> +++ b/parent/pom.xml
>>>> @@ -142,8 +142,8 @@
>>>>
>>> <ehcache-version>
>>> 2.9.1
>>> </ehcache-version>
>>>>
>>> <egit-github-core-version>
>>> 2.1.5
>>> </egit-github-core-version>
>>>>
>>> <egit-github-core-bundle-version>
>>> 2.1.5_1
>>> </egit-github-core-bundle-version>
>>>> -
>>> <elasticsearch-bundle-version>
>>> 1.4.2_1
>>> </elasticsearch-bundle-version>
>>>> -
>>> <elasticsearch-version>
>>> 1.4.2
>>> </elasticsearch-version>
>>>> +
>>> <elasticsearch-bundle-version>
>>> 1.4.4_1
>>> </elasticsearch-bundle-version>
>>>> +
>>> <elasticsearch-version>
>>> 1.4.4
>>> </elasticsearch-version>
>>>>
>>> <el-api-1.0-version>
>>> 1.0.1
>>> </el-api-1.0-version>
>>>>
>>> <embedmongo-version>
>>> 1.47.0
>>> </embedmongo-version>
>>>>
>>> <exec-maven-plugin-version>
>>> 1.2.1
>>> </exec-maven-plugin-version>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> -----------------
>>> Red Hat, Inc.
>>> Email:
>>
>>> cibsen@
>>
>>> Twitter: davsclaus
>>> Blog: http://davsclaus.com
>>> Author of Camel in Action: http://www.manning.com/ibsen
>>> hawtio: http://hawt.io/
>>> fabric8: http://fabric8.io/
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>>http://camel.465427.n5.nabble.com/Re-camel-git-commit-Upgrade-to-Elastics
>>earch-1-4-4-tp5765284p5765293.html
>> Sent from the Camel Development mailing list archive at Nabble.com.
>
>
>
>--
>Claus Ibsen
>-----------------
>Red Hat, Inc.
>Email: cib...@redhat.com
>Twitter: davsclaus
>Blog: http://davsclaus.com
>Author of Camel in Action: http://www.manning.com/ibsen
>hawtio: http://hawt.io/
>fabric8: http://fabric8.io/
