Hi,
I've created the pull request: https://github.com/apache/storm/pull/2029.
I've used embedded ES 2 for the integration tests for now. However, if
we did want to test multiple versions, then we wouldn't need to install
ES manually on Travis/Jenkins, because either the Ant scripts or
docker-maven-plugin would do that for us. We would need Docker for
docker-maven-plugin though.
However, as you say, let's worry about this pull request first. I'm
happy to look at integration testing another time if you feel it's
important.
Kind regards,
Heather
On 23/03/17 00:14, Jungtaek Lim wrote:
IMHO, testing against embedded ES feels enough to me.
If we would like to add ES integration test, we should make sure
Travis script installs and runs ES before running tests, and I'm not
sure we can do this with ASF Jenkins. Not only docker but also ES for
prerequisite.
Let's create PR as it as, and watch more opinions around this thread.
Thanks,
Jungtaek Lim (HeartSaVioR)
2017년 3월 23일 (목) 오전 6:35, Heather McCartney
<[email protected] <mailto:[email protected]>>님이 작성:
Hi again,
I've done some work on this over at
https://github.com/hmcc/storm/tree/STORM-2379. I've used the new
Elasticsearch Java REST client so that we can (hopefully) support ES 2
and ES 5 simultaneously.
However, I'm not sure how best to test it. There doesn't appear to
be a
straightforward way to test multiple ES versions in a single build, or
to test against ES 5 at all without significant changes to the tests.
I've commented in more detail at
https://issues.apache.org/jira/browse/STORM-2379, but in short,
the best
option I could find was this blog post by someone at Elastic using
Maven
Ant tasks:
http://david.pilato.fr/blog/2016/10/18/elasticsearch-real-integration-tests-updated-for-ga/.
Sree Vaddi then suggested using the docker-maven-plugin, which I think
would be much nicer, but I'm unsure whether we're able to require
Docker
for running the integration tests.
Suggestions welcome!
Kind regards,
Heather
On 06/03/17 14:56, Heather McCartney wrote:
> Hi Jungtaek,
>
> If you're sure we don't need to support 1.x then I'll just make
> changes to the existing storm-elasticsearch module. No need to
rewrite
> the whole thing :) As well as the transport layer, there will be
some
> changes to the request/response mapping, but I'll try to keep
these to
> a minimum.
>
> Kind regards,
> Heather
>
> On 06/03/17 02:14, Jungtaek Lim wrote:
>> Hi Heather,
>>
>> Sure I think it will be useful, and great news to hear that you'd
>> like to
>> contribute!
>>
>> I think we can even drop support 1.x (given that 1.x version is
EOL) for
>> next minor version in same module, like storm-kafka-client (0.9 for
>> 1.0.x,
>> 0.10 for 1.1.x). That means, you can reuse current
implementation and
>> just
>> replace transport layer with REST client.
>>
>> If you want to rewrite whole thing, please let us know so that
we can
>> start
>> discussion about how to maintain existing module.
>>
>> Thanks,
>> Jungtaek Lim (HeartSaVioR)
>>
>> 2017년 3월 6일 (월) 오전 5:10, Heather McCartney
>> <[email protected] <mailto:[email protected]>>님이
작성:
>>
>> Hi,
>>
>> I’m a Java developer looking to contribute to the Storm project.
>>
>> If we use the new Elasticsearch Java REST client, we should be
able to
>> support ES 2 and ES 5 simultaneously. I believe this is what
Elastic
>> recommend:
>>
https://www.elastic.co/blog/state-of-the-official-elasticsearch-java-clients
>>
>> <
>>
https://www.elastic.co/blog/state-of-the-official-elasticsearch-java-clients
>>
>>> .
>> We could create a new module using the REST client to support ES 2
>> and ES 5
>> and continue to offer the 1.6 module alongside. If you think it
would be
>> useful, I’d be happy to have a go at writing the new module.
>>
>> Kind regards,
>> Heather
>>
>>> On 3 Mar 2017, at 00:24, Jungtaek Lim <[email protected]
<mailto:[email protected]>> wrote:
>>>
>>> We don't have PR for supporting ES 5.0.
>>>
>>> There were some PR for supporting ES 2.0 but we didn't
establish how to
>>> remove duplicated codes between ES1 vs ES2 and PRs are stale now.
>>>
>>> I'm googling a bit, and 1.x is now end of life.
>>> https://www.elastic.co/support/eol
>>> Since 2.x is still supported, we might have to decide whether
we would
>>> support ES 2 and ES 5, or either ES 2 or ES 5. In any chance, your
>>> contribution is highly appreciated.
>>>
>>> Thanks,
>>> Jungtaek Lim (HeartSaVioR)
>>>
>>> 2017년 3월 3일 (금) 오전 3:39, Sree V
>>> <[email protected]
<mailto:[email protected]>.invalid>님이 작성:
>>>
>>> Hi,
>>> Would someone look into this, please ?
>>> https://jira.apache.org/jira/browse/STORM-2379
>>>
>>> I am curious to know, if any other pull requests and patches
addressing
>>> this ? Oris it open for anyone to fix it ?
>>> Thanking you, in advance.
>>> With Regards
>>> Sree
>>>
>>>
>
