Nice! Thanks Ferenc. Answered better than I could have done. :)
Sorry, I meant to send this last week but I just found it in my drafts.
Helmut, please let us know if you need more help with this.
Mike
On Tue, Mar 6, 2018 at 7:25 AM, Ferenc Szabo <fsz...@cloudera.com> wrote:
> the createJetty method of a Test class became final in the new versions of
> solr.
> the kite sdk test-jar has to be removed because it depends on a different
> incompatible version of solr
>
> <dependency>
> <groupId>org.kitesdk</groupId>
> <artifactId>kite-morphlines-solr-core</artifactId>
> <version>${kite.version}</version>
> <type>test-jar</type>
> <scope>test</scope>
> </dependency>
>
> TestEnvironment.java has to be removed as well because it depends on the
> incompatible dependency
>
> then we need this class:
> https://github.com/kite-sdk/kite/blob/master/kite-morphlines
> /kite-morphlines-solr-core/src/test/java/org/kitesdk/
> morphline/solr/TestEmbeddedSolrServer.java
> I believe it is ok to have a copy of this because it is part of the
> incompatible test dependency we just removed
>
> then we need a newer version of commons-compress:
> <commons-compress.version>1.10</commons-compress.version>
>
> from here you can fix the actual solr related test errors :)
>
>
>
> On Tue, Mar 6, 2018 at 11:56 AM, Wahrmann, Helmut <helmut.wahrm...@rsa.com
> >
> wrote:
>
> > Hi Ferenc,
> >
> > Thanks for offering help.
> > In agreement with Mike I want to have support for Solr 7.2.1 in the
> > morphline solr sink, so that we can easily upgrade the Elasticsearch
> sink.
> >
> > My updates are here: https://github.com/hwahrmann/
> > flume/tree/Upgrade_Morphline_Sink
> >
> > I changed the solr version to 7.2.1 and was able to compile the sink
> > withput any problems.
> > I can also compile the tests, but when running, I get multiple errors
> like
> > this:
> >
> > [INFO] Running org.apache.flume.sink.solr.morphline.
> > TestMorphlineInterceptor
> > [ERROR] Tests run: 66, Failures: 0, Errors: 66, Skipped: 0, Time elapsed:
> > 24.438 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> > TestMorphlineInterceptor
> > [ERROR] testIfDetectMimeTypeRouteToNorthPole(org.apache.flume.sink.
> > solr.morphline.TestMorphlineInterceptor) Time elapsed: 1.985 s <<<
> > ERROR!
> > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> tractSolrMorphlineZkTest
> > overrides final method createJetty.(Ljava/io/File;
> > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> JettySolrRunner;
> > at org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > build(TestMorphlineInterceptor.java:151)
> > at org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > testIfDetectMimeTypeRouteToNorthPole(TestMorphlineInterceptor.java:139)
> >
> > [ERROR] testGrokIfNotMatchDropEventRetain(org.apache.flume.sink.
> > solr.morphline.TestMorphlineInterceptor) Time elapsed: 0.363 s <<<
> > ERROR!
> > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> tractSolrMorphlineZkTest
> > overrides final method createJetty.(Ljava/io/File;
> > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> JettySolrRunner;
> > at org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > build(TestMorphlineInterceptor.java:151)
> > at org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > testGrokIfNotMatchDropEventRetain(TestMorphlineInterceptor.java:83)
> >
> >
> > The all have problems with createJetty. So it seems that I maybe need a
> > different version of jetty or something like that.
> > And for that I have too less knowledge about maven.
> >
> > thx,
> >
> > Helmut
> >
> > -----Original Message-----
> > From: Ferenc Szabo [mailto:fsz...@cloudera.com]
> > Sent: Dienstag, 6. März 2018 11:04
> > To: dev@flume.apache.org
> > Subject: Re: Merge of patch in Flume-3021?
> >
> > Hi Helmut,
> >
> > let me know what can I help You with.
> > share your current code on a github fork and describe the issue. I will
> > see what can we do to solve it.
> >
> >
> > On Tue, Mar 6, 2018 at 10:50 AM, Wahrmann, Helmut <
> helmut.wahrm...@rsa.com
> > >
> > wrote:
> >
> > > Hi Mike,
> > >
> > > I am stuck with Solr.
> > > The morphline-solr sink compiles without any problems, but I am
> > > struggling with the tests.
> > > Seems I need to exclude some stuff from maven, but my knowledge about
> > > maven is not good enough to figure out what to do.
> > > Anyone able to help?
> > >
> > > regards,
> > >
> > > Helmut
> > >
> > > -----Original Message-----
> > > From: Mike Percy [mailto:mpe...@apache.org]
> > > Sent: Mittwoch, 14. Februar 2018 19:19
> > > To: dev@flume.apache.org
> > > Subject: Re: Merge of patch in Flume-3021?
> > >
> > > Hi Helmut,
> > > As long as the integration tests still pass and the packaging issues
> > > are not exacerbated, I don't see why we couldn't merge an upgrade
> > > patch, barring any serious concerns with the patch.
> > >
> > > Mike
> > >
> > > On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut
> > > <helmut.wahrm...@rsa.com
> > > >
> > > wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > I won't have a problem upgrading the Solr sink to the latest version.
> > > > I am missing test environment however.
> > > > So while it may build correctly and all integration tests work, I
> > > > have no real environment to test with.
> > > >
> > > > best regards,
> > > > Helmut
> > > >
> > > > -----Original Message-----
> > > > From: Mike Percy [mailto:mpe...@apache.org]
> > > > Sent: Mittwoch, 14. Februar 2018 00:38
> > > > To: dev@flume.apache.org
> > > > Subject: Re: Merge of patch in Flume-3021?
> > > >
> > > > OK. In the pull request, it would be nice if whoever submits or
> > > > merges it mentions all of the contributors to the patch in the commit
> > message.
> > > >
> > > > I asked Wolfgang H. about the SolrServer thing and this is what he
> > > > told
> > > me:
> > > >
> > > > Hi Mike, the class has been renamed to "SolrClient" (which
> > > > unfortunately
> > > > > breaks compat). It's just a class rename. The functionality is the
> > > > > same as before. It was called SolrServer in Solr4 because it was a
> > > > > client proxy that sends RPCs to a Solr server, but calling it
> > > > > SolrClient is more straightforward to understand, hence the
> > > > > community decided to rename the class.
> > > > > It's possible to spawn an embedded Solr server, for example for
> > > > > testing purposes, via class EmbeddedSolrServer (a class that
> > > > > retains the same name in Solr7 and Solr4), which extends the
> > SolrClient class.
> > > >
> > > >
> > > > Hope this helps,
> > > > Mike
> > > >
> > > > On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut
> > > > <helmut.wahrm...@rsa.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Mike,
> > > > >
> > > > > Thanks for the response. Would be cool if we get that sorted out.
> > > > >
> > > > > I've asked Yonghao Zou to submit the Pull Request, since he did
> > > > > most of the work and should get the credit.
> > > > > He'll do so after the Chinese New Year's Eve.
> > > > >
> > > > > I will then issue a Pull request for the new ES Rest client, which
> > > > > is dependent on the above work.
> > > > >
> > > > > best regards,
> > > > > Helmut
> > > > >
> > > > > -----Original Message-----
> > > > > From: Mike Percy [mailto:mpe...@apache.org]
> > > > > Sent: Dienstag, 13. Februar 2018 04:30
> > > > > To: dev@flume.apache.org
> > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > >
> > > > > Hi Helmut,
> > > > > I see that I neglected to follow up on the other thread on this
> > > > > topic after your reply about SolrServer missing from the solrj jar.
> > > > > Let me ask around w/ some folks I know that work on Solr and see
> > > > > if there is any way to retain the SolrServer for our tests after
> > > > > upgrading to the
> > > > new version.
> > > > >
> > > > > Thank you very much for working on upgrading Solr. Would you mind
> > > > > submitting a pull request with your (apparently work-in-progress)
> > > > > patch to upgrade both Solr and ES?
> > > > >
> > > > > To reply to your email in this thread, the JAR packaging situation
> > > > > is largely the same after merging FLUME-2957 so unfortunately most
> > > > > of what I noted in my reply in the other thread (
> > > > > https://s.apache.org/GqcX ) still holds.
> > > > >
> > > > > I hope that we can upgrade the Solr dependencies as part of the
> > > > > same commit as the ES dependencies to avoid worrying about which
> > > > > lucene jar is first in the classpath, and ensure we are not adding
> > > > > any additional dependency conflicts to mvn dependency:tree.
> > > > >
> > > > > Regards,
> > > > > Mike
> > > > >
> > > > > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut <
> > > > > helmut.wahrm...@rsa.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > now that the blocker for FLUME-3021 is removed by committing
> > > > > > FLUME-2957, can we get the patch from 3021 merged to trunk?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Helmut
> > > > > >
> > > > >
> > > >
> > >
> >
>
