[GitHub] flume pull request #204: FLUME-2605 - Remove usage of CounterGroup from org....

[tmgstevens]

GitHub user tmgstevens opened a pull request:

https://github.com/apache/flume/pull/204

FLUME-2605 - Remove usage of CounterGroup from 
org.apache.flume.source.StressSource

Rework of original patch by Bartosz Ługowski

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tmgstevens/flume FLUME-2605

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flume/pull/204.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #204


commit 6fe2b001f38a02effe20185aee855bb80c01c85a
Author: Tristan Stevens 
Date:   2018-03-19T21:44:58Z

Rework of original patch for FLUME-2605




---

[GitHub] flume pull request #203: FLUME-3227 Add Rate Limiter to stresssource

[tmgstevens]

GitHub user tmgstevens opened a pull request:

https://github.com/apache/flume/pull/203

FLUME-3227 Add Rate Limiter to stresssource

This uses a shaded version of the guava rate limiter and includes unit 
tests.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tmgstevens/flume FLUME-3227

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flume/pull/203.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #203


commit 51e81aa3310639f988258a1cdfb7a9ec92fb2d61
Author: Tristan Stevens 
Date:   2018-03-19T16:13:20Z

Add Rate Limiter to stresssource




---

RE: Merge of patch in Flume-3021?

[Wahrmann, Helmut]

Hi Mike,

With the help of Ferenc, I got rid of the initial errors.

Only one is remaining now:

[INFO] ---
[INFO]  T E S T S
[INFO] ---
[INFO] Running org.apache.flume.sink.solr.morphline.TestBlobDeserializer
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.126 s 
- in org.apache.flume.sink.solr.morphline.TestBlobDeserializer
[INFO] Running org.apache.flume.sink.solr.morphline.TestBlobHandler
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.125 s 
- in org.apache.flume.sink.solr.morphline.TestBlobHandler
[INFO] Running org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.724 s 
- in org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor
[INFO] Running org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.797 s 
<<< FAILURE! - in org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
[ERROR] org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink  Time 
elapsed: 1.797 s  <<< ERROR!
java.util.IllformedLocaleException: Invalid subtag: en_us [at index 0]

[INFO] Running org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.391 s 
- in org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   TestMorphlineSolrSink>LuceneTestCase.localeForLanguageTag:1588 ╗ 
IllformedLocale
[INFO]
[ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0 [INFO] [INFO] 

[INFO] BUILD FAILURE


The LuceneTestCase is extended by SolrTestCaseJ4.
No idea, what I could do against this.

best regards,

Helmut

-Original Message-
From: Mike Percy [mailto:mpe...@apache.org] 
Sent: Montag, 19. März 2018 05:12
To: dev@flume.apache.org
Subject: Re: Merge of patch in Flume-3021?

Nice! Thanks Ferenc. Answered better than I could have done. :)

Sorry, I meant to send this last week but I just found it in my drafts.

Helmut, please let us know if you need more help with this.

Mike

On Tue, Mar 6, 2018 at 7:25 AM, Ferenc Szabo  wrote:

> the createJetty method of a Test class became final in the new 
> versions of solr.
> the kite sdk test-jar has to be removed because it depends on a 
> different incompatible version of solr
>
> 
>   org.kitesdk
>   kite-morphlines-solr-core
>   ${kite.version}
>   test-jar
>   test
> 
>
> TestEnvironment.java has to be removed as well because it depends on 
> the incompatible dependency
>
> then we need this class:
> https://github.com/kite-sdk/kite/blob/master/kite-morphlines
> /kite-morphlines-solr-core/src/test/java/org/kitesdk/
> morphline/solr/TestEmbeddedSolrServer.java
> I believe it is ok to have a copy of this because it is part of the 
> incompatible test dependency we just removed
>
> then we need a newer version of commons-compress:
> 1.10
>
> from here you can fix the actual solr related test errors :)
>
>
>
> On Tue, Mar 6, 2018 at 11:56 AM, Wahrmann, Helmut 
>  >
> wrote:
>
> > Hi Ferenc,
> >
> > Thanks for offering help.
> > In agreement with Mike I want to have support for Solr 7.2.1 in the 
> > morphline solr sink, so that we can easily upgrade the Elasticsearch
> sink.
> >
> > My updates are here: https://github.com/hwahrmann/ 
> > flume/tree/Upgrade_Morphline_Sink
> >
> > I changed the solr version to 7.2.1 and was able to compile the sink 
> > withput any problems.
> > I can also compile the tests, but when running, I get multiple 
> > errors
> like
> > this:
> >
> > [INFO] Running org.apache.flume.sink.solr.morphline.
> > TestMorphlineInterceptor
> > [ERROR] Tests run: 66, Failures: 0, Errors: 66, Skipped: 0, Time elapsed:
> > 24.438 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> > TestMorphlineInterceptor
> > [ERROR] testIfDetectMimeTypeRouteToNorthPole(org.apache.flume.sink.
> > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 1.985 s  <<< 
> > ERROR!
> > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> tractSolrMorphlineZkTest
> > overrides final method createJetty.(Ljava/io/File; 
> > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> JettySolrRunner;
> > at 
> > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > build(TestMorphlineInterceptor.java:151)
> > at 
> > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > testIfDetectMimeTypeRouteToNorthPole(TestMorphlineInterceptor.java:1
> > 39)
> >
> > [ERROR] testGrokIfNotMatchDropEventRetain(org.apache.flume.sink.
> > solr.morphline.TestMorphlineInterceptor)  Time elapsed: