date:20161127

[GitHub] flume pull request #95: update

2016-11-27 Thread Popo-lth

Github user Popo-lth closed the pull request at:

https://github.com/apache/flume/pull/95


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flume pull request #95: update

2016-11-27 Thread Popo-lth

GitHub user Popo-lth opened a pull request:

https://github.com/apache/flume/pull/95

update



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Popo-lth/flume trunk

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flume/pull/95.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #95


commit 71d6ab307c6234e47c0e23084a296b7beccf713f
Author: çªäº 
Date:   2016-10-21T07:01:17Z

Merge pull request #1 from apache/trunk

Fix startup order for ExecSource

commit e25eb64fff95ffac47581109fec8831a318802e5
Author: Bessenyei BalÃ¡zs DonÃ¡t 
Date:   2016-10-22T19:39:54Z

Add .travis.yml

ASF infra has enabled Travis for Apache Flume.

This change adds .travis.yml, because the option "Build only if .travis.yml 
is present" is turned on.

With this commit, builds will run with -DskipTests.

commit 410d4ae772d54ceaa6aa50c78294ed810add7cdd
Author: Bessenyei BalÃ¡zs DonÃ¡t 
Date:   2016-10-22T20:16:22Z

Fix .travis.yml

License was missing from .travis.yml, so RAT checks failed.

This commit adds the license header to .travis.yml.
After this change RAT passes again.

This closes #74

commit b1b61f72fca6b66bd79a05122ab20a588e8cb85f
Author: Bessenyei BalÃ¡zs DonÃ¡t 
Date:   2016-10-24T06:44:38Z

Set MAVEN_OPTS for Travis-CI

Travis builds occasionally fail with Java heap space error.

This commit sets higher limits to fix the issue mentioned above.

Reviewers: Lior Zeno, Mike Percy

This closes #75

commit 7477759d166aa4dd940cbc8fbe7ed67c30545441
Author: Bessenyei BalÃ¡zs DonÃ¡t 
Date:   2016-10-24T12:27:08Z

FLUME-3002. Fix tests in TestBucketWriter

Some tests are flaky in TestBucketWriter.

This commit fixes the flakiness mentioned before by
adding a new constructor with an extra Clock parameter.

Reviewers: Attila Simon, Denes Arvay

commit ea685d61bed28a4a93805d4d1fb10fd46944ced8
Author: Bessenyei BalÃ¡zs DonÃ¡t 
Date:   2016-10-24T14:53:29Z

Add Developer Section / How to Release from cwiki.apache.org

Reviewers: Mike Percy

This closes #77

commit 750dec5986acc5d48020aed3742101b84d4dc80c
Author: Bessenyei BalÃ¡zs DonÃ¡t 
Date:   2016-10-25T09:11:02Z

Add e-mail templates to HowToRelease.md

Reviewers: Attila Simon

This closes #78

commit 4753743bf5479e234c3c44f6c57ab3eea00eea82
Author: Bessenyei BalÃ¡zs DonÃ¡t 
Date:   2016-10-25T11:36:33Z

Add Developer Section / How to Contribute from cwiki.apache.org

Reviewers: Attila Simon

This closes #79

commit 4e041565b45e243107c2bd018662fe2b7f5097b1
Author: Bessenyei BalÃ¡zs DonÃ¡t 
Date:   2016-10-25T13:13:28Z

Add Developer Section / Developers Quick Hack Sheet from cwiki.apache.org

Reviewers: Attila Simon

This closes #81

commit c111aad2882f8e39c04ac71625e0c1b68861bf1f
Author: Attila Simon 
Date:   2016-10-27T11:31:40Z

FLUME-2997. Fix flaky test in SpillableMemoryChannel

When the mock sinks are slower than sources, 
testParallelSingleSourceAndSink sometimes fails.
In such situations the channel can get full, thus uncaught 
ChannelFullException breaks the test.

Since testCapacityWithOverflow was designed to cover such a scenario, we
can safely fix the test by increasing the channel capacity to make sure it 
won't get full.

This commit does the previously mentioned change.

Reviewers: Bessenyei BalÃ¡zs DonÃ¡t

(Attila Simon via Bessenyei BalÃ¡zs DonÃ¡t)

commit bf616eb313e726cf36e08d4fe466f2ce41d79e39
Author: Bessenyei BalÃ¡zs DonÃ¡t 
Date:   2016-10-28T10:36:08Z

Add Developer Section / How to Commit from cwiki.apache.org

Reviewers: Attila Simon

This closes #80

commit aac619a2bd4102f2fa72ce069acbf18ca28eb4aa
Author: Tristan Stevens 
Date:   2016-10-28T12:24:02Z

FLUME-2857. Make Kafka Source/Channel/Sink restore default values when live 
updating config

This commit changes Kafka Channel, Sink and Source to fix an error where
sub-configurations aren't tolerant of the configure() method being called 
more
than once (as happens when a Live Config Update happens).

Reviewers: Denes Arvay, Attila Simon, Bessenyei BalÃ¡zs DonÃ¡t

(Tristan Stevens via Bessenyei BalÃ¡zs DonÃ¡t)

commit 7e0609f975de12f75db9330371ebc0fc8ce97dad
Author: Denes Arvay 
Date:   2016-10-28T15:38:33Z

FLUME-2812. Fix semaphore leak causing java.lang.Error: Maximum permit 
count exceeded in MemoryChannel

[jira] [Commented] (FLUME-3032) taildir source sleeps frequently.

2016-11-27 Thread Attila Simon (JIRA)


[ 
https://issues.apache.org/jira/browse/FLUME-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15699547#comment-15699547
 ] 

Attila Simon commented on FLUME-3032:
-

Nice finding and the change also looks good to me. Would you consider adding a 
junit test to cover this change? 

> taildir source sleeps frequently.
> -
>
> Key: FLUME-3032
> URL: https://issues.apache.org/jira/browse/FLUME-3032
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.7.0
> Environment: CentOS Linux release 7.2.1511 (Core) 
> java version "1.7.0_80"
>Reporter: Liu Tianhao
>  Labels: newbie
> Attachments: FLUME-3032.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Test configuration.
> source - taildir
> interceptor -  The custom interceptor drops some events
> channel - anyone
> sink - none
> I found that taildir source sleeps frequently.
> The tailFileProcess() function in TaildirSource.java break the loop by 
> (events.size() < batchSize), but interceptor may change events.size().
> I think the events.size() should be used before interceptor processing. 
> Avoid unnecessary sleep.
> {code:title=TaildirSource.java|borderStyle=solid}
> private void tailFileProcess(TailFile tf, boolean backoffWithoutNL)
> throws IOException, InterruptedException {
> long receivedSize = 0;
> while (true) {
> reader.setCurrentFile(tf);
> List events = reader.readEvents(batchSize, 
> backoffWithoutNL);
> if (events.isEmpty()) {
> break;
> }
> receivedSize = events.size();
> sourceCounter.addToEventReceivedCount(receivedSize);
> sourceCounter.incrementAppendBatchReceivedCount();
> try {
> getChannelProcessor().processEventBatch(events);
> reader.commit();
> } catch (ChannelException ex) {
> logger.warn("The channel is full or unexpected failure. "
> + "The source will try again after " + retryInterval
> + " ms");
> TimeUnit.MILLISECONDS.sleep(retryInterval);
> retryInterval = retryInterval << 1;
> retryInterval = Math.min(retryInterval, maxRetryInterval);
> continue;
> }
> retryInterval = 1000;
> sourceCounter.addToEventAcceptedCount(events.size());
> sourceCounter.incrementAppendBatchAcceptedCount();
> if (receivedSize < batchSize) {
> break;
> }
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] flume pull request #95: update

[GitHub] flume pull request #95: update

[jira] [Commented] (FLUME-3032) taildir source sleeps frequently.

3 matches

Site Navigation

Mail list logo

Footer information