Re: Flume-1.4.0 doesn't suport HBaseSink/AsyncHBaseSink?

2013-12-16 Thread Bing Li
Thanks, Hari I tried an HBaseSink scenario on a real cluster, it could work well now. But it failed when ran TestHBaseSink with hbase0.96, is it an known issue as well? 2013/12/16 Hari Shreedharan hshreedha...@cloudera.com You should be able to. I have not seen any issue of HBase

[jira] [Updated] (FLUME-2243) AvroSource to use TransceiverThreadFactory for Thread naming while initializing NioServerSocketChannelFactory

2013-12-16 Thread Ashish Paliwal (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2243: -- Attachment: FLUME-2243-1.patch Rebased the patch. Removed TransreceiverThreadFactory class and

[jira] [Updated] (FLUME-1521) Document the StressSource

2013-12-16 Thread Ashish Paliwal (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-1521: -- Attachment: (was: FLUME-1521-0.patch) Document the StressSource

[jira] [Updated] (FLUME-1521) Document the StressSource

2013-12-16 Thread Ashish Paliwal (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-1521: -- Attachment: FLUME-1521-0.patch Document the StressSource -

[jira] [Updated] (FLUME-1521) Document the StressSource

2013-12-16 Thread Ashish Paliwal (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-1521: -- Attachment: FLUME-1521-1.patch Document the StressSource -

Review Request 16292: Patch for FLUME-1521 - Document the StressSource

2013-12-16 Thread Ashish Paliwal
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16292/ --- Review request for Flume. Bugs: FLUME-1521

[jira] [Commented] (FLUME-1521) Document the StressSource

2013-12-16 Thread Ashish Paliwal (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849003#comment-13849003 ] Ashish Paliwal commented on FLUME-1521: --- Updated the User Guide. We can also update

[jira] [Updated] (FLUME-2216) HDFS file rotations get disabled once maxUnderReplicationCount is reached

2013-12-16 Thread Gopinathan A (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopinathan A updated FLUME-2216: Attachment: FLUME-2216-0.patch Please review this patch. HDFS file rotations get disabled once

Review Request 16294: FLUME-2216 HDFS file rotations get disabled once maxUnderReplicationCount is reached

2013-12-16 Thread Gopinathan Andavar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16294/ --- Review request for Flume. Bugs: FLUME-2216

Re: Review Request 16294: FLUME-2216 HDFS file rotations get disabled once maxUnderReplicationCount is reached

2013-12-16 Thread Gopinathan Andavar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16294/ --- (Updated Dec. 16, 2013, 2:57 p.m.) Review request for Flume. Changes ---

[jira] [Updated] (FLUME-2216) HDFS file rotations get disabled once maxUnderReplicationCount is reached

2013-12-16 Thread Gopinathan A (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopinathan A updated FLUME-2216: Attachment: FLUME-2216-1.patch corrected the testcase failure HDFS file rotations get disabled

[jira] [Created] (FLUME-2273) ElasticSearchSink: Add handling for header substitution in indexName

2013-12-16 Thread Paul Merry (JIRA)
Paul Merry created FLUME-2273: - Summary: ElasticSearchSink: Add handling for header substitution in indexName Key: FLUME-2273 URL: https://issues.apache.org/jira/browse/FLUME-2273 Project: Flume

[jira] [Updated] (FLUME-2273) ElasticSearchSink: Add handling for header substitution in indexName

2013-12-16 Thread Paul Merry (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Merry updated FLUME-2273: -- Attachment: FLUME-2273.patch ElasticSearchSink: Add handling for header substitution in indexName

[jira] [Commented] (FLUME-2273) ElasticSearchSink: Add handling for header substitution in indexName

2013-12-16 Thread Paul Merry (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849227#comment-13849227 ] Paul Merry commented on FLUME-2273: --- Patch attached, includes tests and doc update. This

[jira] [Updated] (FLUME-1856) HDFS sink idleTimeout does not close removed BucketWriters

2013-12-16 Thread Gopinathan A (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopinathan A updated FLUME-1856: Attachment: FLUME-1856-0.patch HDFS sink idleTimeout does not close removed BucketWriters

Re: [DISCUSS] Feature bloat and contrib module

2013-12-16 Thread Ashish
Quick question before I put down my thoughts. What does core Flume represents? Is it the current Flume trunk structure or just few modules? On Sat, Dec 14, 2013 at 5:51 AM, Hari Shreedharan hshreedha...@cloudera.com wrote: Hi all Over the past few weeks, there were some discussions about

[jira] [Commented] (FLUME-2181) Optionally disable File Channel fsyncs

2013-12-16 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849633#comment-13849633 ] Hari Shreedharan commented on FLUME-2181: - [~brocknoland] - What do you think of

[jira] [Created] (FLUME-2274) Feature bloat and contrib module

2013-12-16 Thread Israel Ekpo (JIRA)
Israel Ekpo created FLUME-2274: -- Summary: Feature bloat and contrib module Key: FLUME-2274 URL: https://issues.apache.org/jira/browse/FLUME-2274 Project: Flume Issue Type: Brainstorming

[jira] [Commented] (FLUME-2274) Feature bloat and contrib module

2013-12-16 Thread Israel Ekpo (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849678#comment-13849678 ] Israel Ekpo commented on FLUME-2274: Initial post by [~hshreedharan] Hi all Over the

Re: [DISCUSS] Feature bloat and contrib module

2013-12-16 Thread Israel Ekpo
I have created a JIRA Brainstorming task to track this. I think it will be very helpful for us to collect our thoughts on the JIRA ticket for future references. I think this is a very important topic. https://issues.apache.org/jira/browse/FLUME-2274 On Mon, Dec 16, 2013 at 12:51 PM, Ashish

[jira] [Commented] (FLUME-2274) Feature bloat and contrib module

2013-12-16 Thread Israel Ekpo (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849685#comment-13849685 ] Israel Ekpo commented on FLUME-2274: Follow up post by [~jarcec] Hi Hari, thank you

[jira] [Commented] (FLUME-2274) Feature bloat and contrib module

2013-12-16 Thread Israel Ekpo (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849688#comment-13849688 ] Israel Ekpo commented on FLUME-2274: Follow up post by [~paliwalashish] Quick

[jira] [Updated] (FLUME-2274) Feature bloat and contrib module

2013-12-16 Thread Israel Ekpo (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Israel Ekpo updated FLUME-2274: --- Assignee: Hari Shreedharan Feature bloat and contrib module

[jira] [Commented] (FLUME-1227) Introduce some sort of SpillableChannel

2013-12-16 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849852#comment-13849852 ] Hari Shreedharan commented on FLUME-1227: - Hey [~roshan_naik] - Any updates here?

[jira] [Commented] (FLUME-2056) Allow SpoolDir to pass just the filename that is the source of an event

2013-12-16 Thread Mike Percy (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849941#comment-13849941 ] Mike Percy commented on FLUME-2056: --- +1 LGTM Allow SpoolDir to pass just the filename

[jira] [Commented] (FLUME-2181) Optionally disable File Channel fsyncs

2013-12-16 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849963#comment-13849963 ] Hari Shreedharan commented on FLUME-2181: - If during replay we hit a bad event,

Re: [DISCUSS] Feature bloat and contrib module

2013-12-16 Thread Mike Percy
I have created a JIRA Brainstorming task to track this. Hmm, I think there is a risk of losing this discussion in the flood of JIRA traffic due to email filters. So I'm going to respond here on this thread. If you want to reference this thread in the future, you can use this URL:

[jira] [Commented] (FLUME-2181) Optionally disable File Channel fsyncs

2013-12-16 Thread Brock Noland (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850079#comment-13850079 ] Brock Noland commented on FLUME-2181: - Ideally I'd only like to implement this if we

[jira] [Commented] (FLUME-2056) Allow SpoolDir to pass just the filename that is the source of an event

2013-12-16 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850093#comment-13850093 ] Hudson commented on FLUME-2056: --- FAILURE: Integrated in flume-trunk #530 (See

Build failed in Jenkins: flume-trunk #530

2013-12-16 Thread Apache Jenkins Server
See https://builds.apache.org/job/flume-trunk/530/changes Changes: [mpercy] FLUME-2056. Allow SpoolDir to pass just the filename that is the source of an event -- [...truncated 1959 lines...] [INFO] Reactor Summary: [INFO] [INFO] Apache Flume

[jira] [Assigned] (FLUME-2089) ElasticsearchSink blocks and raises exceptions when event body has unexpected encoding

2013-12-16 Thread Ashish Paliwal (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-2089: - Assignee: Ashish Paliwal ElasticsearchSink blocks and raises exceptions when event body

[jira] [Commented] (FLUME-2254) Improve log when there is a failure in a BulkRequest in ES Sink and avoid stuck failures in the channel

2013-12-16 Thread Ashish Paliwal (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850176#comment-13850176 ] Ashish Paliwal commented on FLUME-2254: --- This looks like to an exactly once

Re: [DISCUSS] Feature bloat and contrib module

2013-12-16 Thread Bruno Mahé
I will probably end up repeating the very same thing than the other discussion. Summarizing my suggestions: * Commiters are not the sole developers. There is no reason for commiters to take all these responsibilities on their shoulders. Also developer != commiter. * Easy IN, Easy OUT. If no

[jira] [Updated] (FLUME-2248) Redis sink

2013-12-16 Thread JIRA
[ https://issues.apache.org/jira/browse/FLUME-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Mahé updated FLUME-2248: -- Attachment: (was: FLUME-2248-1.patch) Redis sink -- Key: FLUME-2248

[jira] [Comment Edited] (FLUME-2248) Redis sink

2013-12-16 Thread JIRA
[ https://issues.apache.org/jira/browse/FLUME-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850208#comment-13850208 ] Bruno Mahé edited comment on FLUME-2248 at 12/17/13 7:40 AM: -

[jira] [Commented] (FLUME-2248) Redis sink

2013-12-16 Thread JIRA
[ https://issues.apache.org/jira/browse/FLUME-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850208#comment-13850208 ] Bruno Mahé commented on FLUME-2248: --- Since then, I fixed and added a few things (added a

[jira] [Commented] (FLUME-2181) Optionally disable File Channel fsyncs

2013-12-16 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850211#comment-13850211 ] Hari Shreedharan commented on FLUME-2181: - I am planning to start doing the new