[jira] [Created] (FLUME-1844) HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Brock Noland (JIRA)
Brock Noland created FLUME-1844: --- Summary: HDFSEventSink should have option to use RawLocalFileSystem Key: FLUME-1844 URL: https://issues.apache.org/jira/browse/FLUME-1844 Project: Flume Issue

[jira] [Updated] (FLUME-1844) HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Brock Noland (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated FLUME-1844: Description: Due to HADOOP-7844, we should have a way to use RawLocalFileSystem, which does

[jira] [Assigned] (FLUME-1731) SpoolableDirectorySource should have configurable support for deleting files it has already completed instead of renaming

2013-01-15 Thread Mike Percy (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Percy reassigned FLUME-1731: - Assignee: Mike Percy SpoolableDirectorySource should have configurable support for deleting

[jira] [Updated] (FLUME-1731) SpoolableDirectorySource should have configurable support for deleting files it has already completed instead of renaming

2013-01-15 Thread Mike Percy (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Percy updated FLUME-1731: -- Attachment: FLUME-1731-1.patch SpoolableDirectorySource should have configurable support for

[jira] [Created] (FLUME-1845) Document plugin.d directory structure

2013-01-15 Thread Brock Noland (JIRA)
Brock Noland created FLUME-1845: --- Summary: Document plugin.d directory structure Key: FLUME-1845 URL: https://issues.apache.org/jira/browse/FLUME-1845 Project: Flume Issue Type: Bug

Re: New blog post on Flume performance tuning

2013-01-15 Thread Mohit Anchlia
We have memoryChannel capacity set to 1 and transactionSize set to 500. This gives us good performance. I am trying to understand what's the downside of using this value. Is it ok to have it so large? webanalytics.channels.memoryChannel.capacity = 1

[jira] [Commented] (FLUME-1844) HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Brock Noland (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13554147#comment-13554147 ] Brock Noland commented on FLUME-1844: - Shoot, it doesn't look like that will work,

[jira] [Commented] (FLUME-1844) HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Brock Noland (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13554238#comment-13554238 ] Brock Noland commented on FLUME-1844: - Here is the exception. {noformat}

[jira] [Commented] (FLUME-1844) HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Connor Woodson (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13554298#comment-13554298 ] Connor Woodson commented on FLUME-1844: --- What if you just use the file:/// tag but

[jira] [Commented] (FLUME-1844) HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Brock Noland (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13554370#comment-13554370 ] Brock Noland commented on FLUME-1844: - Thanks Connor! Yeah I haven't tried that.

Review Request: FLUME-1844: HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Brock Noland
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/8964/ --- Review request for Flume. Description --- Add's flag to HDFSEventSink

[jira] [Updated] (FLUME-1844) HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Brock Noland (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated FLUME-1844: Issue Type: Bug (was: Improvement) HDFSEventSink should have option to use

Re: Review Request: FLUME-1731: SpoolableDirectorySource should have configurable support for deleting files it has already completed instead of renaming

2013-01-15 Thread Brock Noland
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/8957/#review15355 ---

[jira] [Created] (FLUME-1847) NPE in SourceConfiguration

2013-01-15 Thread Brock Noland (JIRA)
Brock Noland created FLUME-1847: --- Summary: NPE in SourceConfiguration Key: FLUME-1847 URL: https://issues.apache.org/jira/browse/FLUME-1847 Project: Flume Issue Type: Improvement

Re: Review Request: FLUME-1844: HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Hari Shreedharan
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/8964/#review15361 --- Brock, Looks good in general. One question I have is if we could

Re: Review Request: FLUME-1844: HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Brock Noland
On Jan. 16, 2013, 12:21 a.m., Hari Shreedharan wrote: Brock, Looks good in general. One question I have is if we could just do without a flume flag. Rather check the Configuration object to see if it is set to write to local file system. Does that make sense? I thought about that

Re: Review Request: FLUME-1844: HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Hari Shreedharan
On Jan. 16, 2013, 12:21 a.m., Hari Shreedharan wrote: Brock, Looks good in general. One question I have is if we could just do without a flume flag. Rather check the Configuration object to see if it is set to write to local file system. Does that make sense? Brock Noland wrote:

[jira] [Resolved] (FLUME-1844) HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Shreedharan resolved FLUME-1844. - Resolution: Fixed Fix Version/s: v1.4.0 Assignee: Brock Noland Patch

[jira] [Updated] (FLUME-1848) HDFSDataStream logger is actually for a sequence file

2013-01-15 Thread Brock Noland (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated FLUME-1848: Description: FLUME-1844 added the following to HDFSDataStream: + private static final Logger

Review Request: FLUME-1842.AsyncHBaseSink timeout is not calculated correctly

2013-01-15 Thread Hari Shreedharan
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/8968/ --- Review request for Flume. Description --- Made changes to calculate

[jira] [Updated] (FLUME-1842) AsyncHBaseSink timeout is not calculated correctly

2013-01-15 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Shreedharan updated FLUME-1842: Attachment: FLUME-1842-1.patch AsyncHBaseSink timeout is not calculated correctly

[jira] [Commented] (FLUME-1844) HDFSEventSink should have option to use RawLocalFileSystem

2013-01-15 Thread Juhani Connolly (JIRA)
[ https://issues.apache.org/jira/browse/FLUME-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13554690#comment-13554690 ] Juhani Connolly commented on FLUME-1844: Thanks for this, I'll give it a spin on