[
https://issues.apache.org/jira/browse/FLUME-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229511#comment-13229511
]
[email protected] commented on FLUME-896:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4325/
-----------------------------------------------------------
(Updated 2012-03-14 19:04:52.742262)
Review request for Flume.
Changes
-------
Once again this not meant for commit, I personally wanted to give this a try
myself. Might be useful, might not. Regardless, I just wanted to upload the
latest patch before I move on to other work.
The latest changes keep the large memory consumption during replay in an off
heap buffer. If the amount of SequenceIDs to be replayed is greater than 64MB
-XX:MaxDirectMemorySize would need to be increased. The advantage of this
approach is that if a large amount of ram is needed for replaying it can be
explicitly allocated and deallocated during that period of time as opposed to
being in the heap always.
Summary
-------
Attached patch not meant for commit. Just posting here for easy review.
This addresses bug FLUME-896.
https://issues.apache.org/jira/browse/FLUME-896
Diffs (updated)
-----
flume-ng-channels/flume-file-channel/pom.xml ee2d20f
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java
a279453
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannelEvent.java
PRE-CREATION
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/wal/SequenceIDBuffer.java
PRE-CREATION
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/wal/WAL.java
PRE-CREATION
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/wal/WALDataFile.java
PRE-CREATION
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/wal/WALEntry.java
PRE-CREATION
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/wal/WALReplayResult.java
PRE-CREATION
flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannel.java
ab66998
flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/wal/TestSequenceIDBuffer.java
PRE-CREATION
flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/wal/TestWAL.java
PRE-CREATION
Diff: https://reviews.apache.org/r/4325/diff
Testing
-------
Thanks,
Brock
> Implement file write ahead log channel
> --------------------------------------
>
> Key: FLUME-896
> URL: https://issues.apache.org/jira/browse/FLUME-896
> Project: Flume
> Issue Type: New Feature
> Components: Channel
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: E. Sammer
> Fix For: v1.2.0
>
> Attachments: FLUME-896-1.patch, FLUME-896-4.patch
>
>
> Implement a channel that uses a regular file system and a write ahead log for
> durable event delivery.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
