-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3704/
-----------------------------------------------------------
(Updated 2012-02-21 06:51:00.751898)
Review request for Flume.
Changes
-------
Rebased patch and tested with following:
1005 git pull origin flume-728
1009 git diff 44850b9989a4c13716cbc6 > FLUME-936-4.patch
1010 git checkout origin/flume-728 -b temp
1012 git apply FLUME-936-4.patch
1014 mvn clean
1015 mvn test
Results are fine
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Flume ...................................... SUCCESS [4.458s]
[INFO] Flume NG Core ..................................... SUCCESS [1:47.853s]
[INFO] Flume NG Sinks .................................... SUCCESS [0.096s]
[INFO] Flume NG HDFS Sink ................................ SUCCESS [14.040s]
[INFO] Flume NG IRC Sink ................................. SUCCESS [0.890s]
[INFO] Flume NG Channels ................................. SUCCESS [0.065s]
[INFO] Flume NG JDBC channel ............................. SUCCESS [28.434s]
[INFO] Flume NG Node ..................................... SUCCESS [31.250s]
[INFO] Flume NG file-based channel ....................... SUCCESS [1.793s]
[INFO] Flume NG distribution ............................. SUCCESS [0.754s]
[INFO] Flume legacy Sources .............................. SUCCESS [0.134s]
[INFO] Flume legacy Thrift Source ........................ SUCCESS [3.043s]
[INFO] Flume legacy Avro source .......................... SUCCESS [2.876s]
[INFO] Flume NG Clients .................................. SUCCESS [0.062s]
[INFO] Flume NG Log4j Appender ........................... SUCCESS [3.732s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:19.980s
[INFO] Finished at: Tue Feb 21 15:49:08 JST 2012
[INFO] Final Memory: 83M/229M
[INFO] ------------------------------------------------------------------------
Summary
-------
This is an initial go at fixing the threading issues with memory channel.
It uses the preliminary work on FLUME-935 and I have included the code from
that.
The tagging of the events became unnecessary so I dropped that. One thing that
concerns me slightly is how to deal with not having enough space in the queue
to rollback failed takes. One method would be to keep a minimum buffer of
transactionCapacity. Another would be to implement the queue of queues as
suggested in FLUME-889
Anyway, just putting up this early version to see what people think
This addresses bug FLUME-936.
https://issues.apache.org/jira/browse/FLUME-936
Diffs (updated)
-----
flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java
6a17f06
flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannel.java
b44030e
flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannelConcurrency.java
PRE-CREATION
flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannelTransaction.java
d18045b
flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java
3014368
flume-ng-node/src/test/java/org/apache/flume/source/TestNetcatSource.java
9e465e1
Diff: https://reviews.apache.org/r/3704/diff
Testing
-------
The original tests pass, though I had to take out the state checks because of
the changes to semantics from the flume-935 code. I also had to add a
transaction.close statement where semantics were not properly being followed
I have to retrofit my new concurrency test since without the tagged events it
cannot fail without checking that the content is correct. I'll put that up
asap, just wanted to get some eyes on this before I head out.
Thanks,
Juhani
