-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23588/
-----------------------------------------------------------
(Updated July 23, 2014, 7:40 p.m.)
Review request for samza.
Changes
-------
10,000,000 messages in 12s.
Bugs: SAMZA-245
https://issues.apache.org/jira/browse/SAMZA-245
Repository: samza
Description (updated)
-------
use array dequeue in SystemConsumers and linkedlist in blocking envelope map
renaming unprocessed messages to unprocessed messages by SSP. caching
unprocessed message counts
updating with martin and yan's feedback.
Revert "changing SystemConsumer API to return only one message per SSP"
This reverts commit 098dcef6b7e99ba9cf5204d204b32dfa379182e8.
changing SystemConsumer API to return only one message per SSP
test one more piece of logic in poll
set defaults in system consumers as variables. add a poll interval test.
improve basic system consumers test
add a test for refreshing in system consumers
refactoring to make more scala-ish
save a few operations by only getting a consumer in poll when we need one
update should crank through bad deserialization messages rather than giving up
after one failure
switch refreshThreshold to pollIntervalMs
refresh consumers when system consumer is started
add a refresh threshold config
cleaning up metrics in system consumers
add docs
all tests pass. misc test fixes to make new api work. slight refactoring in
SystemConsumers.
clean up erroneous import
initial rebase without fixing file system conusmer.
Diffs (updated)
-----
docs/learn/documentation/0.7.0/jobs/configuration-table.html
edcb74fc47c140a110cb78c7289493c79c169413
samza-api/src/main/java/org/apache/samza/system/SystemConsumer.java
591f8fbfe5784766078789b00d5e7d777f051e2b
samza-api/src/main/java/org/apache/samza/system/SystemStreamPartitionIterator.java
9acfb108906e24cc3b9d589bce252c764e3ae2ce
samza-api/src/main/java/org/apache/samza/util/BlockingEnvelopeMap.java
9503739fc323f07aa156b4b18061b8a7ba838dd3
samza-api/src/test/java/org/apache/samza/system/TestSystemStreamPartitionIterator.java
3ecabab9230f64dab203e3e19659828cbaefeb0e
samza-api/src/test/java/org/apache/samza/util/TestBlockingEnvelopeMap.java
cb4d14854671ea602785581f7ed243dffde4ec26
samza-core/src/main/scala/org/apache/samza/config/TaskConfig.scala
8b881f2533975eea960e7a96b76763ef33af2a78
samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala
b303615ca0978a92febe227fcf88b5acecce4223
samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala
9eb70f298cbaef02e558c7a9dac0d0c5a914bd3f
samza-core/src/main/scala/org/apache/samza/system/SystemConsumersMetrics.scala
b065ae6c9f1295d9036d0285e3fa8c5dd1042ec9
samza-core/src/main/scala/org/apache/samza/system/chooser/BufferingMessageChooser.scala
c7ef6ef051dd694d7f5f80e780b02ac4152ac81b
samza-core/src/main/scala/org/apache/samza/system/chooser/RoundRobinChooser.scala
537412114105ee9124a32e7bda4ef95139612ec8
samza-core/src/main/scala/org/apache/samza/util/DoublingBackOff.scala
e1d6d4c9573ba1d0c02dda876e97715fc33a33b9
samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala
97e65eb2c4269f32decd364af07de2ddce784f97
samza-core/src/test/scala/org/apache/samza/system/chooser/TestBufferingMessageChooser.scala
c96c53bcf5dc89147166319a29930b6961e2dd41
samza-core/src/test/scala/org/apache/samza/system/filereader/TestFileReaderSystemConsumer.scala
b2e04a74d690847e4494752b8d14deaaba5348b8
samza-core/src/test/scala/org/apache/samza/util/TestDoublingBackOff.scala
eaeb0050f3aa1d6f2fe04c45ca43bf8d04b0d28e
samza-test/src/main/scala/org/apache/samza/test/performance/TestPerformanceTask.scala
23d122e54fbefc6edfdfc1c0e3e6fa18f7141fcb
samza-test/src/test/scala/org/apache/samza/test/performance/TestSamzaContainerPerformance.scala
4016768d9dd0098d74e35b88bcf95d238edec427
Diff: https://reviews.apache.org/r/23588/diff/
Testing
-------
Thanks,
Chris Riccomini
