----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13725/#review26699 -----------------------------------------------------------
Ship it! Ship It! - Sriram Subramanian On Oct. 3, 2013, 9:32 p.m., Chris Riccomini wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/13725/ > ----------------------------------------------------------- > > (Updated Oct. 3, 2013, 9:32 p.m.) > > > Review request for samza. > > > Repository: samza > > > Description > ------- > > adding docs for default chooser. > > > adding metrics and logging. > > > Adding metrics to round robin chooser. updating MessageChooserFactory to take > a metrics registry. > > > adding logs to kafka system admin > > > handle interrups in KafkaSystemAdmin. > > > rename wrappedchooser back to default chooser. fix some comments. > > > use topic metadata cache to get metadata > > > adding test for kafkasystemadmin.getlatestoffsets. bumping up gradle heap for > samsa-test and samsa-kafka tests. shutting down zookeeper in tests, so we > don't leak ports. > > > adding a first pass at KafkaSystemAdmin.getLastOffsets > > > use the systemAdmins from samza container in wrapped chooser, rather than > doing wiring internally. > > > changing ordering of wrapped chooser from bootstrap -> priority -> batching > to bootstrap -> batching -> priority. > > > switch default chooser to WrappedChooser. Always use wrapped chooser. Remove > task.chooser.wrapped.class. > > > wrote default chooser test that bootstraps, prioritizes, and batches. > > > turning on priority tests for default chooser. refactor default chooser a bit > to make it easier to test. > > > test default chooser with just round robin. > > > bug fix to use round robin chooser factory in default chooser > > > change wiring for default chooser to make it more testable. add > start/stop/register tests to all choosers. > > > add one more test in priority chooser. > > > add unit test for tiered priority chooser > > > add unit test for bootstrapping chooser. > > > turn default chooser on in samza container. add license to test stateful > task. add more docs to default chooser. clean up batching unit test. > > > add unit test for batching chooser. > > > only wire in wrapping message choosers when we need them. add docs to the > default chooser factory. > > > rename class to BootstrappingChooser. > > > refactor to move into org.apache.samza.system.chooser. > > > build latest message offset map. > > > add a streams-behind-chooser that guarantees one message from each > SystemStream before choose is called. > > > add start/stop/register back. all tests pass. > > > fix bug -- should allow manual override if Int.MaxInt for bootstrap streams. > > > minor bug in default chooser. was re-using same chooser everywhere. > > > adding wiring in default chooser. > > > initial pass adding composed message choosers. > > > rebase to master, which includes SAMZA-25 metrics. fix several tests that > were broken after removing start/stop/register. > > > adding more docs for round robin > > > remove start/stop/register > > > cleanup some wiki markdown in MessageChooser javadoc. > > > add more javadocs to the message chooser. > > > Merge branch 'SAMZA-2_fine-grain-control-over-stream-consumption' of > github.com:criccomini/incubator-samza into > SAMZA-2_fine-grain-control-over-stream-consumption > > > added start, stop, and register to message chooser. > > > adding docs for message chooser. swiching round robin chooser back to a queue. > > > missed license in message chooser factory > > > add apache licensing > > > samza container was using message chooser, not message chooser factory. fixed. > > > add stream chooser test. update stream chooser to invert priority due to bug. > > > add round robin test. fix compile error in round robin chooser. > > > add priority chooser test. fix bug in priority chooser that was reversing > ordering. > > > adding stream chooser. adding message chooser factory. > > > adding priority chooser. moving default chooser to round robin chooser. > adding config for chooser > > > Diffs > ----- > > build.gradle 077e44b0ed521504096b47f41b6180449e97280e > docs/learn/documentation/0.7.0/container/streams.md > b15c34d3f68e32d56e0da8af91d78e47a5110f67 > samza-api/src/main/java/org/apache/samza/system/MessageChooser.java > 306b2902303c72f3d7a3eb313f55d7e88d21e00d > samza-api/src/main/java/org/apache/samza/system/SystemAdmin.java > c902d414484e05ae75c9ca58ad9629cb01120f62 > samza-api/src/main/java/org/apache/samza/system/chooser/MessageChooser.java > PRE-CREATION > > samza-api/src/main/java/org/apache/samza/util/SinglePartitionSystemAdmin.java > e4ed30bf363142d82a3c40909e160b5825fe60fd > > samza-core/src/main/scala/org/apache/samza/config/DefaultChooserConfig.scala > PRE-CREATION > samza-core/src/main/scala/org/apache/samza/config/TaskConfig.scala > 0c742d83c2f60d2448a79376677713a1ff0b11ec > samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala > 62bd243115e71612e00784124baa972b33e56cb7 > samza-core/src/main/scala/org/apache/samza/system/DefaultChooser.scala > 5a72e7a3bfba0f06a5a98c6ba26865800d7780b9 > samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala > d24671ec64a42ede6f779effe9c845e1cbbc5e51 > > samza-core/src/main/scala/org/apache/samza/system/chooser/BatchingChooser.scala > PRE-CREATION > > samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala > PRE-CREATION > > samza-core/src/main/scala/org/apache/samza/system/chooser/DefaultChooser.scala > PRE-CREATION > > samza-core/src/main/scala/org/apache/samza/system/chooser/RoundRobinChooser.scala > PRE-CREATION > > samza-core/src/main/scala/org/apache/samza/system/chooser/TieredPriorityChooser.scala > PRE-CREATION > samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala > PRE-CREATION > > samza-core/src/test/scala/org/apache/samza/system/chooser/MockMessageChooser.scala > PRE-CREATION > > samza-core/src/test/scala/org/apache/samza/system/chooser/TestBatchingChooser.scala > PRE-CREATION > > samza-core/src/test/scala/org/apache/samza/system/chooser/TestBootstrappingChooser.scala > PRE-CREATION > > samza-core/src/test/scala/org/apache/samza/system/chooser/TestDefaultChooser.scala > PRE-CREATION > > samza-core/src/test/scala/org/apache/samza/system/chooser/TestRoundRobinChooser.scala > PRE-CREATION > > samza-core/src/test/scala/org/apache/samza/system/chooser/TestTieredPriorityChooser.scala > PRE-CREATION > > samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemAdmin.scala > 183c6ccce39dedaef9dba56d5b61ffdedfc9d08a > > samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemFactory.scala > ba08af85892f0793d01e3118fa7ad6d569a0f238 > > samza-kafka/src/test/scala/org/apache/samza/checkpoint/TestKafkaCheckpointManager.scala > baf4695108b62fb2826cf0d9cdc484ba6d7bb524 > > samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestKafkaSystemAdmin.scala > PRE-CREATION > > samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala > 7d4e20a8bdc7a45b0a1b464a6f4b868d1d03eab0 > > samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMasterTaskManager.scala > 68050528cd5f8acfe3a1f7563b4e7fe6c7473be5 > > Diff: https://reviews.apache.org/r/13725/diff/ > > > Testing > ------- > > > Thanks, > > Chris Riccomini > >
