-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27649/
-----------------------------------------------------------
(Updated Dec. 22, 2014, 6:40 p.m.)
Review request for samza.
Bugs: SAMZA-448
https://issues.apache.org/jira/browse/SAMZA-448
Repository: samza
Description (updated)
-------
remove coordinator system admin. move methods into system admin to match
SAMZA-226 style.
merge post-SAMZA-226 master
fix log4j appender to use job coordinator to fetch configs when in the am
merge master
make serde pluggable for coordinator stream, but hardcode to JSON serde.
more docs on wire format
update docs
switching coordinator stream messages to a key
turn down verbosity of logging
fix ordering of keys in coordinator stream message
add test for coordinator stream system producer
add coordinator system consumer test
add coordinator stream message tests
add kafka system admin create cooridnator stream test
remove SystemAdmin.createCoordinatorStream, and create a CoordinatorStreamAdmin
instead.
more feedback from yan
partial fix of yan's review comments
add logging info
make deletes actually work
add javadocs to mock coordinator stream classes.
delete old configs when job runner publishes new configs
add rewriting into job coordinator
make process job properly set only coordinator stream config
all tests pass
fix samza container performance test
explicitly flush all buffers when closing the kafka producer. fix stateful task
test.
fix kafka tests
all core tests work
fix test checkpoint tool by adding a mock coordinator consumer that dumps the
entire config
working on fixing checkpoint tool tests
fleshing out the coordinator stream message javadocs
remove duplicate code from coordinator system factory
add more javadocs. clean up todos inkafka system admin.
remove yarn.container.count from yarn config, but use it as a fallback to
job.container.count
add some docs and headers to the coodinator stream and system admin
refactoring to add coordinator stream system consumer
cleanup source in job runne
abstract coordinator system producer creation into a factory
add todos
config stream works
create coordinator stream in system admin
connecting job coordinator to job runner via coordinator stream
add util and logging methods
adding coordinator message and system producer wrapper
Diffs (updated)
-----
build.gradle 38383bd9e3f0847d6088a4ea4c1ee6f3dcd1e430
samza-api/src/main/java/org/apache/samza/system/SystemAdmin.java
8995ba30c823bddcdfd3af7100e1440e71ef7998
samza-api/src/main/java/org/apache/samza/util/SinglePartitionWithoutOffsetsSystemAdmin.java
01997ae22641b735cd452a0e89a49219e2874892
samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamMessage.java
PRE-CREATION
samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamSystemConsumer.java
PRE-CREATION
samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamSystemProducer.java
PRE-CREATION
samza-core/src/main/java/org/apache/samza/serializers/JsonSerde.java
PRE-CREATION
samza-core/src/main/scala/org/apache/samza/checkpoint/CheckpointTool.scala
ddc30af7c52d8a4d5c5de02f6757c040b1f31c93
samza-core/src/main/scala/org/apache/samza/config/JobConfig.scala
3b6685e00837a4aaf809813e62b7e52823bc07a9
samza-core/src/main/scala/org/apache/samza/config/ShellCommandConfig.scala
1a2dd4413f56e53dbeeb47b5637d7b0c50522f02
samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala
c14f2f623bb4bae911dd3085ce428175930e4545
samza-core/src/main/scala/org/apache/samza/coordinator/stream/CoordinatorStreamSystemFactory.scala
PRE-CREATION
samza-core/src/main/scala/org/apache/samza/job/JobRunner.scala
16345cd1c1354a0d25a0000d81a307dbe3abbe81
samza-core/src/main/scala/org/apache/samza/job/local/ProcessJobFactory.scala
6985af6e7cc0d408fa07fbac60141d1126323777
samza-core/src/main/scala/org/apache/samza/job/local/ThreadJobFactory.scala
530255e5866bc49ec5ce1a0b7437470cd4e17010
samza-core/src/main/scala/org/apache/samza/serializers/JsonSerde.scala
744eec05857a4ea14c718e3750fb575d3678b1f8
samza-core/src/main/scala/org/apache/samza/system/filereader/FileReaderSystemAdmin.scala
ec1d749cb5186f788b402877996a4caa37e99362
samza-core/src/main/scala/org/apache/samza/util/Util.scala
1a67586eeec95dabfeb3b6881af9b3865c3029ca
samza-core/src/test/java/org/apache/samza/coordinator/stream/MockCoordinatorStreamSystemFactory.java
PRE-CREATION
samza-core/src/test/java/org/apache/samza/coordinator/stream/MockCoordinatorStreamWrappedConsumer.java
PRE-CREATION
samza-core/src/test/java/org/apache/samza/coordinator/stream/TestCoordinatorStreamMessage.java
PRE-CREATION
samza-core/src/test/java/org/apache/samza/coordinator/stream/TestCoordinatorStreamSystemConsumer.java
PRE-CREATION
samza-core/src/test/java/org/apache/samza/coordinator/stream/TestCoordinatorStreamSystemProducer.java
PRE-CREATION
samza-core/src/test/resources/test.properties
9348c7de956ebf02f58a163dc6fb391a7e29ae64
samza-core/src/test/scala/org/apache/samza/checkpoint/TestCheckpointTool.scala
af800dfeedbfea75abaac3f15fd53bc55b743daf
samza-core/src/test/scala/org/apache/samza/checkpoint/TestOffsetManager.scala
d18d4c45c5de3b50a24d6c776364e1f589db8f4d
samza-core/src/test/scala/org/apache/samza/coordinator/TestJobCoordinator.scala
a8e5d36921464a2e36693279e8083e4544c4e289
samza-core/src/test/scala/org/apache/samza/serializers/TestJsonSerde.scala
60460713a2d4f7b7b389f21c1450d45c1afaa0f4
samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemAdmin.scala
b790be17cfe08da28220ffb381cbd618ebe25cf0
samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemFactory.scala
4506ea367eec4e40da45feee777ba73069025a4c
samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemProducer.scala
a0e1ccbfe9dc4fd26ca6b30fc2d1348fb7d007e4
samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestKafkaSystemAdmin.scala
c759a7bea7b67714eaa90a97f828079f26acbca4
samza-log4j/src/main/java/org/apache/samza/config/Log4jSystemConfig.java
5f5195c8f303261cde069390db543ee86d43148d
samza-log4j/src/main/java/org/apache/samza/logging/log4j/StreamAppender.java
9a9d6483ffb92ba6ad5c730deec87651a7cbc64a
samza-test/src/main/java/org/apache/samza/system/mock/MockSystemAdmin.java
c0a20af5a2f4329ad4a2cff378ced3bececbc1cb
samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala
ca25258217e5ebc44b34fbc4d69ecb28c81df618
samza-test/src/test/scala/org/apache/samza/test/performance/TestSamzaContainerPerformance.scala
d589d762a18f9425aa8d8dd589011a151bcb59a4
samza-yarn/src/main/scala/org/apache/samza/config/YarnConfig.scala
03395e2efa0fec723e354177d06bfacf7d2a9215
samza-yarn/src/main/scala/org/apache/samza/job/yarn/SamzaAppMaster.scala
a1dbe0435ae08c710d4bfc871458ed386e275cd2
samza-yarn/src/main/scala/org/apache/samza/job/yarn/SamzaAppMasterState.scala
8ba435ef2ccf2af64d01eb4bc3b1c362fb03779d
samza-yarn/src/main/scala/org/apache/samza/job/yarn/SamzaAppMasterTaskManager.scala
b0b6543856cb87888c5a719182ad9576b51bba1a
samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala
24b11da06a69da734c85720ef39d65ee46d821d5
samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMasterLifecycle.scala
765f72f4c10bd0f1d1adab28c8ec54d9cbea5fb4
samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMasterService.scala
81dea9d6d1921462b200c62dbdf016c0eb2f01b2
samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMasterTaskManager.scala
58f2464211a1fb7ff40f5978fd41f64d088002d0
Diff: https://reviews.apache.org/r/27649/diff/
Testing
-------
Thanks,
Chris Riccomini
