----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29342/ -----------------------------------------------------------
(Updated Jan. 13, 2015, 10:31 p.m.) Review request for samza. Repository: samza Description (updated) ------- add yarn-site.xml templating support, and set yarn-site.xml for run-job.sh and kill-yarn-job.sh. add some more logging in samza job runner. fix vmem ratio for linux boxes. add rstrip to smoketests to make python work with both 2.6 and 2.7 python. remove AUTH_TYPE adding support for trailing zopkio switches to integration-tests.sh. updating documentation to describe how to use ssh with zopkio. add license to integration-tests.sh. fix spacing in samza job yarn deployer. support relative paths in integration tests. add -e to bash to fail early if integration-tests.sh fails. upgrade virtual env to 12.0.2. do a safety check on pydistutils, which breaks virtualenv remove HDFS dependency in samza-test, since we don't use it. fix ZK dependency version in YARN to be compatible with Kafka's ZK version. aggregate YARN logs add docs to negate number task remove TODO Merge branch 'master' into SAMZA-468 add docs to website and README.md clean up tests.py add docs for python test files. add license headers. add docs to samza job deployer merge samza-test and samza-test-job into one package TODO update re-enable teardown move data out of /tmp all tests pass. no procs leak. logs are copied as expected. all tests pass. logs don't work and zk/nm are leaked. integration tests pass. next up: stop and uninstall. samza yarn job install works. add primitive yarn deployer based off of navina's everything installs except samza job now. yarn starts, kafka doesn't rename simple integration test to tests.py update todo. close kafka consumer in test. rename to smoke tests py use kafka python lib for consumer in test use python kafka producer instead of CLI producer update todo stop using an HTTP server to serve samza job tarballs. sftp instead. don't print, use logger.info name smoke tests fix cp line to properly copy scripts directory and overwrite different files disable passwords remove instance suffix from deployer names update todo fix missing log4j file in samza container make log link work make tests pass. an attempt at log aggregation, but it's not working yet. shutdown yarn, kafka, and zk after tests finish make test a bit more rigorous by checking line counts remove sleep calls cache zk, kafka, and yarn downloads to speed up testing adding TODO use relative paths for configs tests pass integration test starts up, but hangs on kafka convert dtf to zopkio. zopkio starts up, but tests aren't running due to missing imports fix tarball for test jobs to not have a root directory build test tgz and put it into test directory add an integration test script that bootstraps a virtual environment original import Diffs (updated) ----- .gitignore dfc6533b46ece74074c833408c2ce29611834b73 README.md 4453984bb518635a4a1ea57384aac5edebff038b bin/integration-tests.sh PRE-CREATION build.gradle 38383bd9e3f0847d6088a4ea4c1ee6f3dcd1e430 docs/contribute/tests.md 84e1dbd1cd10863f8703db14a7876745946a3f9d samza-test/src/main/config/negate-number.properties PRE-CREATION samza-test/src/main/java/org/apache/samza/test/integration/NegateNumberTask.java PRE-CREATION samza-test/src/main/python/configs/downloads.json PRE-CREATION samza-test/src/main/python/configs/kafka.json PRE-CREATION samza-test/src/main/python/configs/smoke-tests/smoke-tests.json PRE-CREATION samza-test/src/main/python/configs/yarn.json PRE-CREATION samza-test/src/main/python/configs/zookeeper.json PRE-CREATION samza-test/src/main/python/deployment.py PRE-CREATION samza-test/src/main/python/perf.py PRE-CREATION samza-test/src/main/python/requirements.txt PRE-CREATION samza-test/src/main/python/samza_job_yarn_deployer.py PRE-CREATION samza-test/src/main/python/templates.py PRE-CREATION samza-test/src/main/python/templates/yarn-site.xml PRE-CREATION samza-test/src/main/python/tests.py PRE-CREATION samza-test/src/main/python/tests/smoke_tests.py PRE-CREATION samza-test/src/main/resources/hello-stateful-world.samsa samza-test/src/main/resources/join/checker.samsa samza-test/src/main/resources/join/emitter.samsa samza-test/src/main/resources/join/joiner.samsa samza-test/src/main/resources/join/watcher.samsa samza-test/src/main/resources/perf/counter.samsa samza-test/src/main/resources/perf/kv-perf.properties Diff: https://reviews.apache.org/r/29342/diff/ Testing ------- Thanks, Chris Riccomini
