[GitHub] samza pull request #605: Enabling auto-discovery of regex input topics
Github user rmatharu closed the pull request at: https://github.com/apache/samza/pull/605 ---
[GitHub] samza pull request #796: Enabling auto-discovery of regex input topics
GitHub user rmatharu opened a pull request: https://github.com/apache/samza/pull/796 Enabling auto-discovery of regex input topics This PR makes the following changes * Enriches StreamPartitionCountMonitor to periodically monitor input-regexes to match to actual inputs and stop the job when a new input stream is discovered. * Add a new API to SysAdmin to allow listing of all streams, e.g., Kafka-topics. KafkaSysAdmin implementation of this uses KafkaConsumer's listTopics API. (Even if listTopics had 1 million topics with 100 bytes per topic total, temporary memory overhead will be 100 MB). * Added config job.coordinator.monitor-input-regex.frequency.ms for the monitoring frequency, and job.coordinator.monitor-input-regex.%s for each input system. Users can then choose desired regex for each input system, e.g., job.coordinator.monitor-input-regex.kafka=test-.*. * We can later enrich RegexTopicGen rewriter to add a monitor-input-regex config to allow periodic jonitoring You can merge this pull request into a Git repository by running: $ git pull https://github.com/rmatharu/samza newtopic-test Alternatively you can review and apply these changes as the patch at: https://github.com/apache/samza/pull/796.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #796 commit f33839e9b7eae354a790d0002352c732c5f6868f Author: Ray Matharu Date: 2018-11-06T01:58:13Z Full-working logic for new topic discovery ---
Re: [VOTE] Apache Samza 1.0.0 RC4
1. ./bin/check-all.sh succeeded. 2. Both the commands ./bin/integration-tests.sh yarn-integration-tests and ./bin/integration-tests.sh standalone-integration-tests succeeded. 3. Verified the SQL console available in samza-tool tgz. +1 Thanks. On Mon, Nov 5, 2018 at 2:13 PM Daniel Nishimura wrote: > Ran check-all and integration tests. All passed. > Verified signatures. > Also as an extra sanity check, I ran a Samza snapshot build with > samza-hello-samza (High level and low level jobs). > +1 > > On Wed, Oct 31, 2018 at 7:15 PM Jagadish Venkatraman > wrote: > > > Hi all, > > > > This is a call for a vote on a release of Apache Samza 1.0.0. Thanks to > > everyone who has contributed to this release. > > > > The release candidate can be downloaded from here: > > http://home.apache.org/~jagadish/samza-1.0.0-rc4/ > > > > The release candidate is signed with pgp key AF81FFBF, which can be found > > on keyservers: > > http://pgp.mit.edu/pks/lookup?op=get=0xAF81FFBF > > > > The git tag is release-1.0.0-rc4 and signed with the same pgp key: > > > > > https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.0.0-rc4 > > > > Test binaries have been published to Maven's staging repository, and are > > available here: > > https://repository.apache.org/content/repositories/orgapachesamza-1055/ > > > > The vote will be open for 72 hours (ending at 7:00 PM PST Saturday, > > November 3). > > > > Please download the release candidate, check the hashes/signature, build > it > > and test it, and then please vote: > > > > [ ] +1 approve > > > > [ ] +0 no opinion > > > > [ ] -1 disapprove (and reason why) > > > > For me, I ran check-all.sh, integration tests and verified the SQL > console > > in samza-tool tgz. So +1 (binding) from my side. > > > > Thanks, > > Jagadish > > > > -- > > Jagadish V > > >
[GitHub] samza pull request #795: SAMZA-1977: Shutdown the container threadPool when ...
Github user shanthoosh closed the pull request at: https://github.com/apache/samza/pull/795 ---
[GitHub] samza pull request #795: SAMZA-1977: Shutdown the container threadPooln stop...
GitHub user shanthoosh opened a pull request: https://github.com/apache/samza/pull/795 SAMZA-1977: Shutdown the container threadPooln stopping AsyncRunLoop. You can merge this pull request into a Git repository by running: $ git pull https://github.com/shanthoosh/samza shut_down_container_threadPool_on_runloop_stop Alternatively you can review and apply these changes as the patch at: https://github.com/apache/samza/pull/795.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #795 commit 1df48baa072742743053f902c87a59d34df62951 Author: Shanthoosh Venkataraman Date: 2018-11-05T16:44:10Z SAMZA-1977: Stop the container threadPool when stopping AsyncRunLoop. ---
[GitHub] samza pull request #794: SAMZA-1616: Samza-Sql - Support remote table for st...
GitHub user atoomula opened a pull request: https://github.com/apache/samza/pull/794 SAMZA-1616: Samza-Sql - Support remote table for stream-table join You can merge this pull request into a Git repository by running: $ git pull https://github.com/atoomula/samza remote Alternatively you can review and apply these changes as the patch at: https://github.com/apache/samza/pull/794.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #794 commit c87461e42594216876d3c5e6df19ad1eca1adcc0 Author: Aditya Toomula Date: 2018-11-05T15:34:41Z Implement remote table join ---
Re: [VOTE] Apache Samza 1.0.0 RC4
Ran check-all and deployed locally with the test jobs. All tests passed. +1 (binding) from my end. Thank you all! -Yi On Fri, Nov 2, 2018 at 6:02 PM Jagadish Venkatraman wrote: > Extending the vote till Monday 4PM so that more folks can try it out. > > Thanks, > Jagadish > > On Fri, Nov 2, 2018 at 4:34 PM Prateek Maheshwari > wrote: > > > Verified signatures and successfully ran check-all and integration tests. > > > > +1 (binding) from me. > > > > Thanks, > > Prateek > > > > On Fri, Nov 2, 2018 at 2:39 PM Boris S wrote: > > > > > > ran check-all and integration tests. All passed. > > > verified signatures. > > > +1 > > > > > > On Wed, Oct 31, 2018 at 7:15 PM Jagadish Venkatraman < > > jagad...@apache.org> > > > wrote: > > > > > > > Hi all, > > > > > > > > This is a call for a vote on a release of Apache Samza 1.0.0. Thanks > to > > > > everyone who has contributed to this release. > > > > > > > > The release candidate can be downloaded from here: > > > > http://home.apache.org/~jagadish/samza-1.0.0-rc4/ > > > > > > > > The release candidate is signed with pgp key AF81FFBF, which can be > > found > > > > on keyservers: > > > > http://pgp.mit.edu/pks/lookup?op=get=0xAF81FFBF > > > > > > > > The git tag is release-1.0.0-rc4 and signed with the same pgp key: > > > > > > > > > > > https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tag;h=refs/tags/release-1.0.0-rc4 > > > > > > > > Test binaries have been published to Maven's staging repository, and > > are > > > > available here: > > > > > > https://repository.apache.org/content/repositories/orgapachesamza-1055/ > > > > > > > > The vote will be open for 72 hours (ending at 7:00 PM PST Saturday, > > > > November 3). > > > > > > > > Please download the release candidate, check the hashes/signature, > > build it > > > > and test it, and then please vote: > > > > > > > > [ ] +1 approve > > > > > > > > [ ] +0 no opinion > > > > > > > > [ ] -1 disapprove (and reason why) > > > > > > > > For me, I ran check-all.sh, integration tests and verified the SQL > > console > > > > in samza-tool tgz. So +1 (binding) from my side. > > > > > > > > Thanks, > > > > Jagadish > > > > > > > > -- > > > > Jagadish V > > > > > > > > > -- > Jagadish V, > Graduate Student, > Department of Computer Science, > Stanford University >
