Re: [VOTE] Apache Samza 1.8.0 RC0

+1 (binding). Verified the md5 and sha1 checksums. Run check-all.sh on linux and the build/tests all passed. Please also generate the sha256 checksums for the release artifacts, according to Apache's requirements for open source releases. Thanks, Xinyu On Wed, Dec 21, 2022 at 9:47 AM Ajo Thomas

Re: [DISCUSS] Apache Samza 1.8.0 RC0

+1. Thanks for driving this release! Thanks, Xinyu On Sat, Dec 17, 2022 at 5:18 AM James DeMichele wrote: > Ship it!  > > On Fri, Dec 16, 2022, 12:53 PM Ajo Thomas wrote: > > > Hi All, > > > > Since the last Samza 1.7.0 release earlier this year, we have added a few > > major features and

[ANNOUNCE] Welcome Ajo Thomas as Samza Committer

with the responsibilities of a Samza Committer. Please join me to give him a warm welcome! Xinyu Liu on behalf of the Apache Samza PMC

Re: [VOTE] SEP-31: Pipeline Drain- Support the ability to drain pipelines to allow incompatible intermediate schema changes

+1. Overall the design looks good. Thanks for contributing to this feature. Thanks, Xinyu On Tue, Nov 29, 2022 at 10:44 AM Ajo Thomas wrote: > Hi All, > > This is a call for a vote on *SEP-31: Pipeline Drain- Support the ability > to drain pipelines to allow incompatible intermediate schema

Re: [VOTE] Apache Samza 1.7.0 RC1

+1 (binding). Verified the signature and checksums, and also ran check-all tests which all passed. Thanks, Xinyu On Fri, Mar 11, 2022 at 2:00 PM Bob S wrote: > +1 > Ran build,test and both integration tests (regular + standalone) and > check-all. > Verified signatures, sha and md5. > Thanks

Re: [VOTE] SEP-30: Support Updates in Table API

+1 on my side. Glad to see this feature coming. Please make sure the api changes are reflected in the documents, e.g. https://samza.apache.org/learn/documentation/1.0.0/api/table-api.html. Thanks, Xinyu On Mon, Dec 20, 2021 at 10:44 AM Ajo Thomas wrote: > Hi All, > > This is a call for a vote

[ANNOUNCE] Welcome Daniel Chen as Samza Committer

, the Samza PMC trusts Daniel with the responsibilities of a Samza Committer. Please join me to give him a warm welcome! Xinyu Liu on behalf of the Apache Samza PMC

Re: Samza Runner for Beam Processing Time support

is working for me as I can see the 1 second early firing within a 10 sec window and the fired panes are accumulated. You can also use this git repo as a reference. Thanks, Xinyu On Mon, Feb 8, 2021 at 9:41 AM Xinyu Liu wrote: > Hi, Jan, > > Thanks for reporting this issue to us. Proce

Re: [DISCUSS] Samza 1.5.1 release

+1. Hopefully it's not a bug in my code :) Thanks, Xinyu On Wed, Aug 19, 2020 at 10:29 AM Yi Pan wrote: > +1 to this release as well! > > On Wed, Aug 19, 2020 at 9:21 AM Prateek Maheshwari > wrote: > > > +1, this is a critical bug and we should release the fix ASAP. > > > > Thanks, > >

Re: [VOTE] Apache Samza 1.4.0 RC1

+1 (binding). Run check-all.sh and integration tests for both yarn and standalone. All passed. Thanks, Xinyu On Fri, Mar 6, 2020 at 6:46 PM Yi Pan wrote: > Have downloaded the files, build with check-all.sh, and ran both YARN and > standalone integration tests. All passed. > > +1 (binding).

Re: [DISCUSS] 1.4 release

This is great! Thanks for driving the new release. Thanks, Xinyu On Tue, Feb 25, 2020 at 2:45 PM Cameron Lee wrote: > Hi all, > We have made some updates to Samza, and we would like to make a Samza 1.4 > release. > We have been testing some of these changes internally at Linkedin, and we >

[ANNOUNCE] Please welcome Bharath Kumarasubramanian to the Samza PMC

Hi all, I'm very pleased to announce that the Samza PMC has voted Bharath Kumarasubramanian to be a Project Management Committee (PMC) Member. The PMC is responsible for the overall health of a project and for voting in new committers and PMC members, as well as voting on releases. Over the past

Re: [DISCUSS] Samza 1.3.1 release

Cool! Thanks for driving this release. Thanks, Xinyu On Thu, Feb 13, 2020 at 10:28 AM Hai Lu wrote: > Hi all, > > We're going to make a 1.3.1 release to address some critical issues that > were found in 1.3.0 > > 1.3.1 will be based off 1.3.0 but include the following additional commits: > >

Re: [DISCUSS] SEP-22: Container Placements in Samza

Thanks for writing a very detailed design doc. My feedback is below: - The title of the SEP is Container Placements in Samza, which sounds like it's going to reimplement the container placements logic currently in ClusterBasedJobCoordinator. After reading the proposal, I think it should be

Re: [VOTE] SEP 25: PR Title and Description Guidelines

+1 (binding). Thanks, Xinyu On Wed, Dec 18, 2019 at 7:49 PM Jagadish Venkatraman wrote: > +1 (binding) > > On Thursday, December 19, 2019, Daniel Nishimura > wrote: > > > +1 non-binding > > > > > On Dec 18, 2019, at 7:09 PM, Yi Pan wrote: > > > > > > +1 (binding) > > > > > > On Wed, Dec 18,

Re: [VOTE] SEP-23: Simplify Job Runner

+1 (binding). This proposal will help future split deployment as well as make the deployment simple. Thanks for making the effort! Thanks, Xinyu On Wed, Dec 11, 2019 at 10:30 AM Ke Wu wrote: > Hi, > > This is a call for a vote on SEP-23: Simplify Job Runner. Thanks to > everyone who help

Re: [DISCUSS] SEP-23: Simplify Job Runner

config factory/path, how > >> do we pass the parameters needed for Yarn submission, e.g. job name, > id, > >> and tgz path? > > > > We can either pass them by --config or introduce delicate command line > arguments for it in CommandLine.scala. > > > > > &g

Re: [VOTE] Apache Samza 1.3.0 RC2

+ 1 (binding) Verified the signatures, built and ran the integration tests. All passed. There is one flaky test failure during running check-all.sh: org.apache.samza.table.batching.TestBatchProcessor$TestBatchTriggered.testBatchOperationTriggeredByBatchSize FAILED java.lang.AssertionError at

Re: [DISCUSS] SEP-23: Simplify Job Runner

Thanks a lot for putting out the design for simplifying the job submission process. The motivation makes sense to me that most of the planning and config generation should be done after submitting to the cluster, instead of during the submission, which can happen in a local sandbox without the

Re: [VOTE] SEP-20: Samza on Kubernetes

+1 (binding). Thanks, Xinyu On Thu, Nov 7, 2019 at 10:50 AM Weiqing Yang wrote: > Hi All, > > The feedback from the discussion thread: > http://mail-archives.apache.org/mod_mbox/samza-dev/201911.mbox/browser is > positive. This is a call for a vote for SEP-20: Samza on Kubernetes < > >

Re: [DISCUSS] SEP-20: Samza on Kubernetes

+1 on the design. This is a great feature to allow Samza to expand its deployment to Kubernetes clusters. Nice job! Thanks, Xinyu On Mon, Nov 4, 2019 at 10:10 AM Weiqing Yang wrote: > Hi all, > > We created SEP-20: Samza on Kubernetes, which supports Samza to run on > Kubernetes natively.

Re: [ANNOUNCE] Please welcome Boris Shkolnik to the Samza PMC

Congrats, Boris! Xinyu On Fri, Jun 7, 2019 at 3:13 PM Jakob Homan wrote: > Howdy all- >I'm very pleased to announce that the Samza PMC has voted Boris > Shkolnik to be a Project Management Committee (PMC) Member. The PMC > is responsible for the overall health of a project andl for voting

Re: REMINDER. [VOTE] Apache Samza 1.2.0 RC4

+1 (binding). run check-all.sh and the build passed. Having trouble running the integration tests in both linux and mac, possibly due to my local machine env. Thanks, Xinyu On Mon, Jun 3, 2019 at 11:00 AM Daniel Nishimura wrote: > check-all.sh and integration tests passed. +1 from me. > >

Re: [DISCUSS] 1.2 release

+1 on the 1.2 release. It's time to get the newly added features out! Thanks, Xinyu On Mon, May 20, 2019 at 9:39 AM Jake Maes wrote: > I don't think we did anything for "Making sendTo(table), sendTo(stream) > non-terminal". The ticket was just closed as a "won't fix" IIRC. > > Nevertheless, I

[DISCUSS] Hygene for merging PRs

Hi, all, I've seen different practices around how PRs are contributed, reviewed and merged for Samza open source. I think it's time to bring up our committer guide again to make sure we follow exactly the guidelines. It's also an opportunity to talk about future improvement to the flow. *PR

[ANNOUNCE] New committer announcement: Cameron Lee

Hi, all, Please join me and the rest of the Samza PMC in welcoming a new committer: Cameron Lee. Cameron has been contributing to Samza since early 2018. He worked on multiple areas: the runtime context in Samza, checkpoint enhancements, as well as testing and gradle improvements. He is also

[DISCUSS] Change Apache Samza git comments/merge email recipient to commits@samza

Hi, All, Our dev mailing list has been flooded with github comments/merges so it's really hard to see the meaningful discussions and user engagement. Shall we move dev@Samza off the JIRA/github messages? We will use comm...@samza.apache.org as the recipients. In addition, I think we should have

Re: [VOTE] SEP-21: Samza Async API for High Level

Looks great! +1 (binding). On Thu, Mar 28, 2019 at 12:03 PM Jake Maes wrote: > +1 (binding) > > On Mon, Mar 25, 2019 at 6:09 PM Jagadish Venkatraman < > jagadish1...@gmail.com> > wrote: > > > +1 (binding); > > > > thanks Bharath for the proposal and the implementation. > > > > LGTM; > > > > On

Re: [DISCUSS] Samza 1.1.0 release

+1 (binding) Thanks, Xinyu On Thu, Mar 7, 2019 at 12:43 AM santhosh venkat < santhoshvenkat1...@gmail.com> wrote: > +1 (non-binding) > > Thanks, > > On Wed, Mar 6, 2019 at 10:42 PM Yi Pan wrote: > > > +1 (binding) > > > > On Wed, Mar 6, 2019 at 10:08 PM Daniel Chen wrote: > > > > > Hello

Re: [VOTE] Migration of Samza git repo to gitbox.apache.org

+1 (binding). On Wed, Jan 23, 2019 at 2:39 PM Prateek Maheshwari wrote: > +1 (binding) again > > - Prateek > > On Wed, Jan 23, 2019 at 11:50 AM Pawas Chhokra > wrote: > > > > Hi all, > > > > This is a call for a vote on migrating Samza git repo to > gitbox.apache.org, on > > 11 AM, Jan 29,

Re: Samza runner bundle implementation

nderstanding is for "Batched RPC", I need a batch of keys. So, > I am not sure if I can use GroupIntoBatches. > > On 1/18/19, 10:40 AM, "Xinyu Liu" wrote: > > This email is from an external sender. > > > sorry, the correct link to

Re: Samza runner bundle implementation

sorry, the correct link to the first reference: https://github.com/apache/beam/blob/master/runners/samza/src/main/java/org/apache/beam/runners/samza/runtime/DoFnOp.java . Thanks, Xinyu On Fri, Jan 18, 2019 at 10:35 AM Xinyu Liu wrote: > Hi, Omkar, > > Your observation is correct.

Re: Samza runner bundle implementation

Hi, Omkar, Your observation is correct. Currently bundle is implemented in a per-event basis (Code is DoFnOp.processElement, https://docs.google.com/spreadsheets/d/1pIUQ8J658B7GPNDt5dwiJBRQyWep1n2rXlkONyA6ZMM/edit#gid=1709587251). We are working on supporting bundles in Samza right now so in Beam

Re: InMemorySystemDescriptor ignores serde

Hi, Tom, First, your observation about current InMemorySystem is exactly right and thanks for raising this issue to the community! The current InMemorySystem came up with a tight coupling with the Samza test framework, which I believe put quite a lot of limitations on its uses, e.g. using

[ANNOUNCE] New committer announcement

Hi, all, Please join me and the rest of the Samza PMC in welcoming a new committer: - Shanthoosh Venkataraman: of, but not limited to, his work on Samza standalone. Through his work on developing new features such as host affinity on standalone and refining existing Samza code base, the PMC

Re: app.class or task.class for beam samza runner

, 2019 at 12:14 PM Xinyu Liu wrote: > As Prateek mentioned, I also double checked the exception, which comes > from a class (ApplicationUtil.java) that only exists in Samza 1.0. Please > remove any Samza 1.0 dependency since Beam api currently works with Samza > 0.14.1. > > Your c

Re: app.class or task.class for beam samza runner

As Prateek mentioned, I also double checked the exception, which comes from a class (ApplicationUtil.java) that only exists in Samza 1.0. Please remove any Samza 1.0 dependency since Beam api currently works with Samza 0.14.1. Your config looks mostly correct to me. The following is not needed:

Re: Welcome Hai Lu and Aditya Toomla as committers to Apache Samza!

Well deserved! Congrats! Xinyu On Tue, Nov 6, 2018 at 10:39 AM Jake Maes wrote: > Welcome Hai and Aditya. Nice work! > > On Tue, Nov 6, 2018 at 10:20 AM Yi Pan wrote: > > > Hi, all, > > > > All official steps are completed and please join me to welcome Hai and > > Aditya to Apache Samza

Re: [VOTE] SEP-15: New Runtime Context API

+1 (binding). @Cameron: your first voting email seems not showing up for me. For future communications, please use your personal email instead. Thanks, Xinyu On Mon, Oct 15, 2018 at 10:44 AM Prateek Maheshwari wrote: > +1 (non-binding) for these changes. > > (Resending from a non-LI email due

Re: [VOTE] SEP-14: System and Stream Descriptors

+1 (binding). Great API for IO! Thanks, Xinyu On Fri, Oct 12, 2018 at 5:36 PM Yi Pan wrote: > +1 (binding) from me. Thanks! > > On Fri, Oct 12, 2018 at 12:30 PM Prateek Maheshwari > wrote: > > > Hi folks, > > > > Now that SAMZA-1804 has been implemented and reviewed, we've updated > > SEP-14

Re: [VOTE] SEP-13: unified ApplicationDescriptor and ApplicationRunner APIs for high and low- level APIs in YARN and standalone deployment

+1 (binding). Thanks for the effort to making our API consistent. Thanks, Xinyu On Fri, Oct 12, 2018 at 12:32 PM Prateek Maheshwari wrote: > +1 (non-binding) from me. Thanks for making the changes and updating the > SEP! > > - Prateek > > On Fri, Oct 12, 2018 at 12:15 PM Yi Pan wrote: > > > >

Re: Review Request 68867: Update hell0-samza with latest code

others to test. Thanks, Xinyu Liu

Review Request 68867: Update hell0-samza with latest code

--- Only tested the high-level wikipedia example for now. I will keep testing the rest of it in the next few days. Commit the change for now to unblock others to test. Thanks, Xinyu Liu

Re: [VOTE] Committership for Wei and Srini

Sorry, sent to the wrong mailing list. Please ignore. Thanks, Xinyu On Wed, Jul 4, 2018 at 8:17 AM, Xinyu Liu wrote: > Hi, All, > > Seems the feedback from the discussion thread is very positive and we > didn't receive any objections, this is a vote for committership for Wei

[VOTE] Committership for Wei and Srini

Hi, All, Seems the feedback from the discussion thread is very positive and we didn't receive any objections, this is a vote for committership for Wei and Srini. Please vote: [] +1, Approve to become a new committer [] -1, Do not approve (please provide specific comments) >From my side, it's +1

Re: [VOTE] Apache Samza 0.14.1 RC2

gt; 2) ./bin/integration-tests.sh succeeded > > > 3) expanded samza-tools and followed the tutorial steps for standalone > > SQL > > > examples in http://samza.apache.org/learn/ > tutorials/0.14/samza-sql.html. > > > Succeeded. > > > 4) veri

[VOTE] Apache Samza 0.14.1 RC2

Hi, All, This is a call for a vote on a release of Apache Samza 0.14.1. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~xinyu/samza-0.14.1-rc2/ The release candidate is signed with pgp key C31D7061, which can be

[CANCEL][VOTE] Apache Samza 0.14.1 RC1

We found an issue when running SQL console tool. Cancel this vote and will submit another candidate after it's fixed. Thanks, Xinyu

Re: [VOTE] Apache Samza 0.14.1 RC1

t > org.apache.samza.sql.runner.SamzaSqlApplicationRunner.( > SamzaSqlApplicationRunner.java:66) > at > org.apache.samza.tools.SamzaSqlConsole.executeSql( > SamzaSqlConsole.java:110) > at org.apache.samza.tools.SamzaSqlConsole.main(SamzaSqlConsole.java:104) > {noformat} > > On Thu, May

Re: [VOTE] Apache Samza 0.14.1 RC1

Extend the vote to Friday, May 18 5pm. Please take a chance to run the tests and vote for it. Thanks, Xinyu On Wed, May 16, 2018 at 10:44 AM, Xinyu Liu <xinyuliu...@gmail.com> wrote: > A reminder to download the build and vote for it. Since we are passing the > end date for the

Re: [VOTE] Apache Samza 0.14.1 RC1

, Xinyu Liu <xinyuliu...@gmail.com> wrote: > Hi, All, > > This is a call for a vote on a release of Apache Samza 0.14.1. Thanks to > everyone who has contributed to this release. > > The release candidate can be downloaded from here: > http://home.apache.org/~xinyu/samza-

[VOTE] Apache Samza 0.14.1 RC1

Hi, All, This is a call for a vote on a release of Apache Samza 0.14.1. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~xinyu/samza-0.14.1-rc1/ The release candidate is signed with pgp key C31D7061, which can be

[CANCEL][VOTE] Apache Samza 0.14.1 RC0

We need an performance fix for eventhub system producer (SAMZA-1706) as well as a fix for the unit test in TestRocksDbKeyValueStoreJava. Cancel this vote and will submit another RC soon. Thanks, Xinyu

Re: [VOTE] Apache Samza 0.14.1 RC0

, Xinyu Liu <xinyuliu...@gmail.com> wrote: > Hi, All, > > This is a call for a vote on a release of Apache Samza 0.14.1. Thanks to > everyone who has contributed to this release. > > The release candidate can be downloaded from here: > http://home.apache.org/~xinyu/samza-

[Discuss] Samza 0.14.1 release

Hi, All, We have been adding many improvements and critical bug fixes in the areas of Samza sql, standalone, eventhub system consumer and host-affinity since 0.14.0 release. The changes should warrant a new minor release: SQL - SAMZA-1681: Samza-sql: Add support for handling older record schema

Re: [VOTE] SEP-11: Host affinity in standalone.

+1 (binding). Look forward to the implementation. Xinyu On Wed, Apr 4, 2018 at 2:43 PM, Yi Pan wrote: > +1 (binding). Thanks for the revisions! > > -Yi > > On Wed, Apr 4, 2018 at 2:39 PM, santhosh venkat < > santhoshvenkat1...@gmail.com> wrote: > > > Hi, > > > > This is a

Samza 0.14.0 officially released!

Hi, all, I am pleased to let you know that we have officially released Samza 0.14.0! Huge thanks to everyone for working on the features and bugs in this release, and cooperating in the release process. Here is the announcement blog: https://blogs.apache.org/samza/ Cheers! Xinyu

Re: [VOTE] Apache Samza 0.14.0 RC5

rified the signature. > > > Ran build, tests and integration tests on Unix. > > > All passed (as before requires python 2.7, neither higher nor lower). > > > > > > +1 > > > Thanks guys !! > > > > > > On Fri, Dec 22, 2017 at 2:50 P

[VOTE] Apache Samza 0.14.0 RC5

This is a call for a vote on a release of Apache Samza 0.14.0. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~xinyu/samza-0.14.0-rc5/ The release candidate is signed with pgp key C31D7061, which can be found on

[CANCEL][VOTE] Apache Samza 0.14.0 RC4

Hi, This is an official CANCEL for the RC4 vote as we found an issue in the KafkaCheckpointManager. We will create a new RC with the fix. Thanks, Xinyu

[VOTE] Apache Samza 0.14.0 RC4

This is a call for a vote on a release of Apache Samza 0.14.0. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~xinyu/samza-0.14.0-rc4/ The release candidate is signed with pgp key C31D7061, which can be found on

[CANCEL][VOTE] Apache Samza 0.14.0 RC3

Hi, This is an official CANCEL for the RC3 vote as we found the documentation for Samza SQL (SAMZA-1526) was not merged to the 0.14.0 release branch. We will create a new RC with it. Thanks, Xinyu

[RESULT][VOTE] Apache Samza 0.14.0 RC3

The vote of 0.14.0 RC3 has been more than 72 hours and we got +1 (binding) x 3 and +1 (non-binding) x 3. Samza 0.14.0 officially passed the VOTE! Thanks! Xinyu

Re: [VOTE] Apache Samza 0.14.0 RC3

e signature > > > > > > Ran check-all.sh on both Linux and OSX > > > Ran integration tests from OSX > > > > > > LGTM > > > > > > > > > On Tue, Dec 19, 2017 at 10:00 AM, Boris S <bor...@gmail.com&g

Re: [VOTE] Apache Samza 0.14.0 RC3

Correction: This is a call for a vote on a release of Apache Samza *0.14.0*. On Mon, Dec 18, 2017 at 10:57 AM, xinyu liu <xinyuliu...@gmail.com> wrote: > This is a call for a vote on a release of Apache Samza 0.13.1. Thanks to > everyone > who has contributed to this release.

[VOTE] Apache Samza 0.14.0 RC3

This is a call for a vote on a release of Apache Samza 0.13.1. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~xinyu/samza-0.14.0-rc3/ The release candidate is signed with pgp key C31D7061, which can be found on

[CANCEL][VOTE] Apache Samza 0.14.0 RC2

Hi, This is an official CANCEL for the RC2 vote as we have another javadoc update in the release. Thanks, Xinyu

Re: [DISCUSS] Samza 0.14.0 release

+1. Very happy to see a lot of important features added in this release. Thanks, Xinyu On Mon, Nov 27, 2017 at 10:00 AM, Jagadish Venkatraman wrote: > +1 from my side. > > Thank you Bharath for driving the release! > > > > On Mon, Nov 27, 2017 at 9:50 AM, Bharath Kumara

Re: Samza questions (downtime during deployment and num partition per task)

Hi, Tony, For your questions: 1) Having a hot-standby job instance for fail-over may introduce certain operational complications. For example, if they produce to the same output, then both will be running in a short period of time, which might lead to duplicates in output. If the jobs has local

Review Request 63267: Add instructions to README.md

Description --- Add instructions to README.md Diffs - README.md 0f80e9e52240abc071dcdfb56800826ef5d49d7d build.gradle ec451d576a157eba2d8889b63b8574f397937197 Diff: https://reviews.apache.org/r/63267/diff/1/ Testing --- Thanks, Xinyu Liu

Re: [VOTE] SEP-8: Add in-memory system consumer & producer

+1 on the overall design. This will make testing a lot easier! Thanks, Xinyu On Wed, Sep 6, 2017 at 10:45 AM, Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > Hi all, > > Can you please vote for SEP-8? > You can find the design document here >

Re: [Discuss] Samza 0.13.1 release

+1. Thanks for pushing the release. Xinyu On Mon, Aug 14, 2017 at 10:58 AM, Boris S wrote: > +1 for the release. > > On Mon, Aug 14, 2017 at 10:38 AM, Yi Pan wrote: > > > +1 for the list! Let's proceed! > > > > On Fri, Aug 11, 2017 at 6:13 PM, Ignacio

Re: [VOTE] SEP-5: Enable partition expansion of input streams

+1 (non-binding) on this design. To me the task-count based groupers should work well in practice for partition expansion of system using hash for partitions, e.g. Kafka. It will not cause any state transfer between hosts so the runtime cost will be minimal. In the future when we support

Re: [DISCUSS] SEP-5: Enable partition expansion of input streams

How about making the partition mapping function a pluggable component in the partition expansion? Mathematically, this is a mapping function which is able to map the new partitions to the old ones: *f (new partition) -> old partition* If the function is a surjective function (

Re: [VOTE] Apache Samza 0.13.0 RC6

+1 (non-binding). Downloaded the source tar, built it and run check-all.sh on REHL6 with gradle 2.8. All passed. As a side note to Jagadish's comments, the build doesn't work on a higher gradle version either (gradle 3.5). Seems "-language:implicitConversions -language:reflectiveCalls" is not a

[VOTE] SEP 6: Support Control Message Across Intermediate Streams

Hi, everyone, This is the voting thread for SEP 6: Support Control Message Across Intermediate Streams. The wiki page that discusses the SEP is: https://cwiki.apache.org/confluence/display/SAMZA/SEP-6+Support+Control+Message+Across+Intermediate+Streams . Please vote. Thanks, Xinyu

Re: [DISCUSS] SEP-6: Support Watermark Across Intermediate Streams for Batch Processing

plement > > the exact-once algorithm Backett is working on since the algorithm will > > require buffering message/state that are not committed yet. For now, if > we > > are processing each incoming message immediately and only checkpoint the > > messages being processe

Re: [DISCUSS] SEP-6: Support Watermark Across Intermediate Streams for Batch Processing

dination between all producers at the source? Say, if > > the task taking one input source emits watermark at 1min interval and the > > task taking another input source emits watermark at 5min interval, how > does > > the downstream consumer reconcile the watermarks? > > >

Re: [DISCUSS] SEP-6: Support Watermark Across Intermediate Streams for Batch Processing

lid> wrote: > MessageType and ControlMessage.Type look redundant. You could either use > "ControlMessage" as the type in MessageType or drop ControlMessage.Type. > > On Fri, May 26, 2017 at 5:14 PM, xinyu liu <xinyuliu...@gmail.com> wrote: > > > Thanks a lot fo

Re: [NON-VOTING][RELEASE-TESTING] Apache Samza 0.13.0 RC1

Change subject to [NON-VOTING][RELEASE-TESTING]. Thanks, Xinyu On Wed, May 24, 2017 at 7:42 PM, Navina Ramesh (Apache) wrote: > Hi everyone, > > This is a call for a vote on a release of Apache Samza 0.13.0. Thanks to > everyone who has contributed to this release. We are

[DISCUSS] SEP-6: Support Watermark Across Intermediate Streams for Batch Processing

Hi all, I created SEP-6 for SAMZA-1260 : Support Watermark Across Intermediate Streams for Batch Processing. The link to the SEP is here:

Re: [DISCUSS] SEP-4: Adjunct Data Store for Unbounded DataSets

Hi, Wei, +1 on the proposed design. This is going to reduce a lot of heavy-lifting work that's needed done by user code today to bootstrap a data stream into local store. The configs look quite straightforward and easy to set up. Overall the design looks great to me. I have one question: in the

Re: [DISCUSS] SEP-2: ApplicationRunner Design

to the user. I created https://issues.apache.org/jira/browse/SAMZA-1252 to track it. Thanks, Xinyu On Fri, Apr 28, 2017 at 5:55 PM, xinyu liu <xinyuliu...@gmail.com> wrote: > Right, option #2 seems redundant for defining streams after further > discussion here. StreamSpec itself is fle

Re: [DISCUSS] SEP-2: ApplicationRunner Design

ing to distribute the JARs? Can you use the > same mechanism to distribute the serialized graph? > > On Fri, Apr 28, 2017 at 12:14 AM, xinyu liu <xinyuliu...@gmail.com> wrote: > > > btw, I will get to SAMZA-1246 as soon as possible. > > > > Thanks, > > Xinyu &g

Re: [DISCUSS] SEP-2: ApplicationRunner Design

btw, I will get to SAMZA-1246 as soon as possible. Thanks, Xinyu On Thu, Apr 27, 2017 at 9:11 PM, xinyu liu <xinyuliu...@gmail.com> wrote: > Let me try to capture the updated requirements: > > 1. Set up input streams outside StreamGraph, and treat graph building as a > libr

Re: [DISCUSS] SEP-2: ApplicationRunner Design

e. #2 and #3, I'm a big fan of getting rid of the extra configuration > > indirection you currently have to jump through (this is also related to > > system consumer configuration from #1. It makes it much easier to > discover > > what the configurable parameters are too, if we

Re: [DISCUSS] SEP-2: ApplicationRunner Design

Let me give a shot to summarize the requirements for ApplicationRunner we have discussed so far: - Support environment for passing in user-defined objects (streams potentially) into ApplicationRunner (*Beam*) - Improve ease of use for ApplicationRunner to avoid complex configurations such as

Re: [VOTE] SEP-1: Semantics of ProcessorId in Samza

+1 on my side. Very happy to see this proposal. This is a blocker for integrating fluent API with StreamProcessor, and hopefully we can get it resolved soon :). Thanks, Xinyu On Tue, Mar 28, 2017 at 11:28 AM, Navina Ramesh (Apache) wrote: > Hi everyone, > > This is a voting

Re: Review Request 52168: Tasks endpoint to list the complete details of all tasks related to a job

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52168/#review169231 --- Ship it! Ship It! - Xinyu Liu On Feb. 16, 2017, 6:26 a.m

Re: [DISCUSS] SEP-1: Semantics of ProcessorId in Samza

gt; create the instance ? > > Thanks! > Navina > > > > On Thu, Mar 16, 2017 at 4:31 PM, xinyu liu <xinyuliu...@gmail.com> wrote: > > > The proposal looks great to me! Changing the id type to string will make > > sure this can work with other types of clu

Re: [DISCUSS] SEP-1: Semantics of ProcessorId in Samza

The proposal looks great to me! Changing the id type to string will make sure this can work with other types of cluster which doesn't support integer id. The interface and config provides a pluggable way to have different id generators for different use cases. One minor thing I found is that the

Re: Review Request 52168: Tasks endpoint to list the complete details of all tasks related to a job

che.org/r/52168/#comment241407> rename this as getJobModle. In javadoc, make clear there is no side effect. - Xinyu Liu On Feb. 16, 2017, 6:26 a.m., Shanthoosh Venkataraman wrote: > > --- > This is an automatically generated e-mai

Re: [DISCUSS] SAMZA-1141 - Apache Samza Development Process Improvements

+1 on this proposal too. Could you actually put this proposal as the first SEP (like SEP-0), so it serves an example of how it will look like in practice? Xinyu On Tue, Mar 14, 2017 at 3:34 PM, Navina Ramesh wrote: > Just to clarify: The proposal for code and

Re: Understanding metrics

Hi, Ankit, When running your job in multithreading, block-ns here actually includes the process_ns. This is because after your task.process() is submitted to the thread pool, the run loop thread will be blocked until the process() is complete for one of the task. It's interesting that block-ns

Re: Review Request 56928: SAMZA-1099: Changes to hello-samza (latest branch) for the 0.12.0 release

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56928/#review166399 --- Ship it! Ship It! - Xinyu Liu On Feb. 22, 2017, 3:45 p.m

Re: Review Request 56909: SAMZA-1099: Changes to hello-samza (master branch) for the 0.12.0 release

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56909/#review166398 --- Ship it! Ship It! - Xinyu Liu On Feb. 22, 2017, 3:55 p.m

Re: Review Request 56913: SAMZA-1099: Documentation updates for Samza 0.12 release (for master branch)

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56913/#review166396 --- Ship it! Ship It! - Xinyu Liu On Feb. 22, 2017, 7:29 p.m

Re: Review Request 56911: SAMZA-1099: Documentation updates for Samza 0.12 release (for 0.12.0 branch)

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56911/#review166393 --- Ship it! Ship It! - Xinyu Liu On Feb. 22, 2017, 4:27 p.m

Re: [VOTE] Apache Samza 0.12.0 RC2

Ran build, checkAll and integration tests. All passed. +1 non-binding. Thanks, Xinyu On Wed, Feb 8, 2017 at 4:18 PM, Boris S wrote: > Cloned the release and ran build, test and checkAll.sh > All passed. > Verified MD5 and the signature. > Got warning - "this key is not

Re: Review Request 52570: SAMZA-1025: documentation for hdfs system consumer

link in versioned/index.html, something like "Reading from HDFS" before the link of "Writing to HDFS" - Xinyu Liu On Oct. 5, 2016, 8:54 p.m., Hai Lu wrote: > > --- > This is an automatically generated e-mail.

  1   2   3   >