Re: [VOTE] SEP-32: Elasticity for Samza

2023-02-08 Thread Yi Pan
+1 (binding) Thanks! -Yi On Tue, Feb 7, 2023 at 2:14 PM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > +1 (binding) > > Cheers, > Bharath > > On Tue, Feb 7, 2023 at 12:56 PM Lakshmi Manasa > wrote: > > > Hi folks, > > > > This is a call for vote on SEP-32: Elasticity for

Re: [DISCUSS] SEP-32: Elasticity for Samza

2023-02-06 Thread Yi Pan
(single task) in key ordered executor >> sitting in the same host will increase the load on the host and (c) if one >> or more of the parallel units (threads here) needs more resources, it will >> result in large container which makes scheduling harder as finding large >&

Re: [DISCUSS] SEP-32: Elasticity for Samza

2023-01-19 Thread Yi Pan
Hey, Manasa, Sorry to chime in late. A few questions: a) how are states for the virtual tasks managed during split/merge? b) what's perf impact when we have 2 virtual tasks on the same SSP in the same container, while one virtual task is much faster than the other? c) what's the reason that a

Re: [VOTE] Apache Samza 1.8.0 RC0

2023-01-06 Thread Yi Pan
(+1) binding, Downloaded the src tarball, run check-all.sh and passed all tests. One thing noticed: there are configurations for your personal keys used for publishing the jars checked in gradle.properties. I don't think that we need to include that in the published src tarball. Otherwise,

Re: [ANNOUNCE] Welcome Ajo Thomas as Samza Committer

2022-12-16 Thread Yi Pan
Welcome and congrats, Ajo! - Yi On Wed, Dec 14, 2022 at 3:42 PM Xinyu Liu wrote: > Hi, All, > > I am glad to announce that Ajo Thomas has officially accepted our > invitation and become an Apache Samza Committer now. > > Ajo has made contributions to improve both Samza user experience and >

Re: [VOTE] SEP-31: Pipeline Drain- Support the ability to drain pipelines to allow incompatible intermediate schema changes

2022-12-08 Thread Yi Pan
+1. Long awaited feature! Thanks! -Yi On Tue, Nov 29, 2022 at 11:46 AM Xinyu Liu wrote: > +1. > > Overall the design looks good. Thanks for contributing to this feature. > > Thanks, > Xinyu > > On Tue, Nov 29, 2022 at 10:44 AM Ajo Thomas > wrote: > > > Hi All, > > > > This is a call for a

Re: SEP-31: Pipeline Drain: Support the ability to drain pipelines to allow incompatible intermediate schema changes

2022-12-08 Thread Yi Pan
ready to shutdown, RunLoop > shuts down. > > Do you see any issues with it ? > > - Ajo > > > On Thu, 1 Dec 2022 at 20:06, Yi Pan wrote: > > > Hi, Ajo, > > > > Sorry to reply this late. Could you clarify one thing in the design: For > > watermark triggered

Re: SEP-31: Pipeline Drain: Support the ability to drain pipelines to allow incompatible intermediate schema changes

2022-12-01 Thread Yi Pan
Hi, Ajo, Sorry to reply this late. Could you clarify one thing in the design: For watermark triggered window draining, is the infinitive watermark trigger happen first, or the drain token in all source SSP happen first? Shouldn't it be the following sequence: a) all drain token from all input

[REPORT] Samza - Nov 2022

2022-11-09 Thread Yi Pan
## Description: The mission of Samza is the creation and maintenance of software related to distributed stream processing framework ## Issues: - There are no issues requiring board attention. ## Membership Data: Apache Samza was founded 2015-01-22 (8 years ago) There are currently 29 committers

Re: Request for new release

2022-10-15 Thread Yi Pan
Hi, James, Thanks for the reminder. We are preparing the new 1.8 release. It is expected to be the end of this quarter. Best, -Yi On Mon, Oct 10, 2022 at 12:52 PM James DeMichele wrote: > Hello, we just had a pr merged to main in the Samza app that now supports > Java 11 runtime

Re: Java 11 Checkin again

2022-09-20 Thread Yi Pan
; > > > > ./gradlew build -PscalaSuffix=2.11 > > > > > > The build fails with this command using that version of Java 8 ^. > > > > > > Anyway, just wanted to point that out since I hit this in my branch > > > trying to utilize the "bin/c

Re: Java 11 Checkin again

2022-09-19 Thread Yi Pan
, but just wanted to call it out. > > -Jamie > > > > On Wed, Sep 14, 2022 at 5:52 PM Yi Pan wrote: > > > Hey, James, > > > > In order to merge your PR without breaking the jdk8 older modules, we > will > > need the changes proposed here. Can

Re: Java 11 Checkin again

2022-09-14 Thread Yi Pan
Java 11 and yarn 3 module need to only use the Scala 2.12 > > version of the build. > > > > Jamie > > > > > > On Wed, Sep 14, 2022, 2:38 AM Yi Pan wrote: > > > >> Hi, James, > >> > >> Sorry to reply late. I just came back

Re: Java 11 Checkin again

2022-09-14 Thread Yi Pan
it > >> would > >> > work. Yarn itself is not compatible at 2.10.1 with Java 11 so you > would > >> > have to update yarn even if the Java library here wasn't updated. > >> > > >> > Could we move this version I'm proposing to a 2.x vers

Re: Running v1.7.0 locally

2022-09-02 Thread Yi Pan
Hey, Malcolm, Thanks for reporting this issue. Could you open a JIRA to track that? Best! -Yi On Mon, Aug 29, 2022 at 5:53 PM Malcolm McFarland wrote: > Hey folks, > > I've recently been attempting to upgrade our legacy application from Samza > 1.5.1 to 1.7.0. With version 1.5.1, I've had no

Re: Java 11 Checkin again

2022-09-02 Thread Yi Pan
Hey, James, Thanks for the ping. @prateek, can we have someone to review this change? One question: have you tested the change w/ the older YARN cluster version (running 2.10.1)? If this change requires YARN cluster upgrade to 3.3.4 as well, that may be a breaking change to existing Samza users

[REPORT] Samza - April 2022

2022-04-13 Thread Yi Pan
## Description: The mission of Samza is the creation and maintenance of software related to distributed stream processing framework ## Issues: - There are no issues requiring board attention. ## Membership Data: Apache Samza was founded 2015-01-22 (7 years ago) There are currently 29 committers

Re: [RESULT][VOTE] Apache Samza 1.7.0 RC1

2022-03-15 Thread Yi Pan
Thanks, Daniel! Just want to mention that Boris also voted +1 (binding). Best! -Yi On Tue, Mar 15, 2022 at 9:22 AM Daniel Chen wrote: > Hey all, > > The vote for 1.7.0 release has been out for more than 72 hours and we got > +1(binding) x3 from Yi, Xinyu, Daniel > > Samza 1.7.0 officially

Re: [VOTE] Apache Samza 1.7.0 RC1

2022-03-15 Thread Yi Pan
+1 (binding). Ran check-all, verified the signature and checksums. All passed. Thanks for pushing 1.7.0 out of the door! Yi On Fri, Mar 11, 2022 at 2:31 PM Xinyu Liu wrote: > +1 (binding). > > Verified the signature and checksums, and also ran check-all tests which > all passed. > > Thanks,

[REPORT] Samza - Feb 2022

2022-02-09 Thread Yi Pan
## Description: The mission of Samza is the creation and maintenance of software related to distributed stream processing framework ## Issues: - There are no issues requiring board attention. ## Membership Data: Apache Samza was founded 2015-01-22 (7 years ago) There are currently 29 committers

Re: [DISCUSS] Apache Samza 1.7.0 RC0

2022-01-26 Thread Yi Pan
Huge +1! Can't wait to see this list of features coming out! -Yi On Wed, Jan 26, 2022 at 2:36 PM Daniel Chen wrote: > Hi folks, > > We have added a number of major features and changes to master since > > 1.6, that warrants a major 1.7 release. > > Within LinkedIn, some of these features have

Re: [VOTE] SEP-30: Support Updates in Table API

2022-01-24 Thread Yi Pan
Discussed and resolved the minor concerns offline. +1 (binding) for this one. Thanks! -Yi On Tue, Dec 21, 2021 at 1:28 PM Xinyu Liu wrote: > +1 on my side. > > Glad to see this feature coming. Please make sure the api changes are > reflected in the documents, e.g. >

[Draft] [REPORT] Apache Samza Jan 2022

2022-01-12 Thread Yi Pan
Hi, team, Please read the draft report below and let me know if I missed anything. Thanks! - Yi == ## Description: The mission of Samza is the creation and maintenance of software related to distributed stream processing framework ##

[Draft][REPORT] Apache Samza Oct 2021

2021-10-11 Thread Yi Pan
Hi, team, Please read the following draft report for Oct 2021. Let me know if I missed anything. Thanks a lot! ## Description: The mission of Samza is the creation and maintenance of software related to distributed

Re: [ANNOUNCE] Welcome Daniel Chen as Samza Committer

2021-09-17 Thread Yi Pan
Congrats, Daniel, well deserved!!! -Yi On Fri, Sep 17, 2021 at 11:23 AM Xinyu Liu wrote: > Hi, all, > > I am glad to announce that Daniel Chen has officially accepted our > invitation and become an Apache Samza Committer now. > > Daniel has contributed to many areas of Samza, from his early

Re: Malformed URL issue while deploying Samza on a pure IPv6 VM

2021-08-26 Thread Yi Pan
Hi Vishal, Could you open a JIRA to track this one? I will circle back to our internal team to do a quick assessment on IPv6 related issues. Meanwhile, I strongly encourage you to submit the patch and continue your test, since that's the best way to discover any hidden issues on that front.

[Draft] [REPORT] Samza Report July 2021

2021-07-14 Thread Yi Pan
## Description: The mission of Samza is the creation and maintenance of software related to distributed stream processing framework ## Issues: - There are no issues requiring board attention. ## Membership Data: Apache Samza was founded 2015-01-22 (6 years ago) There are currently 28 committers

Re: [VOTE] SEP-29: Blob Store Based State Backup And Restore

2021-06-22 Thread Yi Pan
+1 (binding). Thanks for rolling out this big feature! -Yi On Tue, Jun 22, 2021 at 1:42 PM Sanil Jain wrote: > +1 (non-binding) Thanks for this contribution! > > -Sanil > > On Tue, 22 Jun 2021 at 13:13, Daniel Chen wrote: > > > +1 (non-binding), thanks! > > > > On Tue, Jun 22, 2021 at 1:10 PM

Re: [VOTE] SEP-28: Samza State Backend Interface and Checkpointing Improvement

2021-06-22 Thread Yi Pan
+1 (binding) this is going to improve our state recovery story significantly! -Yi On Mon, Jun 21, 2021 at 1:03 PM Daniel Chen wrote: > Hi all, > > This is a call for a vote on SEP-28: Samza State Backend Interface and > Checkpointing Improvements. Thanks to everyone who was involved with the >

Reminder: LinkedIn Stream Processing Meetup on Jun 24

2021-06-15 Thread Yi Pan
The meet up is coming in less than 10 days! Please remember to sign up and join us! https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/278266182/ Short abstracts for the contents:

Upcoming streams meetup event

2021-05-25 Thread Yi Pan
& Yuhong Cheng, LinkedIn Scaling Kafka Audit Service - Abhishek Mendhekar, LinkedIn Please visit our Meetup site for access to more details & the live event link (TBD), and to let us know you're coming! https://lnkd.in/g7Fq-3K Hope to see you there! Best, - Yi Pan

Re: How are samza.container.id generated in yarn?

2021-04-22 Thread Yi Pan
Hi, Debraj, In YARN environment, Samza uses YARN generated containerIds as environmental variables to set each container process's samza.container.id. i.e. when containers are requested by Samza AM process in YARN, YARN RM will reply with a set of allocated container objects, which is of class

Welcome Ke Wu as Apache Samza committer

2021-02-23 Thread Yi Pan
Hi, everyone, I am glad to announce that Ke Wu has officially accepted our invitation and become an Apache Samza committer now. Please join me to give him a warm welcome! Cheers! -Yi

Re: Zookeeper and Client upgrade

2021-01-14 Thread Yi Pan
/jira/browse/SAMZA-2617 > > Happy to start looking onto one of these. > > On 2021/01/11 23:04:08, Yi Pan wrote: > > Hi, Stuart,> > > > > Please feel free to raise tickets for update requests like these.> > > > > Thanks for reporting!> > > >

Re: [VOTE] Apache Samza 1.6.0 RC1

2021-01-13 Thread Yi Pan
Verified signature and sha1. Ran check-all and integration tests. All passed. +1 (binding). -Yi On Mon, Jan 11, 2021 at 4:08 PM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > Thanks for driving this release Boris! Verified signatures and ran all the > tests. > check-all and

[REPORT] Apache Samza - Jan 2021

2021-01-13 Thread Yi Pan
## Description Apache Samza is a distributed stream processing engine that is highly configurable to process events from various data sources, including real-time messaging system (e.g. Kafka) and distributed file systems (e.g. HDFS). ## Issues: - There are no issues requiring board attention.

Re: Zookeeper and Client upgrade

2021-01-11 Thread Yi Pan
Hi, Stuart, Please feel free to raise tickets for update requests like these. Thanks for reporting! -Yi On Fri, Jan 8, 2021 at 9:58 PM Stuart Perks wrote: > The Zookeeper version sits at 3.4.6 with 3.6.2 now available bringing > security enhancements. > > >

Re: SAMZA-2612: Kafka topic naming not supported fully

2020-12-28 Thread Yi Pan
Hey, Stuart, Sounds great that you have found the way around it! Thanks! -Yi On Sat, Dec 19, 2020 at 12:28 PM Stuart Perks wrote: > This can be done using withPhysicalName > > Closed the JIRA > > On 2020/12/17 12:19:27, Stuart Perks wrote: > >

Re: Samza jdk11 support

2020-10-16 Thread Yi Pan
ound the below doc but that talks about samza 0.10. > > https://samza.apache.org/contribute/tests.html > > Thanks, > > On Thu, Jun 4, 2020 at 12:18 AM Yi Pan wrote: > > > Hi, Debraj and Jordan, > > > > Thanks a lot for the ping. I dug a bit deeper in the past em

Draft board report for Samza

2020-10-10 Thread Yi Pan
Hi, all, Here is the draft report for Samza. Please let me know if I missed anything. Thanks! == ## Description: The mission of Samza is the creation and maintenance of software related to distributed stream processing framework ##

Re: Issues with Samza-Kafka compatibility while upgrading to latest versions

2020-10-10 Thread Yi Pan
Hi, Choudhary, Thanks for reporting your issues. Samza manages its dependencies in a few gradle files under ${ROOT}/gradle/dependency-versions-*.gradle. Specifically, the Kafka dependency is defined in the file ${ROOT}/gradle/dependency-versions.gradle. I quickly checked the versions listed in

Re: Problem regarding the Monitoring of Samza Applications

2020-09-02 Thread Yi Pan
Hi, Jan, Thanks for reporting this metrics issue. We will take a further look and get back to you. Thanks a lot! -Yi On Tue, Sep 1, 2020 at 8:01 AM Jan Bensien wrote: > Hello, > > I hope this is the right place to ask. > > I am having problems monitoring my Samza application. Using the >

Re: [VOTE] Apache Samza 1.5.1 RC0

2020-08-27 Thread Yi Pan
+1 (binding), ran ./bin/check-all.sh and all integration tests. Verified signatures. On Sat, Aug 22, 2020 at 8:09 PM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > Hi all, > > This is a call for a vote on a release of Apache Samza 1.5.1. We are > releasing 1.5.1 to address a

Re: [DISCUSS] Samza 1.5.1 release

2020-08-19 Thread Yi Pan
+1 to this release as well! On Wed, Aug 19, 2020 at 9:21 AM Prateek Maheshwari wrote: > +1, this is a critical bug and we should release the fix ASAP. > > Thanks, > Prateek > > On Tue, Aug 18, 2020 at 9:02 PM Bharath Kumara Subramanian < > codin.mart...@gmail.com> wrote: > > > Hi all, > > > >

Re: [VOTE] Apache Samza 1.5.0 RC1

2020-06-12 Thread Yi Pan
Hey, Yang, Can you open a JIRA for this failure? This is again related to a test code using sleep() trying to make sure that the test is completed before verification. -Yi On Fri, Jun 12, 2020 at 8:01 AM Yang Zhang wrote: > Being late to the party. Verified apache-samza-1.5.0-src.tgz in both

Re: [VOTE] Apache Samza 1.5.0 RC1

2020-06-10 Thread Yi Pan
+1 (binding). Ran check-all, verified sha1 and signature, ran both standalone and YARN integration tests. LGTM. On Mon, Jun 8, 2020 at 5:16 PM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > Hi all, > > This is a call for a vote on a release of Apache Samza 1.5.0. We are > excited

Re: Samza jdk11 support

2020-06-03 Thread Yi Pan
: > This is a pressing issue for our team as well. > > Jordan > > > On May 31, 2020, at 7:15 PM, Debraj Manna > wrote: > > > > Thanks Yi. > > > > Did you get any update on the JDK 11 roadmap? > > > > > > On Wed, May 27, 2020 at 1:04 AM Yi Pan wrote:

Re: [DISCUSS] Samza 1.5 release

2020-06-01 Thread Yi Pan
lgtm. Thanks for kicking off the discussion! -Yi On Tue, May 26, 2020 at 8:55 PM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > Hi all, > > We have accumulated few features/improvements since the last release and > would like to make Samza 1.5 release. > > I wanted to kick off

Re: Samza jdk11 support

2020-05-26 Thread Yi Pan
Hi, Debraj, Thanks for the reminder. We did discussed about JDK11 before. Unfortunately, I don't know whether JDK11 is up to the roadmap as of now. Let me sync up with the team and get back to you. -Yi On Mon, May 25, 2020 at 9:53 AM Debraj Manna wrote: > Hi > > I have seen a few earlier

Re: Streamtasks not instantiating consumers on startup

2020-04-10 Thread Yi Pan
Hi, Malcolm, Samza 0.14.1 is pretty old and if you have already upgraded Kafka to 2.2.1, I would highly recommend you to migrate to latest Samza version, which has Kafka 2.0 client. Depending on how you configure your broker, a 2.0 Kafka broker can be incompatible with older client like 0.8.6,

[REPORT] Samza - April 2020

2020-04-08 Thread Yi Pan
Apache Samza is a distributed stream processing engine that are highly configurable to process events from various data sources, including real-time messaging system (e.g. Kafka) and distributed file systems (e.g. HDFS). ## Issues: - No issues require board attention ## Membership Data: Apache

Re: [VOTE] SEP-24: Cluster-based Job Coordinator Dependency Isolation

2020-03-16 Thread Yi Pan
in the code to make the file > system location configurable (or specified through an environment > variable). The configuration or environment variable for the file system > location would not be YARN-specific, and it would be applicable to any > execution environment. > > On Wed, Ma

Re: Got Error Produce Respons with Correlation Id.

2020-03-13 Thread Yi Pan
h Adams > Software Engineer > www.helixeducation.com > Blog | Twitter | Facebook | LinkedIn > > ________ > From: Yi Pan > Sent: Friday, March 13, 2020 1:27 PM > To: dev@samza.apache.org > Subject: Re: Got Error Produce Respons with Correlation

Re: Got Error Produce Respons with Correlation Id.

2020-03-13 Thread Yi Pan
after testing Confluent's Cloud kafka offering. > We had no issues when running our own kafka clusters in aws. > > > Jeremiah Adams > Software Engineer > www.helixeducation.com > Blog | Twitter | Facebook | LinkedIn > > > From: Yi Pa

Re: [VOTE] SEP-24: Cluster-based Job Coordinator Dependency Isolation

2020-03-11 Thread Yi Pan
ple, what if some > execution environment does not use URLs to specify resource locations > (although maybe this isn't a reasonable concern to worry about?)? > > Thanks, > Cameron > > On Wed, Mar 11, 2020 at 4:43 PM Yi Pan wrote: > > > Hi, Cameron, > > > > T

Re: Got Error Produce Respons with Correlation Id.

2020-03-11 Thread Yi Pan
Hi, Jeremiah, Sorry to reply late. This WARN message indicates that producer failed to flush to checkpoint topic and would retry. Do you see the Samza job hanging after that? Is the checkpointed offset metrics incrementing in this case? Not clear on your claiming: "logs stop at that point". No

Re: [VOTE] SEP-24: Cluster-based Job Coordinator Dependency Isolation

2020-03-11 Thread Yi Pan
run some tests regarding this classloading pattern, but we will do > further verification for general split deployment. > d) Yes, you are correct. Good catch. It should be "described above at > Application classloader". > > Thanks for all of your questions. I will clarify s

Re: [VOTE] SEP-24: Cluster-based Job Coordinator Dependency Isolation

2020-03-09 Thread Yi Pan
Hi, Cameron, Sorry to chime in late. Overall, looks great! I do have a few suggestions/questions before I can cast my vote here: a) for the configuration variable names, why are we limiting ourselves to yarn.resource.*? We have changed some of the configuration variables from yarn specific to

Re: [VOTE] Apache Samza 1.4.0 RC1

2020-03-06 Thread Yi Pan
Have downloaded the files, build with check-all.sh, and ran both YARN and standalone integration tests. All passed. +1 (binding). Thanks! -Yi On Tue, Mar 3, 2020 at 3:03 PM Cameron Lee wrote: > Hi all, > > This is a call for a vote on a release of Apache Samza 1.4.0. Thanks to > everyone who

Re: [DISCUSS] 1.4 release

2020-02-28 Thread Yi Pan
sending out the VOTE thread today, and then release by > the end of next week. > Cameron > > On Thu, Feb 27, 2020 at 9:08 PM Yi Pan wrote: > > > Hey, Cameron, > > > > Briefly browsed through the list and there are total 24 tickets tagged > with > > 1.4

Re: Samza - New Relic integration

2020-02-28 Thread Yi Pan
> > > Yes. That's exactly what I am looking for. > > > > Let me try it out and I will get back to you if I face any issues. > > > > Thanks again > > > > Vaibhav Garg > > +91-9505020924 > > vaibhavgar...@gmail.com > > LinkedIn &

Re: Samza - New Relic integration

2020-02-27 Thread Yi Pan
Hi, Vaibhav, Check the description of task.opts in the configuration doc here: http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html Is this what you are looking for? On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg wrote: > Hi, > > Any help here would be great. > >

Re: [DISCUSS] 1.4 release

2020-02-27 Thread Yi Pan
Hey, Cameron, Briefly browsed through the list and there are total 24 tickets tagged with 1.4 and 11 are assigned/in-progress/done. Are we targeting to finish all 24 for 1.4? And what's the proposed timeline for 1.4? Thanks! -Yi On Wed, Feb 26, 2020 at 1:25 PM Xinyu Liu wrote: > This is

Re: [VOTE] Apache Samza 1.3.1 RC0

2020-02-18 Thread Yi Pan
Ran check-all and integration tests successfully. +1 (binding) On Thu, Feb 13, 2020 at 12:02 PM Hai Lu wrote: > Hi, > > This is a call for a vote on a release of Apache Samza 1.3.1 to redress > certain issues found in 1.3.0 > > The release candidate can be downloaded from here: >

Re: job.coordinator.replication.factor configuration

2020-02-15 Thread Yi Pan
JIRA opened: SAMZA-2460. Thanks! -Yi On Sat, Feb 15, 2020 at 9:42 AM Yi Pan wrote: > Hi, Robert, > > Thanks for pointing this out. This is an error on the documentation > side.The meaning of job.coordinator.replication.factor should not change > post 1.0. I will file a bug

Re: job.coordinator.replication.factor configuration

2020-02-15 Thread Yi Pan
Hi, Robert, Thanks for pointing this out. This is an error on the documentation side.The meaning of job.coordinator.replication.factor should not change post 1.0. I will file a bug against the doc site. Thanks! -Yi On Fri, Feb 14, 2020 at 7:52 AM Robert Wigginton wrote: > Hello, > > > >

Re: [ANNOUNCE] Please welcome Bharath Kumarasubramanian to the Samza PMC

2020-02-13 Thread Yi Pan
Congrats! Bharath, well deserved! -Yi On Thu, Feb 13, 2020 at 10:17 PM Wei Song wrote: > Congrats, Bharath, well deserved !!! > > > On 2/13/20, 8:51 PM, "Jagadish Venkatraman" > wrote: > > Congrats Bharath. Great work! Looking forward to continued > contributions! > > On Thursday,

Re: Samza consumer without Zookeeper

2020-02-05 Thread Yi Pan
the systems.x.consumer.zookeeper.connect from your config? -Yi On Wed, Feb 5, 2020 at 3:12 PM Robert Wigginton wrote: > Currently 1.0.0 but are in the process of upgrading to 1.3.0. > > On Wed, 2020-02-05 at 15:01 -0800, Yi Pan wrote: > > Hi, Robert, > > > > Which version of Samza are you using? > &

Re: Samza consumer without Zookeeper

2020-02-05 Thread Yi Pan
Hi, Robert, Which version of Samza are you using? -Yi On Tue, Feb 4, 2020 at 9:17 AM Robert Wigginton wrote: > Hello, > > We are currently evaluating moving from an on prem Kafka deployment to > Confluent Cloud. Confluent Cloud does not expose Zookeeper to user. > Is it possible to setup a

Re: [VOTE] SEP-26: Add SystemProducer for Azure Blob Storage

2020-01-09 Thread Yi Pan
+1 (binding). Good to see more cloud native integrations in Samza. -Yi On Wed, Jan 8, 2020 at 10:31 AM Prateek Maheshwari wrote: > +1 (binding). Thanks for the contribution. > > - Prateek > > On Tue, Jan 7, 2020 at 7:59 PM Jagadish Venkatraman < > jagadish1...@gmail.com> > wrote: > > > +1

Re: [Draft] Samza quarterly report

2020-01-09 Thread Yi Pan
: Recent Advances and our plans for future in Stream Processing Seattle - Scalable Stream Processing with Apache Samza P.S. just fixing one typo. -Yi On Thu, Jan 9, 2020 at 1:42 PM Yi Pan wrote: > ## Description: > - Apache Samza is a distributed stream processing engine that are

Re: [Draft] Samza quarterly report

2020-01-09 Thread Yi Pan
ache Samza 1.0: Recent Advances and our plans for future in > Stream Processing > Seattle - Scalable Stream Processing with Apache Samza > > On Thu, Jan 9, 2020 at 1:23 AM Yi Pan wrote: > > > Hi, all, > > > > Another time to report our project status. I have a draft

Re: [VOTE] SEP 25: PR Title and Description Guidelines

2019-12-18 Thread Yi Pan
+1 (binding) On Wed, Dec 18, 2019 at 10:49 AM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > +1 (non-binding). > > Thanks, > Bharath > > On Wed, Dec 18, 2019 at 10:42 AM Prateek Maheshwari > wrote: > > > Hi folks, > > > > This is a call for a vote on SEP 25: PR Title and

Re: [DISCUSS] SEP 25: PR Title and Description Guidelines

2019-12-16 Thread Yi Pan
+1 (binding). lgtm. Thanks! -Yi On Mon, Dec 16, 2019 at 8:08 AM Daniel Nishimura wrote: > +1. Thank Prateek for standardizing the PR process better. > > On Sun, Dec 15, 2019 at 10:55 PM Bharath Kumara Subramanian < > codin.mart...@gmail.com> wrote: > > > +1. Template looks good to me. > > It

Re: [VOTE] SEP-23: Simplify Job Runner

2019-12-11 Thread Yi Pan
+1(binding). Thanks! -Yi On Wed, Dec 11, 2019 at 12:13 PM Prateek Maheshwari wrote: > +1 (binding). > > Thanks, > Prateek > > On Wed, Dec 11, 2019 at 11:50 AM Xinyu Liu wrote: > > > +1 (binding). This proposal will help future split deployment as well as > > make the deployment simple.

Re: [VOTE] Apache Samza 1.3.0 RC2

2019-12-01 Thread Yi Pan
+1 (binding), verified the signature, built and local integration tests passed. Thanks! -Yi On Wed, Nov 27, 2019 at 2:49 PM Hai Lu wrote: > Hi, > > This is a call for a vote on a release of Apache Samza 1.3.0. Thanks to > everyone who has contributed to this release. > > The release candidate

[REPORT] Samza - November 2019

2019-11-10 Thread Yi Pan
## Description: - Apache Samza is a distributed stream processing engine that are highly configurable to process events from various data sources, including real-time messaging system (e.g. Kafka) and distributed file systems (e.g. HDFS). ## Issues: - No issues requires board attention ##

Re: [VOTE] SEP-20: Samza on Kubernetes

2019-11-08 Thread Yi Pan
+1 (binding) On Thu, Nov 7, 2019 at 7:38 PM Jagadish Venkatraman wrote: > +1 binding. > > Thanks Weiqing for driving this! > > On Thursday, November 7, 2019, Xinyu Liu wrote: > > > +1 (binding). > > > > Thanks, > > Xinyu > > > > On Thu, Nov 7, 2019 at 10:50 AM Weiqing Yang > > wrote: > > > >

Re: [DISCUSS] SEP-20: Samza on Kubernetes

2019-11-05 Thread Yi Pan
+1! Great to see this coming through! -Yi On Mon, Nov 4, 2019 at 9:20 PM Jagadish Venkatraman wrote: > +1, look forward to Samza K8s integration :) > > On Monday, November 4, 2019, Xinyu Liu wrote: > > > +1 on the design. This is a great feature to allow Samza to expand its > > deployment to

Re: Questions about using custom groupers

2019-09-30 Thread Yi Pan
HI, Malcolm, The configuration should be in *_coordinator_* topic. If your configuration of cleanup policy for this topic is compact only, you should not lose the configuration. If your configuration on this topic is a combination of compact + time retention (i.e. newer Kafka version on the

[REPORT] Samza - July 2019

2019-07-12 Thread Yi Pan (Data Infrastructure)
## Description: - Apache Samza is a distributed stream processing engine that are highly configurable to process events from various data sources, including real-time messaging system (e.g. Kafka) and distributed file systems (e.g. HDFS). ## Issues: - No issues requires board attention ##

Draft July report for Samza

2019-07-11 Thread Yi Pan
Hi, all, Here is the draft report I had for Samza this quarter. Please let me know if I miss anything. Thanks! ## Description: - Apache Samza is a distributed stream processing engine that are highly configurable to process events from various data sources, including real-time messaging

Re: [ANNOUNCE] Welcoming Rayman and Bharath as new Samza committers!

2019-07-09 Thread Yi Pan
Big congrats to Ray and Bharath! Looking forward to more PRs from you guys! -Yi On Tue, Jul 9, 2019 at 9:54 AM Weiqing Yang wrote: > Congrats Rayman and Bharath! > > - Weiqing > > On Mon, Jul 8, 2019 at 10:55 PM Yang Zhang wrote: > > > Congratulations! > > > > Best, > > Yang > > > > On Mon,

Re: Tracing the Samza+YARN startup process

2019-06-19 Thread Yi Pan
Great and detailed report! Really appreciate it! -Yi On Tue, Jun 18, 2019 at 2:37 PM Malcolm McFarland wrote: > Just want to follow up on this, for anybody that might be trying to do > something similar. > > There are two things that were getting in the way of us using YARN+Samza on > ECS: 1)

Re: Apache Samza 1.2.0 is released

2019-06-12 Thread Yi Pan
Thanks, Boris! -Yi On Wed, Jun 12, 2019 at 1:17 PM Boris Shkolnik wrote: > Documentation and Blog are published. > Thanks everyone. >

Re: REMINDER. [VOTE] Apache Samza 1.2.0 RC4

2019-06-10 Thread Yi Pan
+1 (binding), verified signature and built successfully. One more: can we make sure that we address SAMZA-2064? The Infra team has mentioned this issue multiple times in the feedbacks to our report. Thanks! -Yi On Thu, Jun 6, 2019 at 5:59 PM Jagadish Venkatraman wrote: > +1 (binding) > >

Re: [ANNOUNCE] Please welcome Boris Shkolnik to the Samza PMC

2019-06-09 Thread Yi Pan
Welcome and well deserved, Boris! -Yi On Sat, Jun 8, 2019 at 3:28 PM Hai Lu wrote: > Congratulations, Boris! > > On Fri, Jun 7, 2019 at 6:13 PM Aditya wrote: > > > Congrats Boris! > > > > > On Jun 7, 2019, at 4:58 PM, Weiqing Yang > > wrote: > > > > > > Congrats, Boris! > > > > > > On Fri,

Re: [DISCUSS] Hygene for merging PRs

2019-05-19 Thread Yi Pan
Hi, Cameron, That's generally the case. Thanks for Xinyu to bring this to our attention! +1 to the stated guidelines. -Yi On Fri, May 17, 2019 at 11:10 AM Cameron Lee wrote: > Thanks Xinyu for starting this thread. > I support the guidelines that you mentioned, with a couple clarifications >

Re: [ANNOUNCE] New committer announcement: Cameron Lee

2019-04-17 Thread Yi Pan
Congrats Cameron!!! On Wed, Apr 17, 2019 at 6:12 AM Daniel Nishimura wrote: > Congrats! > > Sent from my iPhone > > > On Apr 16, 2019, at 9:22 PM, Jagadish Venkatraman < > jagadish1...@gmail.com> wrote: > > > > Awesome addition! Congrats Cameron. well deserved. > > > >> On Tuesday, April 16,

Fwd: [REPORT] Samza - April 2019

2019-04-11 Thread Yi Pan
-- Forwarded message - From: Yi Pan (Data Infrastructure) Date: Thu, Apr 11, 2019 at 9:16 PM Subject: [REPORT] Samza - April 2019 To: Cc: ## Description: - Apache Samza is a distributed stream processing engine that are highly configurable to process events from various data

Re: Context when Converting from Yarn Task to Standalone LocalApplicationRunner

2019-04-03 Thread Yi Pan
Hi, Jeremiah, In the new apis, you should be using ApplicationContainerContextFactory and ApplicationTaskContextFactory to instantiate context objects used in the whole container or in a task instance, respectively. The context factories should be implemented as dependencies injected to your

Re: Trouble running samza 0.14 in standalone mode

2019-03-07 Thread Yi Pan
ring logs and code snippets. > > Issue is resolved. I was using the default task.name.grouper.factory > instead of GroupByContainerIdsFactory. > Appreciate and thanks for the response. I will consider using 1.0 version. > > -anoop > > > On Thu, 7 Mar 2019 at 01:41, Yi P

Re: [DISCUSS] Samza 1.1.0 release

2019-03-06 Thread Yi Pan
+1 (binding) On Wed, Mar 6, 2019 at 10:08 PM Daniel Chen wrote: > Hello everyone, > > We have added couple of major features to master since 1.0.0 that warrants > a major release. > > Within LinkedIn, some of these features have already been tested as part of > our test suites. We plan to

Re: Trouble running samza 0.14 in standalone mode

2019-03-06 Thread Yi Pan
Hi, Anoop, 1. Please provide the full log file if possible. Just listing out a single log line reporting the failure does not help. 2. Is there any reason that you still stay with Samza 0.14? I would highly recommend to upgrade to 1.0 since there are tons of API and standalone related

Re: code cleanups

2019-02-04 Thread Yi Pan
Hi, Andrey, Thanks for bring this up. See my response below: On Mon, Feb 4, 2019 at 4:54 AM Andrey Paykin wrote: > > 1. Is this idea make sense (code cleanup) > Yes, totally make sense. > 2. Do I need JIRA ticket(s)? May be there is some example ticket or > additional requirements? > I

Re: [VOTE] Migration of Samza git repo to gitbox.apache.org

2019-01-24 Thread Yi Pan
+1 (binding). Thanks a lot! On Thu, Jan 24, 2019 at 6:02 PM Boris S wrote: > +1. Thanks! > > On Thu, Jan 24, 2019 at 8:53 AM santhosh venkat < > santhoshvenkat1...@gmail.com> wrote: > > > +1 (non-binding). > > > > On Thu, Jan 24, 2019 at 7:10 AM Jake Maes wrote: > > > > > +1 (binding) > > > >

Re: [DISCUSS] Mandatory migration of Samza git repo to gitbox.apache.org

2019-01-15 Thread Yi Pan
+1 (binding) for the move as well. @Jake Maes I think our merge script requires setting up a local alias 'apache-samza' for the remote repo. Hence, it should be a simple step to point the 'apache-samza' repo to the new repo on gitbox, hopefully. -Yi On Tue, Jan 15, 2019 at 12:12 PM Jake Maes

[REPORT] Samza - January 2019

2019-01-10 Thread Yi Pan
## Description: - Apache Samza is a distributed stream processing engine that are highly configurable to process events from various data sources, including real-time messaging system (e.g. Kafka) and distributed file systems (e.g. HDFS). ## Issues: - No issues requires board attention

Re: Draft report to board - Jan 2019

2019-01-09 Thread Yi Pan
ce this was an intern project and is not checked in to > master. Otherwise LGTM. > > Thanks, > Prateek > > On Wed, Jan 9, 2019 at 12:28 PM Yi Pan wrote: > > > > Hi, all, > > > > Our quarterly report is due this Wed (1/9). The following is the draft > > repo

Draft report to board - Jan 2019

2019-01-09 Thread Yi Pan
Hi, all, Our quarterly report is due this Wed (1/9). The following is the draft report. Please let me know by the end of the day if I missed anything. Thanks! ## Description: - Apache Samza is a distributed stream processing engine that are highly configurable to process events from

  1   2   3   4   5   6   7   8   >