Re: [Draft] Samza quarterly report
lgtm, thanks Yi. On Thu, Jan 9, 2020 at 6:46 PM Prateek Maheshwari wrote: > Looks good to me, thanks! > > - Prateek > > On Thu, Jan 9, 2020 at 1:43 PM Yi Pan wrote: > > > ## Description: > > - Apache Samza is a distributed stream processing engine that are highly > > configurable to process events from various data sources, including > > real-time messaging system (e.g. Kafka) and distributed file systems > > (e.g. > > HDFS). > > > > ## Issues: > > - No issues require board attention > > > > ## Membership Data: > > Apache Samza was founded 2015-01-22 (5 years ago) > > There are currently 26 committers and 16 PMC members in this project. > > The Committer-to-PMC ratio is roughly 7:4. > > > > Community changes, past quarter: > > - No new PMC members. Last addition was Boris Shkolnik on 2019-06-06. > > - No new committers. Last addition was Rayman Preet Singh on 2019-07-08. > > > > ## Project Activity: > > - New version 1.3 was released on 12/05/2019 > > - New features via SEPs (i.e. Samza Enhancement Proposals) are proposed > > continuously. > > In the last quarter, there are 4 new SEPs. > > > > ## Community Health: > > - We continue engage with new users via the Q on dev email lists. > > - We have Samza talks in many Conferences: > > Strange Loop - Riding the Stream Processing Wave > > Apache Beam Summit (Berlin) - Streaming Pipelines at Scale with > Apache > > Beam and Samza > > ApacheCon North America - Samza 1.0: How we scaled stream processing > at > > LinkedIn > > ApacheCon North America - Samza Portable Runner for Beam > > KubeCon North America - Running Apache Samza on Kubernetes > > - We have organized meetups with the following Samza Talks: > > Sunnyvale - Stream Processing in Python with Samza and Beam > > Sunnyvale - Apache Samza 1.0: Recent Advances and our plans for > future > > in Stream Processing > > Seattle - Scalable Stream Processing with Apache Samza > > > > > > P.S. just fixing one typo. > > > > -Yi > > > > On Thu, Jan 9, 2020 at 1:42 PM Yi Pan wrote: > > > > > ## Description: > > > - Apache Samza is a distributed stream processing engine that are > highly > > > configurable to process events from various data sources, including > > > real-time messaging system (e.g. Kafka) and distributed file systems > > > (e.g. > > > HDFS). > > > > > > ## Issues: > > > - No issues requires board attention > > > > > > ## Membership Data: > > > Apache Samza was founded 2015-01-22 (5 years ago) > > > There are currently 26 committers and 16 PMC members in this project. > > > The Committer-to-PMC ratio is roughly 7:4. > > > > > > Community changes, past quarter: > > > - No new PMC members. Last addition was Boris Shkolnik on 2019-06-06. > > > - No new committers. Last addition was Rayman Preet Singh on > 2019-07-08. > > > > > > ## Project Activity: > > > - New version 1.3 was released on 12/05/2019 > > > - New features via SEPs (i.e. Samza Enhancement Proposals) are proposed > > > continuously. > > > In the last quarter, there are 4 new SEPs. > > > > > > ## Community Health: > > > - We continue engage with new users via the Q on dev email lists. > > > - We have Samza talks in many Conferences: > > > Strange Loop - Riding the Stream Processing Wave > > > Apache Beam Summit (Berlin) - Streaming Pipelines at Scale with > > Apache > > > Beam and Samza > > > ApacheCon North America - Samza 1.0: How we scaled stream > processing > > > at LinkedIn > > > ApacheCon North America - Samza Portable Runner for Beam > > > KubeCon North America - Running Apache Samza on Kubernetes > > > - We have organized meetups with the following Samza Talks: > > > Sunnyvale - Stream Processing in Python with Samza and Beam > > > Sunnyvale - Apache Samza 1.0: Recent Advances and our plans for > > future > > > in Stream Processing > > > Seattle - Scalable Stream Processing with Apache Samza > > > > > > If the above report looks good, I will submit today. > > > > > > Thanks a lot! > > > > > > -Yi > > > > > > On Thu, Jan 9, 2020 at 10:23 AM Prateek Maheshwari < > prateek...@gmail.com > > > > > > wrote: > > > > > >> Thanks for preparing this Yi. We had the following Samza talks and > > meetups > > >> in 2019. Let's highlight them under Community Health: > > >> > > >> Conferences: > > >> Strange Loop - Riding the Stream Processing Wave > > >> Apache Beam Summit (Berlin) - Streaming Pipelines at Scale with Apache > > >> Beam > > >> and Samza > > >> ApacheCon North America - Samza 1.0: How we scaled stream processing > at > > >> LinkedIn > > >> ApacheCon North America - Samza Portable Runner for Beam > > >> KubeCon North America - Running Apache Samza on Kubernetes > > >> > > >> Meetup Talks: > > >> Sunnyvale - Stream Processing in Python with Samza and Beam > > >> Sunnyvale - Apache Samza 1.0: Recent Advances and our plans for future > > in > > >> Stream Processing > > >> Seattle - Scalable Stream Processing with Apache Samza > > >> > > >> On Thu,
Re: [Draft] Samza quarterly report
Looks good to me, thanks! - Prateek On Thu, Jan 9, 2020 at 1:43 PM Yi Pan wrote: > ## Description: > - Apache Samza is a distributed stream processing engine that are highly > configurable to process events from various data sources, including > real-time messaging system (e.g. Kafka) and distributed file systems > (e.g. > HDFS). > > ## Issues: > - No issues require board attention > > ## Membership Data: > Apache Samza was founded 2015-01-22 (5 years ago) > There are currently 26 committers and 16 PMC members in this project. > The Committer-to-PMC ratio is roughly 7:4. > > Community changes, past quarter: > - No new PMC members. Last addition was Boris Shkolnik on 2019-06-06. > - No new committers. Last addition was Rayman Preet Singh on 2019-07-08. > > ## Project Activity: > - New version 1.3 was released on 12/05/2019 > - New features via SEPs (i.e. Samza Enhancement Proposals) are proposed > continuously. > In the last quarter, there are 4 new SEPs. > > ## Community Health: > - We continue engage with new users via the Q on dev email lists. > - We have Samza talks in many Conferences: > Strange Loop - Riding the Stream Processing Wave > Apache Beam Summit (Berlin) - Streaming Pipelines at Scale with Apache > Beam and Samza > ApacheCon North America - Samza 1.0: How we scaled stream processing at > LinkedIn > ApacheCon North America - Samza Portable Runner for Beam > KubeCon North America - Running Apache Samza on Kubernetes > - We have organized meetups with the following Samza Talks: > Sunnyvale - Stream Processing in Python with Samza and Beam > Sunnyvale - Apache Samza 1.0: Recent Advances and our plans for future > in Stream Processing > Seattle - Scalable Stream Processing with Apache Samza > > > P.S. just fixing one typo. > > -Yi > > On Thu, Jan 9, 2020 at 1:42 PM Yi Pan wrote: > > > ## Description: > > - Apache Samza is a distributed stream processing engine that are highly > > configurable to process events from various data sources, including > > real-time messaging system (e.g. Kafka) and distributed file systems > > (e.g. > > HDFS). > > > > ## Issues: > > - No issues requires board attention > > > > ## Membership Data: > > Apache Samza was founded 2015-01-22 (5 years ago) > > There are currently 26 committers and 16 PMC members in this project. > > The Committer-to-PMC ratio is roughly 7:4. > > > > Community changes, past quarter: > > - No new PMC members. Last addition was Boris Shkolnik on 2019-06-06. > > - No new committers. Last addition was Rayman Preet Singh on 2019-07-08. > > > > ## Project Activity: > > - New version 1.3 was released on 12/05/2019 > > - New features via SEPs (i.e. Samza Enhancement Proposals) are proposed > > continuously. > > In the last quarter, there are 4 new SEPs. > > > > ## Community Health: > > - We continue engage with new users via the Q on dev email lists. > > - We have Samza talks in many Conferences: > > Strange Loop - Riding the Stream Processing Wave > > Apache Beam Summit (Berlin) - Streaming Pipelines at Scale with > Apache > > Beam and Samza > > ApacheCon North America - Samza 1.0: How we scaled stream processing > > at LinkedIn > > ApacheCon North America - Samza Portable Runner for Beam > > KubeCon North America - Running Apache Samza on Kubernetes > > - We have organized meetups with the following Samza Talks: > > Sunnyvale - Stream Processing in Python with Samza and Beam > > Sunnyvale - Apache Samza 1.0: Recent Advances and our plans for > future > > in Stream Processing > > Seattle - Scalable Stream Processing with Apache Samza > > > > If the above report looks good, I will submit today. > > > > Thanks a lot! > > > > -Yi > > > > On Thu, Jan 9, 2020 at 10:23 AM Prateek Maheshwari > > > wrote: > > > >> Thanks for preparing this Yi. We had the following Samza talks and > meetups > >> in 2019. Let's highlight them under Community Health: > >> > >> Conferences: > >> Strange Loop - Riding the Stream Processing Wave > >> Apache Beam Summit (Berlin) - Streaming Pipelines at Scale with Apache > >> Beam > >> and Samza > >> ApacheCon North America - Samza 1.0: How we scaled stream processing at > >> LinkedIn > >> ApacheCon North America - Samza Portable Runner for Beam > >> KubeCon North America - Running Apache Samza on Kubernetes > >> > >> Meetup Talks: > >> Sunnyvale - Stream Processing in Python with Samza and Beam > >> Sunnyvale - Apache Samza 1.0: Recent Advances and our plans for future > in > >> Stream Processing > >> Seattle - Scalable Stream Processing with Apache Samza > >> > >> On Thu, Jan 9, 2020 at 1:23 AM Yi Pan wrote: > >> > >> > Hi, all, > >> > > >> > Another time to report our project status. I have a draft below and > >> would > >> > like input from the community to fill in some more details: > >> > > >> > ## Description: > >> > - Apache Samza is a distributed stream processing engine that are > highly > >> > configurable to process events
Re: [VOTE] SEP-26: Add SystemProducer for Azure Blob Storage
+1 (binding). Good to see more cloud native integrations in Samza. -Yi On Wed, Jan 8, 2020 at 10:31 AM Prateek Maheshwari wrote: > +1 (binding). Thanks for the contribution. > > - Prateek > > On Tue, Jan 7, 2020 at 7:59 PM Jagadish Venkatraman < > jagadish1...@gmail.com> > wrote: > > > +1 (binding), looking forward to Samza's integration with Azure blobs > > > > On Wednesday, January 8, 2020, Lakshmi Manasa > > > wrote: > > > > > Hi, > > > > > > This is a call for a vote on SEP-26: Add SystemProducer for Azure Blob > > > Storage. > > > Thanks for taking a look and giving feedback. > > > > > > I have addressed the comments on the SEP and since there were no major > > > questions/objections, starting this vote. > > > > > > Discussion thread: > > > http://mail-archives.apache.org/mod_mbox/samza-dev/202001. > > > mbox/%3CCAEwD47cW2T24C9A_tzj7Qxuv3P%2B2an47GkmaA4- > > > 41WZfvY_vgw%40mail.gmail.com%3E > > > > > > SEP: > > > https://cwiki.apache.org/confluence/display/SAMZA/SEP- > > > 26%3A+Azure+Blob+Storage+Producer > > > > > > Please vote: > > > > > > [ ] +1 approve > > > > > > [ ] +0 no opinion > > > > > > [ ] -1 disapprove (and reason why) > > > > > > Thanks, > > > Manasa > > > > > > > > > -- > > Jagadish > > >
[GitHub] [samza-hello-samza] kw2542 opened a new pull request #69: Update POM to 1.4.0-SNAPSHOT as samza 1.3 has been published
kw2542 opened a new pull request #69: Update POM to 1.4.0-SNAPSHOT as samza 1.3 has been published URL: https://github.com/apache/samza-hello-samza/pull/69 In order to be compatible with Hello Samza documentation, POM needs to be updated to 1.4.0-SNAPSHOT for latest branch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
Re: [Draft] Samza quarterly report
## Description: - Apache Samza is a distributed stream processing engine that are highly configurable to process events from various data sources, including real-time messaging system (e.g. Kafka) and distributed file systems (e.g. HDFS). ## Issues: - No issues require board attention ## Membership Data: Apache Samza was founded 2015-01-22 (5 years ago) There are currently 26 committers and 16 PMC members in this project. The Committer-to-PMC ratio is roughly 7:4. Community changes, past quarter: - No new PMC members. Last addition was Boris Shkolnik on 2019-06-06. - No new committers. Last addition was Rayman Preet Singh on 2019-07-08. ## Project Activity: - New version 1.3 was released on 12/05/2019 - New features via SEPs (i.e. Samza Enhancement Proposals) are proposed continuously. In the last quarter, there are 4 new SEPs. ## Community Health: - We continue engage with new users via the Q on dev email lists. - We have Samza talks in many Conferences: Strange Loop - Riding the Stream Processing Wave Apache Beam Summit (Berlin) - Streaming Pipelines at Scale with Apache Beam and Samza ApacheCon North America - Samza 1.0: How we scaled stream processing at LinkedIn ApacheCon North America - Samza Portable Runner for Beam KubeCon North America - Running Apache Samza on Kubernetes - We have organized meetups with the following Samza Talks: Sunnyvale - Stream Processing in Python with Samza and Beam Sunnyvale - Apache Samza 1.0: Recent Advances and our plans for future in Stream Processing Seattle - Scalable Stream Processing with Apache Samza P.S. just fixing one typo. -Yi On Thu, Jan 9, 2020 at 1:42 PM Yi Pan wrote: > ## Description: > - Apache Samza is a distributed stream processing engine that are highly > configurable to process events from various data sources, including > real-time messaging system (e.g. Kafka) and distributed file systems > (e.g. > HDFS). > > ## Issues: > - No issues requires board attention > > ## Membership Data: > Apache Samza was founded 2015-01-22 (5 years ago) > There are currently 26 committers and 16 PMC members in this project. > The Committer-to-PMC ratio is roughly 7:4. > > Community changes, past quarter: > - No new PMC members. Last addition was Boris Shkolnik on 2019-06-06. > - No new committers. Last addition was Rayman Preet Singh on 2019-07-08. > > ## Project Activity: > - New version 1.3 was released on 12/05/2019 > - New features via SEPs (i.e. Samza Enhancement Proposals) are proposed > continuously. > In the last quarter, there are 4 new SEPs. > > ## Community Health: > - We continue engage with new users via the Q on dev email lists. > - We have Samza talks in many Conferences: > Strange Loop - Riding the Stream Processing Wave > Apache Beam Summit (Berlin) - Streaming Pipelines at Scale with Apache > Beam and Samza > ApacheCon North America - Samza 1.0: How we scaled stream processing > at LinkedIn > ApacheCon North America - Samza Portable Runner for Beam > KubeCon North America - Running Apache Samza on Kubernetes > - We have organized meetups with the following Samza Talks: > Sunnyvale - Stream Processing in Python with Samza and Beam > Sunnyvale - Apache Samza 1.0: Recent Advances and our plans for future > in Stream Processing > Seattle - Scalable Stream Processing with Apache Samza > > If the above report looks good, I will submit today. > > Thanks a lot! > > -Yi > > On Thu, Jan 9, 2020 at 10:23 AM Prateek Maheshwari > wrote: > >> Thanks for preparing this Yi. We had the following Samza talks and meetups >> in 2019. Let's highlight them under Community Health: >> >> Conferences: >> Strange Loop - Riding the Stream Processing Wave >> Apache Beam Summit (Berlin) - Streaming Pipelines at Scale with Apache >> Beam >> and Samza >> ApacheCon North America - Samza 1.0: How we scaled stream processing at >> LinkedIn >> ApacheCon North America - Samza Portable Runner for Beam >> KubeCon North America - Running Apache Samza on Kubernetes >> >> Meetup Talks: >> Sunnyvale - Stream Processing in Python with Samza and Beam >> Sunnyvale - Apache Samza 1.0: Recent Advances and our plans for future in >> Stream Processing >> Seattle - Scalable Stream Processing with Apache Samza >> >> On Thu, Jan 9, 2020 at 1:23 AM Yi Pan wrote: >> >> > Hi, all, >> > >> > Another time to report our project status. I have a draft below and >> would >> > like input from the community to fill in some more details: >> > >> > ## Description: >> > - Apache Samza is a distributed stream processing engine that are highly >> > configurable to process events from various data sources, including >> > real-time messaging system (e.g. Kafka) and distributed file systems >> > (e.g. >> > HDFS). >> > >> > ## Issues: >> > - No issues requires board attention >> > >> > ## Membership Data: >> > Apache Samza was founded 2015-01-22 (5 years ago) >> > There are currently 26 committers and 16 PMC members in
Re: [Draft] Samza quarterly report
## Description: - Apache Samza is a distributed stream processing engine that are highly configurable to process events from various data sources, including real-time messaging system (e.g. Kafka) and distributed file systems (e.g. HDFS). ## Issues: - No issues requires board attention ## Membership Data: Apache Samza was founded 2015-01-22 (5 years ago) There are currently 26 committers and 16 PMC members in this project. The Committer-to-PMC ratio is roughly 7:4. Community changes, past quarter: - No new PMC members. Last addition was Boris Shkolnik on 2019-06-06. - No new committers. Last addition was Rayman Preet Singh on 2019-07-08. ## Project Activity: - New version 1.3 was released on 12/05/2019 - New features via SEPs (i.e. Samza Enhancement Proposals) are proposed continuously. In the last quarter, there are 4 new SEPs. ## Community Health: - We continue engage with new users via the Q on dev email lists. - We have Samza talks in many Conferences: Strange Loop - Riding the Stream Processing Wave Apache Beam Summit (Berlin) - Streaming Pipelines at Scale with Apache Beam and Samza ApacheCon North America - Samza 1.0: How we scaled stream processing at LinkedIn ApacheCon North America - Samza Portable Runner for Beam KubeCon North America - Running Apache Samza on Kubernetes - We have organized meetups with the following Samza Talks: Sunnyvale - Stream Processing in Python with Samza and Beam Sunnyvale - Apache Samza 1.0: Recent Advances and our plans for future in Stream Processing Seattle - Scalable Stream Processing with Apache Samza If the above report looks good, I will submit today. Thanks a lot! -Yi On Thu, Jan 9, 2020 at 10:23 AM Prateek Maheshwari wrote: > Thanks for preparing this Yi. We had the following Samza talks and meetups > in 2019. Let's highlight them under Community Health: > > Conferences: > Strange Loop - Riding the Stream Processing Wave > Apache Beam Summit (Berlin) - Streaming Pipelines at Scale with Apache Beam > and Samza > ApacheCon North America - Samza 1.0: How we scaled stream processing at > LinkedIn > ApacheCon North America - Samza Portable Runner for Beam > KubeCon North America - Running Apache Samza on Kubernetes > > Meetup Talks: > Sunnyvale - Stream Processing in Python with Samza and Beam > Sunnyvale - Apache Samza 1.0: Recent Advances and our plans for future in > Stream Processing > Seattle - Scalable Stream Processing with Apache Samza > > On Thu, Jan 9, 2020 at 1:23 AM Yi Pan wrote: > > > Hi, all, > > > > Another time to report our project status. I have a draft below and would > > like input from the community to fill in some more details: > > > > ## Description: > > - Apache Samza is a distributed stream processing engine that are highly > > configurable to process events from various data sources, including > > real-time messaging system (e.g. Kafka) and distributed file systems > > (e.g. > > HDFS). > > > > ## Issues: > > - No issues requires board attention > > > > ## Membership Data: > > Apache Samza was founded 2015-01-22 (5 years ago) > > There are currently 26 committers and 16 PMC members in this project. > > The Committer-to-PMC ratio is roughly 7:4. > > > > Community changes, past quarter: > > - No new PMC members. Last addition was Boris Shkolnik on 2019-06-06. > > - No new committers. Last addition was Rayman Preet Singh on 2019-07-08. > > > > ## Project Activity: > > - New version 1.3 was released on 12/05/2019 > > *- [please add related project activities you know here]* > > > > ## Community Health: > > - The community is actively pushing new features via SEPs (i.e. Samza > > Enhancement Proposals). In the last quarter, there are 4 new SEPs. > > - We continue engage with new users via the Q on dev email lists. > > *- [please add examples of community health indicators here, like new > > companies/users, new meetups/talks, new initiatives proposed and > > in-progress etc.]* > > >
Re: [Draft] Samza quarterly report
Thanks for preparing this Yi. We had the following Samza talks and meetups in 2019. Let's highlight them under Community Health: Conferences: Strange Loop - Riding the Stream Processing Wave Apache Beam Summit (Berlin) - Streaming Pipelines at Scale with Apache Beam and Samza ApacheCon North America - Samza 1.0: How we scaled stream processing at LinkedIn ApacheCon North America - Samza Portable Runner for Beam KubeCon North America - Running Apache Samza on Kubernetes Meetup Talks: Sunnyvale - Stream Processing in Python with Samza and Beam Sunnyvale - Apache Samza 1.0: Recent Advances and our plans for future in Stream Processing Seattle - Scalable Stream Processing with Apache Samza On Thu, Jan 9, 2020 at 1:23 AM Yi Pan wrote: > Hi, all, > > Another time to report our project status. I have a draft below and would > like input from the community to fill in some more details: > > ## Description: > - Apache Samza is a distributed stream processing engine that are highly > configurable to process events from various data sources, including > real-time messaging system (e.g. Kafka) and distributed file systems > (e.g. > HDFS). > > ## Issues: > - No issues requires board attention > > ## Membership Data: > Apache Samza was founded 2015-01-22 (5 years ago) > There are currently 26 committers and 16 PMC members in this project. > The Committer-to-PMC ratio is roughly 7:4. > > Community changes, past quarter: > - No new PMC members. Last addition was Boris Shkolnik on 2019-06-06. > - No new committers. Last addition was Rayman Preet Singh on 2019-07-08. > > ## Project Activity: > - New version 1.3 was released on 12/05/2019 > *- [please add related project activities you know here]* > > ## Community Health: > - The community is actively pushing new features via SEPs (i.e. Samza > Enhancement Proposals). In the last quarter, there are 4 new SEPs. > - We continue engage with new users via the Q on dev email lists. > *- [please add examples of community health indicators here, like new > companies/users, new meetups/talks, new initiatives proposed and > in-progress etc.]* >
Re: Problem : upgrade 1.2 to 1.3 LocalApplicationRunner + default system in SamzaApplication.describe results in null MetadataStoreFactory
In case someone else runs into this, we had to change the constructor we
used for the LocalApplicationRunner, and explicitly pass in the " new
CoordinatorStreamMetadataStoreFactory()'.
On Mon, Jan 6, 2020 at 3:18 PM Thunder Stumpges wrote:
> Hey dev team. Just upgrading our stand alone low-level tasks to Samza 1.3.
> We use the LocalApplicationRunner and initialize most of our application
> within SamzaApplication.describe() including setting up
> "withDefaultSystem(systemDescriptor)"
>
> However it seems that earlier on in the process, the
> LocalApplicationRunner constructor is calling
> "getDefaultCoordinatorStreamStoreFactory" to pick the MetadataStoreFactory,
> and the "coordinatorSystemName" is not set yet (nor is it ever set
> explicitly now that we use the "default system").
>
> The condition that is failing is in LocalApplicationRunner line 138:
>
>
> // TODO: Remove restriction to only ZkJobCoordinator after next phase of
> metadata store abstraction.
> if (StringUtils.isNotBlank(coordinatorSystemName) &&
> ZkJobCoordinatorFactory.class.getName().equals(jobCoordinatorFactoryClassName))
> {
> return new CoordinatorStreamMetadataStoreFactory();
> }
>
> coordinatorSystemName is null above. (jobCoordinatorFactoryClassName IS
> set to ZkJobCoordinatorFactory)
>
> Please advise!
> Thanks,
> Thunder
>
>
