Re: [DRAFT] [UPDATED] Graduation resolution proposal
Dave / mentors, since this thread was not marked with [VOTE], would we need to have a separate vote thread or do you believe this one is enough to proceed? Thanks, Matteo On Tue, Sep 4, 2018 at 12:56 AM Jia Zhai wrote: > +1. > > I am honored to be asked to be on the PMC. Please keep me on the list. ：） > > On Tue, Sep 4, 2018 at 2:47 PM Hiroyuki Sakai > wrote: > > > +1 > > > > I would like to be part of PMC. > > > > Hiroyuki > > > > > > -Original Message- > > From: Masahiro Sakamoto > > Reply-To: "dev@pulsar.incubator.apache.org" < > > dev@pulsar.incubator.apache.org> > > Date: Tuesday, September 4, 2018 15:43 > > To: "dev@pulsar.incubator.apache.org" > > Subject: RE: [DRAFT] [UPDATED] Graduation resolution proposal > > > > +1 > > > > I would like to be part of PMC. > > > > -- > > Masahiro Sakamoto > > Yahoo Japan Corp. > > E-mail: massa...@yahoo-corp.jp > > -- > > > > > -Original Message- > > > From: Nozomi Kurihara [mailto:nkuri...@yahoo-corp.jp] > > > Sent: Tuesday, September 04, 2018 2:27 PM > > > To: dev@pulsar.incubator.apache.org > > > Subject: RE: [DRAFT] [UPDATED] Graduation resolution proposal > > > > > > +1 > > > > > > And would like to be part of PMC. > > > > > > > > > Nozomi > > > > > > > > > 差出人: Ivan Kelly > > > 送信日時: 2018年9月3日 16:38:41 > > > 宛先: dev@pulsar.incubator.apache.org > > > 件名: Re: [DRAFT] [UPDATED] Graduation resolution proposal > > > > > > +1 > > > > > > I wish to remain part of PMC. > > > > > > Regarding BookKeeper bylaws, BookKeeper has them because ZooKeeper > > had them > > > when we branched off, and my understanding is that zookeeper had > > them, > > > because hadoop had them when they branched off. As far as I'm > > concerned, > > > they're unnecessary given the default asf bylaws, but that's a > > discussion > > > for another list. > > > > > > -Ivan > > > > > > On Sun, Sep 2, 2018 at 10:36 PM, P. Taylor Goetz < > ptgo...@gmail.com> > > wrote: > > > > +1 > > > > > > > > -Taylor > > > > > > > >> On Sep 2, 2018, at 3:18 PM, Matteo Merli > > wrote: > > > >> > > > >> Updated the initial draft by removing paragraph on the "have a > > > >> tasking for creating set of bylaws". > > > >> > > > >> > > > >> > > > >> Establish the Apache Pulsar Project > > > >> > > > >> WHEREAS, the Board of Directors deems it to be in the best > > interests > > > >> of the Foundation and consistent with the Foundation's purpose > to > > > >> establish a Project Management Committee charged with the > creation > > > >> and maintenance of open-source software, for distribution at no > > > >> charge to the public, related to a highly scalable, low latency > > > >> messaging platform running on commodity hardware. It provides > > simple > > > >> pub-sub and queue semantics over topics, lightweight compute > > > >> framework, automatic cursor management for subscribers, and > > > cross-datacenter replication. > > > >> > > > >> NOW, THEREFORE, BE IT RESOLVED, that a Project Management > > Committee > > > >> (PMC), to be known as the "Apache Pulsar Project", be and hereby > > is > > > >> established pursuant to Bylaws of the Foundation; and be it > > further > > > >> > > > >> RESOLVED, that the Apache Pulsar Project be and hereby is > > responsible > > > >> for the creation and maintenance of software related to a highly > > > >> scalable, low latency messaging platform running on commodity > > hardware. > > > >> It provides simple pub-sub and queue semantics over topics, > > > >> lightweight compute framework, automatic cursor management for > > > >> subscribers, and cross-datacenter replication; and be it further > > > >> > > > >> RESOLVED, that the office of "Vice President, Apache Pulsar" be > > and > > > >> hereby is created, the person holding such office to serve at > the > > > >> direction of the Board of Directors as the chair of the Apache > > Pulsar > > > >> Project, and to have primary responsibility for management of > the > > > >> projects within the scope of responsibility of the Apache Pulsar > > > >> Project; and be it further > > > >> > > > >> RESOLVED, that the persons listed immediately below be and > hereby > > are > > > >> appointed to serve as the initial members of the Apache Pulsar > > Project: > > > >> > > > >> * Boyang Jerry Peng > > > >> * Brad McMillen > > > >> * David Fisher > > > >> * Francis Christopher Liu > > > >> * Hiroyuki Sakai > > > >> * Ivan Brendan Kelly > > > >> * Jai Asher > > > >> * Jia Zhai > > > >> * Jim Jagielski >
Re: [DRAFT] Podling report
+1 Thanks Matteo. -Taylor > On Sep 5, 2018, at 8:36 PM, Matteo Merli wrote: > > Sorry for sending out at very last minute (deadline is today). Please take > a look. > > I have added a paragraph regarding the branding discussion that happen in > early June (as discussed with Taylor then) since the previous draft was > already out at that point. > > > > --- > > Pulsar > Pulsar is a highly scalable, low latency messaging platform running on > commodity hardware. It provides simple pub-sub semantics over topics, > guaranteed at-least-once delivery of messages, automatic cursor management > for > subscribers, and cross-datacenter replication. > > Pulsar has been incubating since 2017-06-01. > > Most important issues to address in the move towards graduation: > > None > > Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be > aware of? > > Earlier in June there have been few discussions on the private list > regarding communications regarding Pulsar that were not coming from > PPMC or that were not respecting the ASF policies. Clarifications > followed between PPMC members, mentors and interested parties to > ensure the mistakes were made in good faith and, in particular, to > make sure everyone was fully has full understanding of ASF > policies. There was no other branding related issue after the first > occourence. > > How has the community developed since the last report? > > The community added 8 new contributors that submitted pull-requests > which were merged into master. > > The number of users approaching the team on the Slack channel has > kept steadily increasing since the last report. Many users have > actively deployed. Pulsar for evaluation and production use cases. > > Different meetups were organized by project members and hosted by > Yahoo in Sunnyvale and Yahoo Japan in Tokyo. We have presented > Pulsar's introductions, updates on the state of the projects, > deep-dives and hands-on tutorial, including recorded podcasts. > > One talk on Pulsar was presented at one at OSCon in July and there > are several scheduled talks: 2 at ApacheCon in September, and 2 > others at Strata New York in September. > > Since the last report the number of weekly-active-users on the Slack > channel has increased from 53 to 88. > > > How has the project developed since the last report? > > 28 authors have pushed 494 commits to master in the last 3 months. > > The project has made the its seventh release since joining the > Apache Incubator (2.1.0-incubating on Aug 2nd). > > This release introduced these new features: > > * Pulsar IO: A connector framework for moving data in and out of > Apache Pulsar leveraging Pulsar Functions runtime. > * A number of builtin connectors: (Aerospike, Cassandra, Kafka, > Kinesis, RabbitMQ, Twitter) > * Tiered Storage: An extension in Pulsar segment store to offload > older segments into long term storage (e.g. HDFS, S3). S3 support > is supported in 2.1 release. > * Stateful function: Pulsar Functions is able to use State API for > storing state within Pulsar. > * Pulsar Go Client > * Avro and Protobuf Schema support > > Community is actively working on a bug-fix release > (2.1.1-incubating) and on the next milestone, 2.2 release for which > the biggest feature will be support for SQL within Pulsar. > > Since June, 5 new PIPs (Pulsar Improvement Proposals) for > major feature/changes, have been submitted to the wiki and > discussed in the mailing list. > >PIP 23: Message Tracing By Interceptors >PIP 22: Pulsar Dead Letter Topic >PIP 21: Pulsar Edge Component >PIP 20: Mechanism to revoke TLS authentication >PIP 19: Pulsar SQL > > > How would you assess the podling's maturity? > Please feel free to add your own commentary. > > [ ] Initial setup > [ ] Working towards first release > [ ] Community building > [X] Nearing graduation > [ ] Other: > > Date of last release: > 2018-08-02, 2.1.0-incubating > > When were the last committers or PPMC members elected? > > 2018-06-11 - Ivan Kelly > 2018-06-11 - Jia Zhai > > > -- > Matteo Merli >
[DRAFT] Podling report
Sorry for sending out at very last minute (deadline is today). Please take a look. I have added a paragraph regarding the branding discussion that happen in early June (as discussed with Taylor then) since the previous draft was already out at that point. --- Pulsar Pulsar is a highly scalable, low latency messaging platform running on commodity hardware. It provides simple pub-sub semantics over topics, guaranteed at-least-once delivery of messages, automatic cursor management for subscribers, and cross-datacenter replication. Pulsar has been incubating since 2017-06-01. Most important issues to address in the move towards graduation: None Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? Earlier in June there have been few discussions on the private list regarding communications regarding Pulsar that were not coming from PPMC or that were not respecting the ASF policies. Clarifications followed between PPMC members, mentors and interested parties to ensure the mistakes were made in good faith and, in particular, to make sure everyone was fully has full understanding of ASF policies. There was no other branding related issue after the first occourence. How has the community developed since the last report? The community added 8 new contributors that submitted pull-requests which were merged into master. The number of users approaching the team on the Slack channel has kept steadily increasing since the last report. Many users have actively deployed. Pulsar for evaluation and production use cases. Different meetups were organized by project members and hosted by Yahoo in Sunnyvale and Yahoo Japan in Tokyo. We have presented Pulsar's introductions, updates on the state of the projects, deep-dives and hands-on tutorial, including recorded podcasts. One talk on Pulsar was presented at one at OSCon in July and there are several scheduled talks: 2 at ApacheCon in September, and 2 others at Strata New York in September. Since the last report the number of weekly-active-users on the Slack channel has increased from 53 to 88. How has the project developed since the last report? 28 authors have pushed 494 commits to master in the last 3 months. The project has made the its seventh release since joining the Apache Incubator (2.1.0-incubating on Aug 2nd). This release introduced these new features: * Pulsar IO: A connector framework for moving data in and out of Apache Pulsar leveraging Pulsar Functions runtime. * A number of builtin connectors: (Aerospike, Cassandra, Kafka, Kinesis, RabbitMQ, Twitter) * Tiered Storage: An extension in Pulsar segment store to offload older segments into long term storage (e.g. HDFS, S3). S3 support is supported in 2.1 release. * Stateful function: Pulsar Functions is able to use State API for storing state within Pulsar. * Pulsar Go Client * Avro and Protobuf Schema support Community is actively working on a bug-fix release (2.1.1-incubating) and on the next milestone, 2.2 release for which the biggest feature will be support for SQL within Pulsar. Since June, 5 new PIPs (Pulsar Improvement Proposals) for major feature/changes, have been submitted to the wiki and discussed in the mailing list. PIP 23: Message Tracing By Interceptors PIP 22: Pulsar Dead Letter Topic PIP 21: Pulsar Edge Component PIP 20: Mechanism to revoke TLS authentication PIP 19: Pulsar SQL How would you assess the podling's maturity? Please feel free to add your own commentary. [ ] Initial setup [ ] Working towards first release [ ] Community building [X] Nearing graduation [ ] Other: Date of last release: 2018-08-02, 2.1.0-incubating When were the last committers or PPMC members elected? 2018-06-11 - Ivan Kelly 2018-06-11 - Jia Zhai -- Matteo Merli
Re: [discuss] how to label milestone
> The pain comes from issues marked for a bug release milestone but it is > merged directly to master and never cherry-picked to branch for bugfix > releases. > So the release manager has to compare branches and master and also have to > compare labels, milestones and even look into the details of pull requests > to understood whether a change is cherry-picked or not. This is a problem, but it should be 99% automatable. In fact it should be automated, that before cutting a release, that we have a script that validates that all issues labelled for the release have a change present in the commit log, and all commit log messages are referenced by some issue or PR. Gerrit has a change-id that makes this easy, but I'm not sure about github. -Ivan
Re: [discuss] how to label milestone
My motivation is simple - I would like to see a process for managing merging pull requests, add labels and milestone and cherry-picks for releases. Otherwise things just become very messy with different people merging pull requests in different ways. I know there was a discussion about the merge script. but people doesn't like the way how bk merge script closes the pull request. http://mail-archives.apache.org/mod_mbox/pulsar-dev/201708.mbox/%3CCAGjw%2BkO1ucxp%3DKn0aK077kANhiZiVRcL5exhPJPgHkdfa7XSCA%40mail.gmail.com%3E So I changed the merge script to use github pull request merge api on merging a PR. here is the script: https://github.com/apache/incubator-pulsar/pull/2526 This is one example PR how it was merged by the script: https://github.com/sijie/incubator-pulsar/pull/8 Let me know what do you think. - Sijie On Mon, Aug 27, 2018 at 2:45 AM Sijie Guo wrote: > Hi all, > > I think we need to discuss how to label the issues and how to cherry-pick > the issues for bug fixes. Currently it is painful for doing a bugfix > release. > All the bug fixes are deferred to release manager cherry-picking to bug > fix branches. Since master and branches are quickly diverged with changes, > it is very painful for cherry-picks later and there is no clear process > for cherry-pick and labelling. > > Shall we consider following the process what BookKeeper is using, having a > merge script to handle labelling issues/milestones and also cherry-picks? > > Or any suggestions on how to improve this? > > - Sijie > > On Mon, Aug 6, 2018 at 10:50 AM Sijie Guo wrote: > >> Hi all, >> >> It seems we don't have a standard for labelling milestone. sometimes I am >> not lost and confused when I see a change was merged to master but marked >> in a bug fix milestone. so I am raising a discussion about this. >> >> I think every pull request that was created based on master, which would >> be merged as a commit of master, those pull requests should be first marked >> as major release milestone, for example: `2.2.0` release. If a pull request >> that was created based on a branch, which would be merged as a commit of >> that branch, those pull requests should be marked as minor release >> milestones, for example: `2.1.1` release. so that we have a clearer way on >> tracking what changes have been made to what releases. otherwise it is a >> bit hard to track when we have multiple releases ongoing. >> >> Thoughts? >> >> Thanks, >> Sijie >> >>
Re: [discuss] how to label milestone
On Mon, Sep 3, 2018 at 12:14 AM Ivan Kelly wrote: > > All the bug fixes are deferred to release manager cherry-picking to bug > fix > > branches. Since master and branches are quickly diverged with changes, > > it is very painful for cherry-picks later and there is no clear process > for > > cherry-pick and labelling. > > What is painful here? > If it's the conflict resolution, this pain would > just be spread out, with it being felt by whomever is merging, but > then it would also be worse, because cherry-picks would happen out of > order. > Sure. But if when merging a PR also cherry-pick to corresponding branches, the conflicts will be reduced a lot. The pain comes from issues marked for a bug release milestone but it is merged directly to master and never cherry-picked to branch for bugfix releases. So the release manager has to compare branches and master and also have to compare labels, milestones and even look into the details of pull requests to understood whether a change is cherry-picked or not. > > In the past I've had more success with processes which keep day to day > development on master 100% of the time. If and when you want to cut a > release branch, at that point, you go through the commits in master > since the last release, and select those that should make the bugfix > release. Once you have the list, cherry-picking, in-order, can occur. > This creates more work for the release manager, but it removes the > need to think about the releases from the developer in their > day-to-day. > That works for me. what I really want is a process here that everyone follows. I have been seeing issues merged to master but marked for bugfixes releases, and issues merged without labels and milestones. That will just kill the release manager. > > > Shall we consider following the process what BookKeeper is using, having > a > > merge script to handle labelling issues/milestones and also cherry-picks? > > That script turns a 5 second task into a 5 minute task. Sure it requires more steps on merging issues, however it enforces people label components, types and releases correctly. If we don't label things correctly, the project and release processes will quickly become out of control. No one is able to quickly figure out how a change lands at what releases. People sometimes are just lazy. If they can just click a button to merge one PR, they do it and things quickly become messy because there is no enforcement. > It also > creates a decision point for the developer where the answer is not > always clear. I think as a committer, he/she should try to be aware of those things (e.g. whether this bug is critical, whether it should be included to a release or not). It is sort of the committer's commitment to the project. If he doesn't know whether he should cherry-pick or not, he can just merge to master and leave it to release manager to decide. The point here is we need to some enforcement to make sure a merged change is labelled correctly for a release. so when the release manager going through the issues and commits, he is able to quickly know what he should do. > If no bugfix releases are currently planned, should I > still cherry-pick in branch-1.22? > > > -Ivan >
