Re: Signing artefacts during release

[Ahmet Altay]

This sounds good to me.

On Thu, Mar 7, 2019 at 3:32 PM Michael Luckey  wrote:

> Thanks for your comments.
>
> So to continue here, I ll prepare a PR implementing C:
>
> Pass the sign key to the relevant scripts and use that for signing. There
> is something similar already implemented [1]
>
> We might discuss on that, whether it will work for us or if we need to
> implement something different.
>
> This should affect at least 'build_release_candidate.sh' and
> 'sign_hash_python_wheels.sh'. The release manager is responsible for
> selecting the proper key. Currently there is no 'state passed between the
> scripts', so the release manager will have to specify this repeatedly. This
> could probably be improved later on.
>

This might become a problem. Is it possible for us to tackle this sooner
than later?


>
> @Ahmet Altay  Could you elaborate which global state
> you are referring to? Is it only that git global configuration of the
> signing key? [2]
>

I was referring to things not related to signing. I do not want to digress
this thread but briefly I was referring to global installations of binaries
with sudo and changes to bashrc file. We can work on those improvements
separately.


>
> [1]
> https://github.com/apache/beam/blob/master/release/src/main/scripts/preparation_before_release.sh#L42-L45
> [2]
> https://github.com/apache/beam/blob/master/release/src/main/scripts/preparation_before_release.sh#L47-L48
>
> On Thu, Mar 7, 2019 at 6:03 PM Ahmet Altay  wrote:
>
>>
>>
>> On Wed, Mar 6, 2019 at 8:09 PM Kenneth Knowles  wrote:
>>
>>> What is the precise purpose of each signature? My naïve impression is
>>> that what matters is that users can verify that the release and tag are
>>> authentically products of the release manager, so just whatever keys are
>>> used have to be in the KEYS file. The test/verification of this is more
>>> important that the interface to the scripts.
>>>
>>
>> This is my understanding as well.
>>
>>
>>
>>> I would prefer to leave the script implementation and the user's git/gpg
>>> configuration (or lack thereof) flexible, while giving default guidance
>>> that makes it easy for a release manager.
>>>
>>
>> I am conflicted here. During the release I had issues with scripts making
>> unwanted changes to my global configurations. On that sense I agree with
>> leaving it flexible and not pushing hard changes to user's configuration.
>> On the other hand, ideally the release scripts should just do the right
>> things with the push of a button. Each flexibility in there could mean
>> release managers to read the documentation properly, make the required
>> changes (e.g. signing) in the way they choose. That might lead
>> inconsistencies across releases or missed steps. Arguably we will catch
>> those issues at validation and it might not be a significant issue.
>>
>>
>>>
>>> Kenn
>>>
>>> On Wed, Mar 6, 2019 at 7:29 PM Ahmet Altay  wrote:
>>>
 Your observations about the release process are correct.

 (B) sounds like a better option to me because it will prevent creation
 of one more global setting. However both solutions are workable.

 On Wed, Mar 6, 2019 at 8:08 AM Robert Bradshaw 
 wrote:

> I would not be opposed to make the choice of signing key a required
> argument for the relevant release script(s).
>

 I agree with this. My question would be, how do we ensure that same key
 is passed to different scripts. Today release consists of using multiple
 scripts, it would be a simple mistake for the release manager to use a
 different key for different scripts.


>
> On Wed, Mar 6, 2019 at 3:44 PM Michael Luckey 
> wrote:
> >
> > Hi,
> >
> > After upgrade to gradle 5 @altay (volunteering/selected as release
> manager) was hit by an issue [1] which prevented signing of artefacts. He
> was unfortunately forced to rollback to gradle 4 to do the release.
> >
> > After fixing a configuration issue within beam it seemingly revealed
> an underlying regression in gradle's signing plugin itself [2].
> >
> > If I understand correctly, beams current setup works along the
> following line: On initial configuration any release manager will setup 
> the
> to be used key for git only [3], but we never did something similar on
> gradles behalf. Which results in the signing plugin (delegating to gpg cmd
> line) using whatever gpg considers to be the default key, whether
> explicitly configured with gpg.conf or implicitly.
> >
> > 1. Am I right in assuming that these keys not necessarily have to
> match? I.e. that the key used for signing the release tag ('git tag -s') 
> is
> not necessarily the key used for signing the artifacts?
> > 2. Am I right to assume, that we want/require them to be the same?
> I.e. the key which is uploaded to Beam KEYS file?
> >
> > Now after grade 5 stopped defaulting to gpg's default 

Re: DataflowRunnerTest.testPathExistsValidation failing on HEAD

[Ryan Williams]

Build has passed on the fix , and
that's the only Java PreCommit that's passed in many hours

.

If someone wants to review/merge it that would be great.

On Thu, Mar 7, 2019 at 6:19 PM Ryan Williams  wrote:

> Saw this on #7934  (scan
> )
> but can also reproduce locally from HEAD, and also from commits that
> previously passed Jenkins precommit:
>
> ./gradlew :beam-runners-google-cloud-dataflow-java:test --tests
> '*testPathExistsValidation'
>
> I just sent an optimistic fix at #8013
> , which changes the expected
> error msg to something that comes from the Beam stack.
>
> Apparently we were asserting on the error-msg returned by some external
> GCS API which just changed.
>
>

Re: Signing artefacts during release

[Michael Luckey]

Thanks for your comments.

So to continue here, I ll prepare a PR implementing C:

Pass the sign key to the relevant scripts and use that for signing. There
is something similar already implemented [1]

We might discuss on that, whether it will work for us or if we need to
implement something different.

This should affect at least 'build_release_candidate.sh' and
'sign_hash_python_wheels.sh'. The release manager is responsible for
selecting the proper key. Currently there is no 'state passed between the
scripts', so the release manager will have to specify this repeatedly. This
could probably be improved later on.

@Ahmet Altay  Could you elaborate which global state you
are referring to? Is it only that git global configuration of the signing
key? [2]

[1]
https://github.com/apache/beam/blob/master/release/src/main/scripts/preparation_before_release.sh#L42-L45
[2]
https://github.com/apache/beam/blob/master/release/src/main/scripts/preparation_before_release.sh#L47-L48

On Thu, Mar 7, 2019 at 6:03 PM Ahmet Altay  wrote:

>
>
> On Wed, Mar 6, 2019 at 8:09 PM Kenneth Knowles  wrote:
>
>> What is the precise purpose of each signature? My naïve impression is
>> that what matters is that users can verify that the release and tag are
>> authentically products of the release manager, so just whatever keys are
>> used have to be in the KEYS file. The test/verification of this is more
>> important that the interface to the scripts.
>>
>
> This is my understanding as well.
>
>
>
>> I would prefer to leave the script implementation and the user's git/gpg
>> configuration (or lack thereof) flexible, while giving default guidance
>> that makes it easy for a release manager.
>>
>
> I am conflicted here. During the release I had issues with scripts making
> unwanted changes to my global configurations. On that sense I agree with
> leaving it flexible and not pushing hard changes to user's configuration.
> On the other hand, ideally the release scripts should just do the right
> things with the push of a button. Each flexibility in there could mean
> release managers to read the documentation properly, make the required
> changes (e.g. signing) in the way they choose. That might lead
> inconsistencies across releases or missed steps. Arguably we will catch
> those issues at validation and it might not be a significant issue.
>
>
>>
>> Kenn
>>
>> On Wed, Mar 6, 2019 at 7:29 PM Ahmet Altay  wrote:
>>
>>> Your observations about the release process are correct.
>>>
>>> (B) sounds like a better option to me because it will prevent creation
>>> of one more global setting. However both solutions are workable.
>>>
>>> On Wed, Mar 6, 2019 at 8:08 AM Robert Bradshaw 
>>> wrote:
>>>
 I would not be opposed to make the choice of signing key a required
 argument for the relevant release script(s).

>>>
>>> I agree with this. My question would be, how do we ensure that same key
>>> is passed to different scripts. Today release consists of using multiple
>>> scripts, it would be a simple mistake for the release manager to use a
>>> different key for different scripts.
>>>
>>>

 On Wed, Mar 6, 2019 at 3:44 PM Michael Luckey 
 wrote:
 >
 > Hi,
 >
 > After upgrade to gradle 5 @altay (volunteering/selected as release
 manager) was hit by an issue [1] which prevented signing of artefacts. He
 was unfortunately forced to rollback to gradle 4 to do the release.
 >
 > After fixing a configuration issue within beam it seemingly revealed
 an underlying regression in gradle's signing plugin itself [2].
 >
 > If I understand correctly, beams current setup works along the
 following line: On initial configuration any release manager will setup the
 to be used key for git only [3], but we never did something similar on
 gradles behalf. Which results in the signing plugin (delegating to gpg cmd
 line) using whatever gpg considers to be the default key, whether
 explicitly configured with gpg.conf or implicitly.
 >
 > 1. Am I right in assuming that these keys not necessarily have to
 match? I.e. that the key used for signing the release tag ('git tag -s') is
 not necessarily the key used for signing the artifacts?
 > 2. Am I right to assume, that we want/require them to be the same?
 I.e. the key which is uploaded to Beam KEYS file?
 >
 > Now after grade 5 stopped defaulting to gpg's default key, we somehow
 need to explicitly specify the key to use for the signing plugin. The
 simplest solution would be to just add a note to the release guide how to
 solve that issue on dev side. Which likely will lead to some frustration as
 it is easy to miss.
 >
 > So I would like to integrate something into the used scripting.
 >
 > Options:
 >
 > A: During one time setup, developer is forced to select the proper
 key. This key will be set to (global) git configuration [4]. We could add
 this key 

DataflowRunnerTest.testPathExistsValidation failing on HEAD

[Ryan Williams]

Saw this on #7934  (scan
)
but can also reproduce locally from HEAD, and also from commits that
previously passed Jenkins precommit:

./gradlew :beam-runners-google-cloud-dataflow-java:test --tests
'*testPathExistsValidation'

I just sent an optimistic fix at #8013
, which changes the expected
error msg to something that comes from the Beam stack.

Apparently we were asserting on the error-msg returned by some external GCS
API which just changed.

Re: [Proposal] Get Metrics API: Metric Extraction via proto RPC API.

[Ryan Williams]

Following up here:

A few PRs have gone in related to this proposal (see BEAM-4775
), and Robert, Alex,
Mikhail, and I are discussing some questions that have come up along the
way.

I wrote up the state of things in this doc
.
It describes PRs that are in-flight as well as some further downstream
questions:

   - Should metrics' {namespace,name} be stored in the URN, or as
   MonitoringInfo "labels"?
   - Should "attempted"/"committed" be surfaced as proto fields, or also
   made into a MonitoringInfo label?
  - Also, is there a discrepancy in what those terms mean, between
  older "physical"/"logical" semantics and new FN-API "bundle progress" /
  "bundle result" ones?

It's not really a design doc at this stage, but I wanted to post it in case
others are interested to follow or discuss. If we get agreement on these
questions, we can update the previous design doc of record covering this
work (https://s.apache.org/get-metrics-api).

Thanks,

-Ryan

On Mon, Feb 4, 2019 at 2:26 PM Robert Bradshaw  wrote:

> To summarize for the list, the plan of record is:
>
> The MonitoringInfo proto will be used again in this querying API, so the
> metric format SDKs report will also be used when extracting metrics for a
> job.
>
> // Job Service for running RunnerAPI pipelines
> service JobService {
>   ...
>   rpc GetJobMetrics (GetJobMetricsRequest) returns (GetJobMetricsResponse);
> }
>
> message GetJobMetricsRequest {
>   string job_id = 1; // (required)
> }
>
> message GetJobMetricsResponse {
>// (Optional) The aggregated value of the metric based on the in
> flight work.
>// SDKs optionally report these metrics in the
> ProcessBundleProgressResponse.
>MonitoringInfo attempted_metric_result = 1;
>// (Required) The aggregated value of the metric based on the
> completed work.
>// SDKs report these metrics in the ProcessBundleResponse.
>MonitoringInfo committed_metric_result = 2;
> }
> The new rpc in beam_job_metrics.proto.
>
> SDKs will continue to implement filtering of Metrics, by providing their
> own language specific convenience functions to filter and obtain metrics.
> In Java for example, the MetricResult and MetricFilter format interfaces
> will continue to exist as interface to filter metrics for a specific step,
> user namespace, username, etc.
>
>
> Looking forward to seeing this happen.
>
>
> On Mon, Feb 4, 2019 at 8:17 PM Alex Amato  wrote:
>
>> Done,it's on the website now.
>>
>> Ryan and I will move forward with the plan in this plan. If there are any
>> major objections to this plan, please let us know by weds. Suggestions will
>> be welcome later as well, as we are happy to iterate on this. But we will
>> be proceeding with some of Ryan's PRs based on this design.
>>
>> On Thu, Jan 31, 2019 at 12:54 PM Ismaël Mejía  wrote:
>>
>>> Please don't forget to add this document to the design documents webpage.
>>>
>>> On Thu, Jan 31, 2019 at 8:46 PM Alex Amato  wrote:
>>> >
>>> > Hello Beam,
>>> >
>>> > Robert Ryan and I have been designing a metric extraction API for
>>> Beam. Please take a look at this design, I would love to get more feedback
>>> on this to improve the design.
>>> >
>>> > https://s.apache.org/get-metrics-api
>>> >
>>> > The primary goal of this proposal is to offer a simple way to obtain
>>> all the metrics for a job. The following issues are addressed:
>>> >
>>> > The current design requires implementing metric querying for every
>>> runner+language combination.
>>> >
>>> > Duplication of MetricResult related classes in each language.
>>> >
>>> > The existing MetricResult format only allows querying metrics defined
>>> by a namespace, name and step, and does not allow generalized labelling as
>>> used by MonitoringInfos.
>>> >
>>> > Enhance Beam’s ability to integration test new metrics
>>> >
>>> >
>>> > Thank for taking a look,
>>> > Alex
>>>
>>

Re: [ANNOUNCE] New committer announcement: Raghu Angadi

[Michael Luckey]

Congrats Raghu!

On Thu, Mar 7, 2019 at 8:06 PM Mark Liu  wrote:

> Congrats!
>
> On Thu, Mar 7, 2019 at 10:45 AM Rui Wang  wrote:
>
>> Congrats Raghu!
>>
>>
>> -Rui
>>
>> On Thu, Mar 7, 2019 at 10:22 AM Thomas Weise  wrote:
>>
>>> Congrats!
>>>
>>>
>>> On Thu, Mar 7, 2019 at 10:11 AM Tim Robertson 
>>> wrote:
>>>
 Congrats Raghu

 On Thu, Mar 7, 2019 at 7:09 PM Ahmet Altay  wrote:

> Congratulations!
>
> On Thu, Mar 7, 2019 at 10:08 AM Ruoyun Huang 
> wrote:
>
>> Thank you Raghu for your contribution!
>>
>>
>>
>> On Thu, Mar 7, 2019 at 9:58 AM Connell O'Callaghan <
>> conne...@google.com> wrote:
>>
>>> Congratulation Raghu!!! Thank you for sharing Kenn!!!
>>>
>>> On Thu, Mar 7, 2019 at 9:55 AM Ismaël Mejía 
>>> wrote:
>>>
 Congrats !

 Le jeu. 7 mars 2019 à 17:09, Aizhamal Nurmamat kyzy <
 aizha...@google.com> a écrit :

> Congratulations, Raghu!!!
> On Thu, Mar 7, 2019 at 08:07 Kenneth Knowles 
> wrote:
>
>> Hi all,
>>
>> Please join me and the rest of the Beam PMC in welcoming a new
>> committer: Raghu Angadi
>>
>> Raghu has been contributing to Beam since early 2016! He has
>> continuously improved KafkaIO and supported on the user@ list
>> but his community contributions are even more extensive, including 
>> reviews,
>> dev@ list discussions, improvements and ideas across SqsIO,
>> FileIO, PubsubIO, and the Dataflow and Samza runners. In
>> consideration of Raghu's contributions, the Beam PMC trusts Raghu 
>> with the
>> responsibilities of a Beam committer [1].
>>
>> Thank you, Raghu, for your contributions.
>>
>> Kenn
>>
>> [1] https://beam.apache.org/contribute/become-a-committer
>> /#an-apache-beam-committer
>>
>
>>
>> --
>> 
>> Ruoyun  Huang
>>
>>

Re: [ANNOUNCE] New committer announcement: Raghu Angadi

[Mark Liu]

Congrats!

On Thu, Mar 7, 2019 at 10:45 AM Rui Wang  wrote:

> Congrats Raghu!
>
>
> -Rui
>
> On Thu, Mar 7, 2019 at 10:22 AM Thomas Weise  wrote:
>
>> Congrats!
>>
>>
>> On Thu, Mar 7, 2019 at 10:11 AM Tim Robertson 
>> wrote:
>>
>>> Congrats Raghu
>>>
>>> On Thu, Mar 7, 2019 at 7:09 PM Ahmet Altay  wrote:
>>>
 Congratulations!

 On Thu, Mar 7, 2019 at 10:08 AM Ruoyun Huang  wrote:

> Thank you Raghu for your contribution!
>
>
>
> On Thu, Mar 7, 2019 at 9:58 AM Connell O'Callaghan <
> conne...@google.com> wrote:
>
>> Congratulation Raghu!!! Thank you for sharing Kenn!!!
>>
>> On Thu, Mar 7, 2019 at 9:55 AM Ismaël Mejía 
>> wrote:
>>
>>> Congrats !
>>>
>>> Le jeu. 7 mars 2019 à 17:09, Aizhamal Nurmamat kyzy <
>>> aizha...@google.com> a écrit :
>>>
 Congratulations, Raghu!!!
 On Thu, Mar 7, 2019 at 08:07 Kenneth Knowles 
 wrote:

> Hi all,
>
> Please join me and the rest of the Beam PMC in welcoming a new
> committer: Raghu Angadi
>
> Raghu has been contributing to Beam since early 2016! He has
> continuously improved KafkaIO and supported on the user@ list but
> his community contributions are even more extensive, including 
> reviews, dev@
> list discussions, improvements and ideas across SqsIO, FileIO, 
> PubsubIO,
> and the Dataflow and Samza runners. In consideration of Raghu's
> contributions, the Beam PMC trusts Raghu with the responsibilities of 
> a
> Beam committer [1].
>
> Thank you, Raghu, for your contributions.
>
> Kenn
>
> [1] https://beam.apache.org/contribute/become-a-committer
> /#an-apache-beam-committer
>

>
> --
> 
> Ruoyun  Huang
>
>

Re: [ANNOUNCE] New committer announcement: Raghu Angadi

[Rui Wang]

Congrats Raghu!


-Rui

On Thu, Mar 7, 2019 at 10:22 AM Thomas Weise  wrote:

> Congrats!
>
>
> On Thu, Mar 7, 2019 at 10:11 AM Tim Robertson 
> wrote:
>
>> Congrats Raghu
>>
>> On Thu, Mar 7, 2019 at 7:09 PM Ahmet Altay  wrote:
>>
>>> Congratulations!
>>>
>>> On Thu, Mar 7, 2019 at 10:08 AM Ruoyun Huang  wrote:
>>>
 Thank you Raghu for your contribution!



 On Thu, Mar 7, 2019 at 9:58 AM Connell O'Callaghan 
 wrote:

> Congratulation Raghu!!! Thank you for sharing Kenn!!!
>
> On Thu, Mar 7, 2019 at 9:55 AM Ismaël Mejía  wrote:
>
>> Congrats !
>>
>> Le jeu. 7 mars 2019 à 17:09, Aizhamal Nurmamat kyzy <
>> aizha...@google.com> a écrit :
>>
>>> Congratulations, Raghu!!!
>>> On Thu, Mar 7, 2019 at 08:07 Kenneth Knowles 
>>> wrote:
>>>
 Hi all,

 Please join me and the rest of the Beam PMC in welcoming a new
 committer: Raghu Angadi

 Raghu has been contributing to Beam since early 2016! He has
 continuously improved KafkaIO and supported on the user@ list but
 his community contributions are even more extensive, including 
 reviews, dev@
 list discussions, improvements and ideas across SqsIO, FileIO, 
 PubsubIO,
 and the Dataflow and Samza runners. In consideration of Raghu's
 contributions, the Beam PMC trusts Raghu with the responsibilities of a
 Beam committer [1].

 Thank you, Raghu, for your contributions.

 Kenn

 [1] https://beam.apache.org/contribute/become-a-committer
 /#an-apache-beam-committer

>>>

 --
 
 Ruoyun  Huang

Re: [ANNOUNCE] New committer announcement: Raghu Angadi

[Thomas Weise]

Congrats!


On Thu, Mar 7, 2019 at 10:11 AM Tim Robertson 
wrote:

> Congrats Raghu
>
> On Thu, Mar 7, 2019 at 7:09 PM Ahmet Altay  wrote:
>
>> Congratulations!
>>
>> On Thu, Mar 7, 2019 at 10:08 AM Ruoyun Huang  wrote:
>>
>>> Thank you Raghu for your contribution!
>>>
>>>
>>>
>>> On Thu, Mar 7, 2019 at 9:58 AM Connell O'Callaghan 
>>> wrote:
>>>
 Congratulation Raghu!!! Thank you for sharing Kenn!!!

 On Thu, Mar 7, 2019 at 9:55 AM Ismaël Mejía  wrote:

> Congrats !
>
> Le jeu. 7 mars 2019 à 17:09, Aizhamal Nurmamat kyzy <
> aizha...@google.com> a écrit :
>
>> Congratulations, Raghu!!!
>> On Thu, Mar 7, 2019 at 08:07 Kenneth Knowles  wrote:
>>
>>> Hi all,
>>>
>>> Please join me and the rest of the Beam PMC in welcoming a new
>>> committer: Raghu Angadi
>>>
>>> Raghu has been contributing to Beam since early 2016! He has
>>> continuously improved KafkaIO and supported on the user@ list but
>>> his community contributions are even more extensive, including reviews, 
>>> dev@
>>> list discussions, improvements and ideas across SqsIO, FileIO, PubsubIO,
>>> and the Dataflow and Samza runners. In consideration of Raghu's
>>> contributions, the Beam PMC trusts Raghu with the responsibilities of a
>>> Beam committer [1].
>>>
>>> Thank you, Raghu, for your contributions.
>>>
>>> Kenn
>>>
>>> [1] https://beam.apache.org/contribute/become-a-committer
>>> /#an-apache-beam-committer
>>>
>>
>>>
>>> --
>>> 
>>> Ruoyun  Huang
>>>
>>>

Re: [ANNOUNCE] New committer announcement: Raghu Angadi

[Tim Robertson]

Congrats Raghu

On Thu, Mar 7, 2019 at 7:09 PM Ahmet Altay  wrote:

> Congratulations!
>
> On Thu, Mar 7, 2019 at 10:08 AM Ruoyun Huang  wrote:
>
>> Thank you Raghu for your contribution!
>>
>>
>>
>> On Thu, Mar 7, 2019 at 9:58 AM Connell O'Callaghan 
>> wrote:
>>
>>> Congratulation Raghu!!! Thank you for sharing Kenn!!!
>>>
>>> On Thu, Mar 7, 2019 at 9:55 AM Ismaël Mejía  wrote:
>>>
 Congrats !

 Le jeu. 7 mars 2019 à 17:09, Aizhamal Nurmamat kyzy <
 aizha...@google.com> a écrit :

> Congratulations, Raghu!!!
> On Thu, Mar 7, 2019 at 08:07 Kenneth Knowles  wrote:
>
>> Hi all,
>>
>> Please join me and the rest of the Beam PMC in welcoming a new
>> committer: Raghu Angadi
>>
>> Raghu has been contributing to Beam since early 2016! He has
>> continuously improved KafkaIO and supported on the user@ list but
>> his community contributions are even more extensive, including reviews, 
>> dev@
>> list discussions, improvements and ideas across SqsIO, FileIO, PubsubIO,
>> and the Dataflow and Samza runners. In consideration of Raghu's
>> contributions, the Beam PMC trusts Raghu with the responsibilities of a
>> Beam committer [1].
>>
>> Thank you, Raghu, for your contributions.
>>
>> Kenn
>>
>> [1] https://beam.apache.org/contribute/become-a-committer/#an-apache-
>> beam-committer
>>
>
>>
>> --
>> 
>> Ruoyun  Huang
>>
>>

Re: [ANNOUNCE] New committer announcement: Raghu Angadi

[Ahmet Altay]

Congratulations!

On Thu, Mar 7, 2019 at 10:08 AM Ruoyun Huang  wrote:

> Thank you Raghu for your contribution!
>
>
>
> On Thu, Mar 7, 2019 at 9:58 AM Connell O'Callaghan 
> wrote:
>
>> Congratulation Raghu!!! Thank you for sharing Kenn!!!
>>
>> On Thu, Mar 7, 2019 at 9:55 AM Ismaël Mejía  wrote:
>>
>>> Congrats !
>>>
>>> Le jeu. 7 mars 2019 à 17:09, Aizhamal Nurmamat kyzy 
>>> a écrit :
>>>
 Congratulations, Raghu!!!
 On Thu, Mar 7, 2019 at 08:07 Kenneth Knowles  wrote:

> Hi all,
>
> Please join me and the rest of the Beam PMC in welcoming a new
> committer: Raghu Angadi
>
> Raghu has been contributing to Beam since early 2016! He has
> continuously improved KafkaIO and supported on the user@ list but his
> community contributions are even more extensive, including reviews, dev@
> list discussions, improvements and ideas across SqsIO, FileIO, PubsubIO,
> and the Dataflow and Samza runners. In consideration of Raghu's
> contributions, the Beam PMC trusts Raghu with the responsibilities of a
> Beam committer [1].
>
> Thank you, Raghu, for your contributions.
>
> Kenn
>
> [1] https://beam.apache.org/contribute/become-a-committer/#an-apache-
> beam-committer
>

>
> --
> 
> Ruoyun  Huang
>
>

Re: [ANNOUNCE] New committer announcement: Raghu Angadi

[Ruoyun Huang]

Thank you Raghu for your contribution!



On Thu, Mar 7, 2019 at 9:58 AM Connell O'Callaghan 
wrote:

> Congratulation Raghu!!! Thank you for sharing Kenn!!!
>
> On Thu, Mar 7, 2019 at 9:55 AM Ismaël Mejía  wrote:
>
>> Congrats !
>>
>> Le jeu. 7 mars 2019 à 17:09, Aizhamal Nurmamat kyzy 
>> a écrit :
>>
>>> Congratulations, Raghu!!!
>>> On Thu, Mar 7, 2019 at 08:07 Kenneth Knowles  wrote:
>>>
 Hi all,

 Please join me and the rest of the Beam PMC in welcoming a new
 committer: Raghu Angadi

 Raghu has been contributing to Beam since early 2016! He has
 continuously improved KafkaIO and supported on the user@ list but his
 community contributions are even more extensive, including reviews, dev@
 list discussions, improvements and ideas across SqsIO, FileIO, PubsubIO,
 and the Dataflow and Samza runners. In consideration of Raghu's
 contributions, the Beam PMC trusts Raghu with the responsibilities of a
 Beam committer [1].

 Thank you, Raghu, for your contributions.

 Kenn

 [1] https://beam.apache.org/contribute/become-a-committer/#an-apache-
 beam-committer

>>>

-- 

Ruoyun  Huang

Re: [ANNOUNCE] New committer announcement: Raghu Angadi

[Connell O'Callaghan]

Congratulation Raghu!!! Thank you for sharing Kenn!!!

On Thu, Mar 7, 2019 at 9:55 AM Ismaël Mejía  wrote:

> Congrats !
>
> Le jeu. 7 mars 2019 à 17:09, Aizhamal Nurmamat kyzy 
> a écrit :
>
>> Congratulations, Raghu!!!
>> On Thu, Mar 7, 2019 at 08:07 Kenneth Knowles  wrote:
>>
>>> Hi all,
>>>
>>> Please join me and the rest of the Beam PMC in welcoming a new committer:
>>> Raghu Angadi
>>>
>>> Raghu has been contributing to Beam since early 2016! He has
>>> continuously improved KafkaIO and supported on the user@ list but his
>>> community contributions are even more extensive, including reviews, dev@
>>> list discussions, improvements and ideas across SqsIO, FileIO, PubsubIO,
>>> and the Dataflow and Samza runners. In consideration of Raghu's
>>> contributions, the Beam PMC trusts Raghu with the responsibilities of a
>>> Beam committer [1].
>>>
>>> Thank you, Raghu, for your contributions.
>>>
>>> Kenn
>>>
>>> [1] https://beam.apache.org/contribute/become-a-committer/#an-apache-
>>> beam-committer
>>>
>>

Re: [ANNOUNCE] New committer announcement: Raghu Angadi

[Ismaël Mejía]

Congrats !

Le jeu. 7 mars 2019 à 17:09, Aizhamal Nurmamat kyzy  a
écrit :

> Congratulations, Raghu!!!
> On Thu, Mar 7, 2019 at 08:07 Kenneth Knowles  wrote:
>
>> Hi all,
>>
>> Please join me and the rest of the Beam PMC in welcoming a new committer:
>> Raghu Angadi
>>
>> Raghu has been contributing to Beam since early 2016! He has continuously
>> improved KafkaIO and supported on the user@ list but his community
>> contributions are even more extensive, including reviews, dev@ list
>> discussions, improvements and ideas across SqsIO, FileIO, PubsubIO, and the
>> Dataflow and Samza runners. In consideration of Raghu's contributions,
>> the Beam PMC trusts Raghu with the responsibilities of a Beam committer [1].
>>
>> Thank you, Raghu, for your contributions.
>>
>> Kenn
>>
>> [1] https://beam.apache.org/contribute/become-a-committer/#an-apache-beam
>> -committer
>>
>

Re: Signing artefacts during release

[Ahmet Altay]

On Wed, Mar 6, 2019 at 8:09 PM Kenneth Knowles  wrote:

> What is the precise purpose of each signature? My naïve impression is that
> what matters is that users can verify that the release and tag are
> authentically products of the release manager, so just whatever keys are
> used have to be in the KEYS file. The test/verification of this is more
> important that the interface to the scripts.
>

This is my understanding as well.



> I would prefer to leave the script implementation and the user's git/gpg
> configuration (or lack thereof) flexible, while giving default guidance
> that makes it easy for a release manager.
>

I am conflicted here. During the release I had issues with scripts making
unwanted changes to my global configurations. On that sense I agree with
leaving it flexible and not pushing hard changes to user's configuration.
On the other hand, ideally the release scripts should just do the right
things with the push of a button. Each flexibility in there could mean
release managers to read the documentation properly, make the required
changes (e.g. signing) in the way they choose. That might lead
inconsistencies across releases or missed steps. Arguably we will catch
those issues at validation and it might not be a significant issue.


>
> Kenn
>
> On Wed, Mar 6, 2019 at 7:29 PM Ahmet Altay  wrote:
>
>> Your observations about the release process are correct.
>>
>> (B) sounds like a better option to me because it will prevent creation of
>> one more global setting. However both solutions are workable.
>>
>> On Wed, Mar 6, 2019 at 8:08 AM Robert Bradshaw 
>> wrote:
>>
>>> I would not be opposed to make the choice of signing key a required
>>> argument for the relevant release script(s).
>>>
>>
>> I agree with this. My question would be, how do we ensure that same key
>> is passed to different scripts. Today release consists of using multiple
>> scripts, it would be a simple mistake for the release manager to use a
>> different key for different scripts.
>>
>>
>>>
>>> On Wed, Mar 6, 2019 at 3:44 PM Michael Luckey 
>>> wrote:
>>> >
>>> > Hi,
>>> >
>>> > After upgrade to gradle 5 @altay (volunteering/selected as release
>>> manager) was hit by an issue [1] which prevented signing of artefacts. He
>>> was unfortunately forced to rollback to gradle 4 to do the release.
>>> >
>>> > After fixing a configuration issue within beam it seemingly revealed
>>> an underlying regression in gradle's signing plugin itself [2].
>>> >
>>> > If I understand correctly, beams current setup works along the
>>> following line: On initial configuration any release manager will setup the
>>> to be used key for git only [3], but we never did something similar on
>>> gradles behalf. Which results in the signing plugin (delegating to gpg cmd
>>> line) using whatever gpg considers to be the default key, whether
>>> explicitly configured with gpg.conf or implicitly.
>>> >
>>> > 1. Am I right in assuming that these keys not necessarily have to
>>> match? I.e. that the key used for signing the release tag ('git tag -s') is
>>> not necessarily the key used for signing the artifacts?
>>> > 2. Am I right to assume, that we want/require them to be the same?
>>> I.e. the key which is uploaded to Beam KEYS file?
>>> >
>>> > Now after grade 5 stopped defaulting to gpg's default key, we somehow
>>> need to explicitly specify the key to use for the signing plugin. The
>>> simplest solution would be to just add a note to the release guide how to
>>> solve that issue on dev side. Which likely will lead to some frustration as
>>> it is easy to miss.
>>> >
>>> > So I would like to integrate something into the used scripting.
>>> >
>>> > Options:
>>> >
>>> > A: During one time setup, developer is forced to select the proper
>>> key. This key will be set to (global) git configuration [4]. We could add
>>> this key also to gradle.gradleHome gradle.properties file as
>>> 'signing.gnupg.keyName' which then would be used by gradles signing plugin.
>>> >
>>> > Obvious drawback here would be that this is a global configuration
>>> (ok, the same problem we have already for git), which might not be
>>> appropriate for all devs.
>>> >
>>> > B: Read the 'git config user.signingkey' on script execution and pass
>>> this as '-Psigning.gnupg.keyName' parameter to the gradle run. Of course
>>> this will only work, iff the git config is set. So would it be save to
>>> assume such?
>>> >
>>> > Drawback here, of course, is someone not using the release script
>>> missing to set the signing key.
>>> >
>>> > Of course, both will not solve any issue with source.zip releases and
>>> pythons signing key, which, as far as I can tell also rely on gpgs default
>>> key which might conflict with 1. above?
>>> >
>>> > Any thoughts about this?
>>> >
>>> > michel
>>> >
>>> >
>>> >
>>> >
>>> > [1] https://issues.apache.org/jira/browse/BEAM-6726
>>> > [2] https://github.com/gradle/gradle/issues/8657
>>> > [3]
>>> https://github.com/apache/beam/blob/master/website/src/

Re: [ANNOUNCE] New committer announcement: Raghu Angadi

[Aizhamal Nurmamat kyzy]

Congratulations, Raghu!!!
On Thu, Mar 7, 2019 at 08:07 Kenneth Knowles  wrote:

> Hi all,
>
> Please join me and the rest of the Beam PMC in welcoming a new committer:
> Raghu Angadi
>
> Raghu has been contributing to Beam since early 2016! He has continuously
> improved KafkaIO and supported on the user@ list but his community
> contributions are even more extensive, including reviews, dev@ list
> discussions, improvements and ideas across SqsIO, FileIO, PubsubIO, and the
> Dataflow and Samza runners. In consideration of Raghu's contributions,
> the Beam PMC trusts Raghu with the responsibilities of a Beam committer [1].
>
> Thank you, Raghu, for your contributions.
>
> Kenn
>
> [1] https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-
> committer
>

[ANNOUNCE] New committer announcement: Raghu Angadi

[Kenneth Knowles]

Hi all,

Please join me and the rest of the Beam PMC in welcoming a new committer:
Raghu Angadi

Raghu has been contributing to Beam since early 2016! He has continuously
improved KafkaIO and supported on the user@ list but his community
contributions are even more extensive, including reviews, dev@ list
discussions, improvements and ideas across SqsIO, FileIO, PubsubIO, and the
Dataflow and Samza runners. In consideration of Raghu's contributions, the
Beam PMC trusts Raghu with the responsibilities of a Beam committer [1].

Thank you, Raghu, for your contributions.

Kenn

[1] https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-
committer

contributor permission for Beam Jira tickets

[Radosław Stankiewicz]

Hi,

Hi,

This is Radek from Google. I'm working on features around Templates and
ValueProviders for CassandraIO and in the future other IOs. Can someone add
me as a contributor for Beam's Jira issue tracker? I would like to
create/assign tickets for my work. My jira username is rrydziu, I've
already created BEAM-6773  and
draft PR 8001 .

Thanks,
Radek