Re: [VOTE] Release 2.4.0, release candidate #3

2018-03-19 Thread Valentyn Tymofieiev
+1.
Ran a Python Streaming wordcount pipeline on Direct and Dataflow runners
and Batch mobile gaming examples on Dataflow runner.

On Mon, Mar 19, 2018 at 6:02 PM, Alan Myrvold  wrote:

> +1 I ran the java quickstarts against 2.4.0 and they passed.
> ./gradlew :release:runQuickstartsJava -Prepourl=https://repository.
> apache.org/content/repositories/orgapachebeam-1031 -Pver=2.4.0
>
>
> On Mon, Mar 19, 2018 at 5:41 PM Ahmet Altay  wrote:
>
>> I was able to run hourly_team_score. I was passing a wrong argument. No
>> need for an alarm. :)
>>
>> On Mon, Mar 19, 2018 at 5:33 PM, Ahmet Altay  wrote:
>>
>>> +1 Thank you Robert.
>>>
>>> Verified python mobile gaming examples using the wheel files on direct
>>> runner. Got user_score working but hourly_team_score failed with (
>>> https://issues.apache.org/jira/browse/BEAM-3824). Since this is an
>>> example, I think it is fine to continue with the release. I will work on
>>> fixing the example post release.
>>>
>>> On Mon, Mar 19, 2018 at 2:46 PM, Konstantinos Katsiapis <
>>> katsia...@google.com> wrote:
>>>
 +1, since Tf.Transform  0.6
 depends on (and is blocked by) Beam 2.4

 On Sat, Mar 17, 2018 at 2:19 AM, Robert Bradshaw 
 wrote:

> Hi everyone,
>
> Please review and vote on the release candidate #3 for the version
> 2.4.0,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2],
> which is signed with the key with fingerprint BDC9 89B0 1BD2 A463 6010
> A1CA
> 8F15 5E09 610D 69FB [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.4.0-RC3" [5],
> * website pull request listing the release and publishing the API
> reference
> manual [6].
> * Java artifacts were built with Maven 3.2.5 and OpenJDK 1.8.0_112.
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
>
> The validation spreadsheet is available at
> https://docs.google.com/spreadsheets/d/1qk-
> N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit?ts=5a1c7310#
> gid=1663314475
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> - Robert
>
> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> version=12342682=12319527
> [2] https://dist.apache.org/repos/dist/dev/beam/2.4.0/
> [3] https://dist.apache.org/repos/dist/dev/beam/KEYS
> [4] https://repository.apache.org/content/repositories/
> orgapachebeam-1031/
> [5] https://github.com/apache/beam/tree/v2.4.0-RC3
> [6] https://github.com/apache/beam-site/pull/398
>



 --
 Gus Katsiapis | Software Engineer | katsia...@google.com | 650-918-7487
 <(650)%20918-7487>

>>>
>>>
>>


Re: [VOTE] Release 2.4.0, release candidate #3

2018-03-19 Thread Ahmet Altay
I was able to run hourly_team_score. I was passing a wrong argument. No
need for an alarm. :)

On Mon, Mar 19, 2018 at 5:33 PM, Ahmet Altay  wrote:

> +1 Thank you Robert.
>
> Verified python mobile gaming examples using the wheel files on direct
> runner. Got user_score working but hourly_team_score failed with (
> https://issues.apache.org/jira/browse/BEAM-3824). Since this is an
> example, I think it is fine to continue with the release. I will work on
> fixing the example post release.
>
> On Mon, Mar 19, 2018 at 2:46 PM, Konstantinos Katsiapis <
> katsia...@google.com> wrote:
>
>> +1, since Tf.Transform  0.6
>> depends on (and is blocked by) Beam 2.4
>>
>> On Sat, Mar 17, 2018 at 2:19 AM, Robert Bradshaw 
>> wrote:
>>
>>> Hi everyone,
>>>
>>> Please review and vote on the release candidate #3 for the version 2.4.0,
>>> as follows:
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>
>>> The complete staging area is available for your review, which includes:
>>> * JIRA release notes [1],
>>> * the official Apache source release to be deployed to dist.apache.org
>>> [2],
>>> which is signed with the key with fingerprint BDC9 89B0 1BD2 A463 6010
>>> A1CA
>>> 8F15 5E09 610D 69FB [3],
>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>> * source code tag "v2.4.0-RC3" [5],
>>> * website pull request listing the release and publishing the API
>>> reference
>>> manual [6].
>>> * Java artifacts were built with Maven 3.2.5 and OpenJDK 1.8.0_112.
>>> * Python artifacts are deployed along with the source release to the
>>> dist.apache.org [2].
>>>
>>> The validation spreadsheet is available at
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkS
>>> ZTR8AGqyNUM-oLFo_ZXBpJw/edit?ts=5a1c7310#gid=1663314475
>>>
>>> The vote will be open for at least 72 hours. It is adopted by majority
>>> approval, with at least 3 PMC affirmative votes.
>>>
>>> Thanks,
>>> - Robert
>>>
>>> [1]
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?versi
>>> on=12342682=12319527
>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.4.0/
>>> [3] https://dist.apache.org/repos/dist/dev/beam/KEYS
>>> [4] https://repository.apache.org/content/repositories/orgapache
>>> beam-1031/
>>> [5] https://github.com/apache/beam/tree/v2.4.0-RC3
>>> [6] https://github.com/apache/beam-site/pull/398
>>>
>>
>>
>>
>> --
>> Gus Katsiapis | Software Engineer | katsia...@google.com | 650-918-7487
>> <(650)%20918-7487>
>>
>
>


Re: [VOTE] Release 2.4.0, release candidate #3

2018-03-19 Thread Ahmet Altay
+1 Thank you Robert.

Verified python mobile gaming examples using the wheel files on direct
runner. Got user_score working but hourly_team_score failed with (
https://issues.apache.org/jira/browse/BEAM-3824). Since this is an example,
I think it is fine to continue with the release. I will work on fixing the
example post release.

On Mon, Mar 19, 2018 at 2:46 PM, Konstantinos Katsiapis <
katsia...@google.com> wrote:

> +1, since Tf.Transform  0.6
> depends on (and is blocked by) Beam 2.4
>
> On Sat, Mar 17, 2018 at 2:19 AM, Robert Bradshaw 
> wrote:
>
>> Hi everyone,
>>
>> Please review and vote on the release candidate #3 for the version 2.4.0,
>> as follows:
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>>
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org
>> [2],
>> which is signed with the key with fingerprint BDC9 89B0 1BD2 A463 6010
>> A1CA
>> 8F15 5E09 610D 69FB [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "v2.4.0-RC3" [5],
>> * website pull request listing the release and publishing the API
>> reference
>> manual [6].
>> * Java artifacts were built with Maven 3.2.5 and OpenJDK 1.8.0_112.
>> * Python artifacts are deployed along with the source release to the
>> dist.apache.org [2].
>>
>> The validation spreadsheet is available at
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkS
>> ZTR8AGqyNUM-oLFo_ZXBpJw/edit?ts=5a1c7310#gid=1663314475
>>
>> The vote will be open for at least 72 hours. It is adopted by majority
>> approval, with at least 3 PMC affirmative votes.
>>
>> Thanks,
>> - Robert
>>
>> [1]
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?versi
>> on=12342682=12319527
>> [2] https://dist.apache.org/repos/dist/dev/beam/2.4.0/
>> [3] https://dist.apache.org/repos/dist/dev/beam/KEYS
>> [4] https://repository.apache.org/content/repositories/orgapache
>> beam-1031/
>> [5] https://github.com/apache/beam/tree/v2.4.0-RC3
>> [6] https://github.com/apache/beam-site/pull/398
>>
>
>
>
> --
> Gus Katsiapis | Software Engineer | katsia...@google.com | 650-918-7487
> <(650)%20918-7487>
>


Pubsub API feedback

2018-03-19 Thread Udi Meiri
Hi,
I wanted to get feedback about the upcoming Python Pubsub API. It is
currently experimental and only supports reading and writing UTF-8 strings.
My current proposal only concerns reading from Pubsub.

Classes:
- PubsubMessage: encapsulates Pubsub message payload and attributes.

PTransforms:
- ReadMessagesFromPubSub: Outputs elements of type ``PubsubMessage``.

- ReadPayloadsFromPubSub: Outputs elements of type ``str``.

- ReadStringsFromPubSub: Outputs elements of type ``unicode``, decoded from
UTF-8.

Description of common PTransform arguments:
  topic: Cloud Pub/Sub topic in the form
"projects//topics/".
If provided, subscription must be None.
  subscription: Existing Cloud Pub/Sub subscription to use in the
form "projects//subscriptions/". If not
specified,
a temporary subscription will be created from the specified topic. If
provided, topic must be None.
  id_label: The attribute on incoming Pub/Sub messages to use as a unique
record identifier. When specified, the value of this attribute (which
can be any string that uniquely identifies the record) will be used for
deduplication of messages. If not provided, we cannot guarantee
that no duplicate data will be delivered on the Pub/Sub stream. In this
case, deduplication of the stream will be strictly best effort.
  timestamp_attribute: Message value to use as element timestamp. If None,
uses message publishing time as the timestamp.
Timestamp values should be in one of two formats:
- A numerical value representing the number of milliseconds since the
Unix
  epoch.
- A string in RFC 3339 format. For example,
  {@code 2015-10-29T23:41:41.123Z}. The sub-second component of the
  timestamp is optional, and digits beyond the first three (i.e., time
units
  smaller than milliseconds) will be ignored.

Code:
https://github.com/udim/beam/blob/b981dd618e9e1f667597eec2a91c7265a389c405/sdks/python/apache_beam/io/gcp/pubsub.py
PR: https://github.com/apache/beam/pull/4901


smime.p7s
Description: S/MIME Cryptographic Signature


Implementing @OnWindowExpiration in StatefulParDo [BEAM-1589]

2018-03-19 Thread Huygaa Batsaikhan
Hi everyone, I am working on BEAM-1589
. In short, currently,
there is no default way of saving/flushing state before a window is garbage
collected.

My current plan is to provide a method annotation, @OnWindowExpiration,
which allows user-provided callback function to be executed before garbage
collection. This annotation behaves very similar to @OnTimer, therefore,
implementation will mostly be a copy of OnTimer code. Let me know if you
have any considerations and suggestions.

Here is an example usage:
```
@OnWindowExpiration
public void myCleanupFunction(OnWindowExpirationContext c, State state) {
  c.output(state.read());
}
```

Thanks, Huygaa


Re: [VOTE] Release 2.4.0, release candidate #3

2018-03-19 Thread Konstantinos Katsiapis
+1, since Tf.Transform  0.6
depends on (and is blocked by) Beam 2.4

On Sat, Mar 17, 2018 at 2:19 AM, Robert Bradshaw 
wrote:

> Hi everyone,
>
> Please review and vote on the release candidate #3 for the version 2.4.0,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2],
> which is signed with the key with fingerprint BDC9 89B0 1BD2 A463 6010 A1CA
> 8F15 5E09 610D 69FB [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.4.0-RC3" [5],
> * website pull request listing the release and publishing the API reference
> manual [6].
> * Java artifacts were built with Maven 3.2.5 and OpenJDK 1.8.0_112.
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
>
> The validation spreadsheet is available at
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-
> oLFo_ZXBpJw/edit?ts=5a1c7310#gid=1663314475
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> - Robert
>
> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> version=12342682=12319527
> [2] https://dist.apache.org/repos/dist/dev/beam/2.4.0/
> [3] https://dist.apache.org/repos/dist/dev/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1031/
> [5] https://github.com/apache/beam/tree/v2.4.0-RC3
> [6] https://github.com/apache/beam-site/pull/398
>



-- 
Gus Katsiapis | Software Engineer | katsia...@google.com | 650-918-7487


Re: [VOTE] Release 2.4.0, release candidate #3

2018-03-19 Thread Charles Chen
+1.  Verified the Python Quickstart on local and Dataflow (Mac / Linux).
Also verified that the Mac / Linux wheels were built correctly with fast /
compiled Cython coder support.

On Mon, Mar 19, 2018 at 1:49 PM Robert Bradshaw  wrote:

> Thanks!
>
> BTW, in case anyone's wondering where the md5 files went, they're now
> discouraged: http://www.apache.org/dev/release-distribution#sigs-and-sums
>
>
> On Mon, Mar 19, 2018 at 12:53 PM Lukasz Cwik  wrote:
>
>> +1 (binding), verified Java quickstart on Apex local, DirectRunner,
>> Dataflow, Flink local, Spark local.
>>
>>
>> On Mon, Mar 19, 2018 at 3:54 AM Romain Manni-Bucau 
>> wrote:
>>
>>> -0 (cause of the teardown issue which is still a blocker), otherwise
>>> spark/direct runners work in my projects
>>>
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau  |  Blog
>>>  | Old Blog
>>>  | Github
>>>  | LinkedIn
>>>  | Book
>>> 
>>>
>>> 2018-03-17 10:19 GMT+01:00 Robert Bradshaw :
>>>
 Hi everyone,

 Please review and vote on the release candidate #3 for the version
 2.4.0,
 as follows:
 [ ] +1, Approve the release
 [ ] -1, Do not approve the release (please provide specific comments)

 The complete staging area is available for your review, which includes:
 * JIRA release notes [1],
 * the official Apache source release to be deployed to dist.apache.org
 [2],
 which is signed with the key with fingerprint BDC9 89B0 1BD2 A463 6010
 A1CA
 8F15 5E09 610D 69FB [3],
 * all artifacts to be deployed to the Maven Central Repository [4],
 * source code tag "v2.4.0-RC3" [5],
 * website pull request listing the release and publishing the API
 reference
 manual [6].
 * Java artifacts were built with Maven 3.2.5 and OpenJDK 1.8.0_112.
 * Python artifacts are deployed along with the source release to the
 dist.apache.org [2].

 The validation spreadsheet is available at

 https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit?ts=5a1c7310#gid=1663314475

 The vote will be open for at least 72 hours. It is adopted by majority
 approval, with at least 3 PMC affirmative votes.

 Thanks,
 - Robert

 [1]

 https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342682=12319527
 [2] https://dist.apache.org/repos/dist/dev/beam/2.4.0/
 [3] https://dist.apache.org/repos/dist/dev/beam/KEYS
 [4]
 https://repository.apache.org/content/repositories/orgapachebeam-1031/
 [5] https://github.com/apache/beam/tree/v2.4.0-RC3
 [6] https://github.com/apache/beam-site/pull/398

>>>
>>>


Re: [VOTE] Release 2.4.0, release candidate #3

2018-03-19 Thread Robert Bradshaw
Thanks!

BTW, in case anyone's wondering where the md5 files went, they're now
discouraged: http://www.apache.org/dev/release-distribution#sigs-and-sums


On Mon, Mar 19, 2018 at 12:53 PM Lukasz Cwik  wrote:

> +1 (binding), verified Java quickstart on Apex local, DirectRunner,
> Dataflow, Flink local, Spark local.
>
>
> On Mon, Mar 19, 2018 at 3:54 AM Romain Manni-Bucau 
> wrote:
>
>> -0 (cause of the teardown issue which is still a blocker), otherwise
>> spark/direct runners work in my projects
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau  |  Blog
>>  | Old Blog
>>  | Github
>>  | LinkedIn
>>  | Book
>> 
>>
>> 2018-03-17 10:19 GMT+01:00 Robert Bradshaw :
>>
>>> Hi everyone,
>>>
>>> Please review and vote on the release candidate #3 for the version 2.4.0,
>>> as follows:
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>
>>> The complete staging area is available for your review, which includes:
>>> * JIRA release notes [1],
>>> * the official Apache source release to be deployed to dist.apache.org
>>> [2],
>>> which is signed with the key with fingerprint BDC9 89B0 1BD2 A463 6010
>>> A1CA
>>> 8F15 5E09 610D 69FB [3],
>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>> * source code tag "v2.4.0-RC3" [5],
>>> * website pull request listing the release and publishing the API
>>> reference
>>> manual [6].
>>> * Java artifacts were built with Maven 3.2.5 and OpenJDK 1.8.0_112.
>>> * Python artifacts are deployed along with the source release to the
>>> dist.apache.org [2].
>>>
>>> The validation spreadsheet is available at
>>>
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit?ts=5a1c7310#gid=1663314475
>>>
>>> The vote will be open for at least 72 hours. It is adopted by majority
>>> approval, with at least 3 PMC affirmative votes.
>>>
>>> Thanks,
>>> - Robert
>>>
>>> [1]
>>>
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342682=12319527
>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.4.0/
>>> [3] https://dist.apache.org/repos/dist/dev/beam/KEYS
>>> [4]
>>> https://repository.apache.org/content/repositories/orgapachebeam-1031/
>>> [5] https://github.com/apache/beam/tree/v2.4.0-RC3
>>> [6] https://github.com/apache/beam-site/pull/398
>>>
>>
>>


Re: [VOTE] Release 2.4.0, release candidate #3

2018-03-19 Thread Lukasz Cwik
+1 (binding), verified Java quickstart on Apex local, DirectRunner,
Dataflow, Flink local, Spark local.


On Mon, Mar 19, 2018 at 3:54 AM Romain Manni-Bucau 
wrote:

> -0 (cause of the teardown issue which is still a blocker), otherwise
> spark/direct runners work in my projects
>
>
> Romain Manni-Bucau
> @rmannibucau  |  Blog
>  | Old Blog
>  | Github
>  | LinkedIn
>  | Book
> 
>
> 2018-03-17 10:19 GMT+01:00 Robert Bradshaw :
>
>> Hi everyone,
>>
>> Please review and vote on the release candidate #3 for the version 2.4.0,
>> as follows:
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>>
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org
>> [2],
>> which is signed with the key with fingerprint BDC9 89B0 1BD2 A463 6010
>> A1CA
>> 8F15 5E09 610D 69FB [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "v2.4.0-RC3" [5],
>> * website pull request listing the release and publishing the API
>> reference
>> manual [6].
>> * Java artifacts were built with Maven 3.2.5 and OpenJDK 1.8.0_112.
>> * Python artifacts are deployed along with the source release to the
>> dist.apache.org [2].
>>
>> The validation spreadsheet is available at
>>
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit?ts=5a1c7310#gid=1663314475
>>
>> The vote will be open for at least 72 hours. It is adopted by majority
>> approval, with at least 3 PMC affirmative votes.
>>
>> Thanks,
>> - Robert
>>
>> [1]
>>
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342682=12319527
>> [2] https://dist.apache.org/repos/dist/dev/beam/2.4.0/
>> [3] https://dist.apache.org/repos/dist/dev/beam/KEYS
>> [4]
>> https://repository.apache.org/content/repositories/orgapachebeam-1031/
>> [5] https://github.com/apache/beam/tree/v2.4.0-RC3
>> [6] https://github.com/apache/beam-site/pull/398
>>
>
>


Re: [Proposal] Defining and Adding SDK Metrics - Looking for Feedback

2018-03-19 Thread Lukasz Cwik
Took a look, made some suggestions.


On Mon, Mar 19, 2018 at 11:46 AM Scott Wegner  wrote:

> Thanks for writing this up! I'll take a look and add comments in the doc.
>
> On Tue, Mar 13, 2018 at 8:55 PM Alex Amato  wrote:
>
>> Hello beam community,
>>
>> I have put together a proposal
>> ,
>> and I would like to get some initial feedback to improve upon the ideas
>> here.
>>
>> The proposal describes how metrics can be defined and communicated across
>> the Fn API. Additionally, the proposal describes how to collect custom
>> metrics without modifying the main beam protos. It also discusses how an
>> SDK and Runner can be gracefully updated separately to support new metrics.
>>
>> The Runner must request Metrics from the SDK using a MetricSpec which
>> describes the metric with a URN string and a corresponding proto. The SDK
>> can then reply with the populated Metric if it supports the metric's URN
>> and related protos.
>>
>> This approach should (hopefully) be general enough to implement all
>> metrics, including user metrics and allow for graceful updates, user
>> enabling/disabling metrics collections, etc.
>>
>> @Etienne
>> Would you be able to review this as well? I believe that you have also
>> been looking into Metrics as well and would like to get your feedback.
>>
>> Proposal (raw link):
>>
>> https://docs.google.com/document/d/1MtBZYV7NAcfbwyy9Op8STeFNBxtljxgy69FkHMvhTMA/edit
>>
>> Just an FYI, I will be offline until the 26th, but I wanted to send this
>> out and get some initial feedback. If there are any urgent questions,
>> Rafael can respond.
>>
>> Please take a look and let me know what you think. I look forward to
>> hearing your feedback
>> Thanks,
>> Alex
>>
> --
>
>
> Got feedback? http://go/swegner-feedback
>


Re: [Proposal] Defining and Adding SDK Metrics - Looking for Feedback

2018-03-19 Thread Scott Wegner
Thanks for writing this up! I'll take a look and add comments in the doc.

On Tue, Mar 13, 2018 at 8:55 PM Alex Amato  wrote:

> Hello beam community,
>
> I have put together a proposal
> ,
> and I would like to get some initial feedback to improve upon the ideas
> here.
>
> The proposal describes how metrics can be defined and communicated across
> the Fn API. Additionally, the proposal describes how to collect custom
> metrics without modifying the main beam protos. It also discusses how an
> SDK and Runner can be gracefully updated separately to support new metrics.
>
> The Runner must request Metrics from the SDK using a MetricSpec which
> describes the metric with a URN string and a corresponding proto. The SDK
> can then reply with the populated Metric if it supports the metric's URN
> and related protos.
>
> This approach should (hopefully) be general enough to implement all
> metrics, including user metrics and allow for graceful updates, user
> enabling/disabling metrics collections, etc.
>
> @Etienne
> Would you be able to review this as well? I believe that you have also
> been looking into Metrics as well and would like to get your feedback.
>
> Proposal (raw link):
>
> https://docs.google.com/document/d/1MtBZYV7NAcfbwyy9Op8STeFNBxtljxgy69FkHMvhTMA/edit
>
> Just an FYI, I will be offline until the 26th, but I wanted to send this
> out and get some initial feedback. If there are any urgent questions,
> Rafael can respond.
>
> Please take a look and let me know what you think. I look forward to
> hearing your feedback
> Thanks,
> Alex
>
-- 


Got feedback? http://go/swegner-feedback


Re: [VOTE] Release 2.4.0, release candidate #3

2018-03-19 Thread Romain Manni-Bucau
-0 (cause of the teardown issue which is still a blocker), otherwise
spark/direct runners work in my projects


Romain Manni-Bucau
@rmannibucau  |  Blog
 | Old Blog
 | Github  |
LinkedIn  | Book


2018-03-17 10:19 GMT+01:00 Robert Bradshaw :

> Hi everyone,
>
> Please review and vote on the release candidate #3 for the version 2.4.0,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2],
> which is signed with the key with fingerprint BDC9 89B0 1BD2 A463 6010 A1CA
> 8F15 5E09 610D 69FB [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.4.0-RC3" [5],
> * website pull request listing the release and publishing the API reference
> manual [6].
> * Java artifacts were built with Maven 3.2.5 and OpenJDK 1.8.0_112.
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
>
> The validation spreadsheet is available at
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-
> oLFo_ZXBpJw/edit?ts=5a1c7310#gid=1663314475
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> - Robert
>
> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> version=12342682=12319527
> [2] https://dist.apache.org/repos/dist/dev/beam/2.4.0/
> [3] https://dist.apache.org/repos/dist/dev/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1031/
> [5] https://github.com/apache/beam/tree/v2.4.0-RC3
> [6] https://github.com/apache/beam-site/pull/398
>


Jenkins build is back to normal : beam_Release_NightlySnapshot #718

2018-03-19 Thread Apache Jenkins Server
See