Hey JB,

I added some instructions about how to create python wheels in this PR:
https://github.com/apache/beam-site/pull/467 . Hope it would be helpful.

Boyuan

On Tue, Jun 12, 2018 at 12:33 PM Jean-Baptiste Onofré <j...@nanthrax.net>
wrote:

> Hi all,
>
> due to issues found by Udi and Pablo, I cancel RC1 to prepare a new
> release.
>
> Thanks Udi and Pablo for catching up.
>
> I will prepare a RC2 after cherry pick/fixed the found issues (tomorrow
> morning my time).
>
> Regards
> JB
>
>
> On 12/06/2018 21:04, Pablo Estrada wrote:
> > Hello all,
> > I've found another release blocker: The current findbugs dependency in
> > build_rules.gradle has a GPL license. Here's the PR to depend on the
> > correct findbugs: https://github.com/apache/beam/pull/5609
> >
> > Best
> > -P.
> >
> > On Tue, Jun 12, 2018 at 11:03 AM Udi Meiri <eh...@google.com
> > <mailto:eh...@google.com>> wrote:
> >
> >     Cherrypick created: https://github.com/apache/beam/pull/5607
> >     Tests still running.
> >
> >     On Tue, Jun 12, 2018 at 9:59 AM Udi Meiri <eh...@google.com
> >     <mailto:eh...@google.com>> wrote:
> >
> >         -1: Would like to cherry pick a fix
> >         for https://issues.apache.org/jira/browse/BEAM-4536
> >
> >         On Tue, Jun 12, 2018 at 9:48 AM Jean-Baptiste Onofré
> >         <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote:
> >
> >             Any update about your vote and fix ?
> >
> >             Thanks !
> >             Regards
> >             JB
> >
> >             On 12/06/2018 04:02, Udi Meiri wrote:
> >             > Another bug: reading from PubSub with_attributes=True is
> >             broken on
> >             > Python with Dataflow.
> >             > https://issues.apache.org/jira/browse/BEAM-4536
> >             >
> >             > JB, I'm making a PR that removes this keyword and I'd like
> >             to propose it
> >             > as a cherrypick to 2.5.0.
> >             > (feature should be fixed in the next release)
> >             >
> >             > On Mon, Jun 11, 2018 at 6:19 PM Chamikara Jayalath
> >             <chamik...@google.com <mailto:chamik...@google.com>
> >             > <mailto:chamik...@google.com
> >             <mailto:chamik...@google.com>>> wrote:
> >             >
> >             >     FYI: looks like Python tests are failing for Windows.
> JIRA
> >             >     is https://issues.apache.org/jira/browse/BEAM-4535.
> >             >
> >             >     I don't think this is a release blocker but this
> >             should probably go
> >             >     in release notes (for any user that tries to run tests
> >             on Python
> >             >     source build). And we should try to incorporate a fix
> >             if we happen
> >             >     to cut another release candidate for some reason.
> >             >
> >             >     Thanks,
> >             >     Cham
> >             >
> >             >     On Mon, Jun 11, 2018 at 5:46 PM Pablo Estrada
> >             <pabl...@google.com <mailto:pabl...@google.com>
> >             >     <mailto:pabl...@google.com
> >             <mailto:pabl...@google.com>>> wrote:
> >             >
> >             >         Thanks everyone who has pitched in to validate the
> >             release!
> >             >
> >             >         Boyuan Zhang and I have also run a few pipelines,
> >             and verified
> >             >         that they work properly (see release validation
> >             spreadsheet[1]).
> >             >
> >             >         We have also found that the Game Stats pipeline is
> >             failing in
> >             >         Python Streaming Dataflow. I have filed
> >             BEAM-4534[2]. This is
> >             >         not a blocker, since Python streaming is not yet
> >             fully supported.
> >             >
> >             >         It seems that the uploaded artifacts look good.
> >             >
> >             >         We have noticed that the Python artifacts are
> >             still missing
> >             >         Python wheel files (compare [3] and [4]). JB,
> >             could you please
> >             >         add the wheel files? Boyuan and I can try to help
> >             you prepare
> >             >         them / upload them if necessary. Please let us
> know.
> >             >
> >             >         Thanks again!
> >             >         -P.
> >             >
> >             >
> >              [1]
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=152451807
> >             >         [2]
> https://issues.apache.org/jira/browse/BEAM-4534
> >             >         [3]
> https://dist.apache.org/repos/dist/dev/beam/2.4.0/
> >             >         [4]
> https://dist.apache.org/repos/dist/dev/beam/2.5.0/
> >             >
> >             >         On Mon, Jun 11, 2018 at 12:37 PM Alan Myrvold
> >             >         <amyrv...@google.com <mailto:amyrv...@google.com>
> >             <mailto:amyrv...@google.com <mailto:amyrv...@google.com>>>
> >             wrote:
> >             >
> >             >             +1 (non-binding)
> >             >
> >             >             tested some of the quickstarts
> >             >
> >             >             On Sun, Jun 10, 2018 at 1:39 AM Tim
> >             >             <timrobertson...@gmail.com
> >             <mailto:timrobertson...@gmail.com>
> >             >             <mailto:timrobertson...@gmail.com
> >             <mailto:timrobertson...@gmail.com>>> wrote:
> >             >
> >             >                 Tested by our team:
> >             >                 - mvn inclusion
> >             >                 - Avro, ES, Hadoop IF IO
> >             >                 - Pipelines run on Spark (Cloudera 5.12.0
> >             YARN cluster)
> >             >                 - Reviewed release notes
> >             >
> >             >                 +1
> >             >
> >             >                 Thanks also to everyone who helped get
> >             over the gradle
> >             >                 hurdle and in particular to JB.
> >             >
> >             >                 Tim
> >             >
> >             >                 > On 9 Jun 2018, at 05:56, Jean-Baptiste
> >             Onofré
> >             >                 <j...@nanthrax.net <mailto:j...@nanthrax.net>
> >             <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net>>> wrote:
> >             >                 >
> >             >                 > No problem Pablo.
> >             >                 >
> >             >                 > The vote period is a minimum, it can be
> >             extended as
> >             >                 requested or if we
> >             >                 > don't have the minimum of 3 binding
> votes.
> >             >                 >
> >             >                 > Regards
> >             >                 > JB
> >             >                 >
> >             >                 >> On 09/06/2018 01:54, Pablo Estrada
> wrote:
> >             >                 >> Hello all,
> >             >                 >> I'd like to request an extension of the
> >             voting period
> >             >                 until Monday
> >             >                 >> evening (US time, so later in other
> >             geographical
> >             >                 regions). This is
> >             >                 >> because we were only now able to
> >             publish Dataflow
> >             >                 Workers, and have not
> >             >                 >> had the chance to run release
> >             validation tests on
> >             >                 them. The extension
> >             >                 >> will allow us to validate and vote by
> >             Monday.
> >             >                 >> Is this acceptable to the community?
> >             >                 >>
> >             >                 >> Best
> >             >                 >> -P.
> >             >                 >>
> >             >                 >> On Fri, Jun 8, 2018 at 6:20 AM Alexey
> >             Romanenko
> >             >                 >> <aromanenko....@gmail.com
> >             <mailto:aromanenko....@gmail.com>
> >             >                 <mailto:aromanenko....@gmail.com
> >             <mailto:aromanenko....@gmail.com>>
> >             >                 <mailto:aromanenko....@gmail.com
> >             <mailto:aromanenko....@gmail.com>
> >             >                 <mailto:aromanenko....@gmail.com
> >             <mailto:aromanenko....@gmail.com>>>> wrote:
> >             >                 >>
> >             >                 >>    Thank you JB for your work!
> >             >                 >>
> >             >                 >>    I tested running simple streaming
> >             (/KafkaIO/) and
> >             >                 batch (/TextIO /
> >             >                 >>    HDFS/) pipelines with SparkRunner on
> >             YARN cluster
> >             >                 - it works fine.
> >             >                 >>
> >             >                 >>    WBR,
> >             >                 >>    Alexey
> >             >                 >>
> >             >                 >>
> >             >                 >>>    On 8 Jun 2018, at 10:00, Etienne
> >             Chauchot
> >             >                 <echauc...@apache.org
> >             <mailto:echauc...@apache.org> <mailto:echauc...@apache.org
> >             <mailto:echauc...@apache.org>>
> >             >                 >>>    <mailto:echauc...@apache.org
> >             <mailto:echauc...@apache.org>
> >             >                 <mailto:echauc...@apache.org
> >             <mailto:echauc...@apache.org>>>> wrote:
> >             >                 >>>
> >             >                 >>>    I forgot to vote:
> >             >                 >>>    +1 (non binding).
> >             >                 >>>    What I tested:
> >             >                 >>>    - no functional or performance
> >             regression
> >             >                 comparing to v2.4
> >             >                 >>>    - dependencies in the poms are ok
> >             >                 >>>
> >             >                 >>>    Etienne
> >             >                 >>>>    Le vendredi 08 juin 2018 à 08:27
> >             +0200, Romain
> >             >                 Manni-Bucau a écrit :
> >             >                 >>>>    +1 (non-binding), mainstream usage
> >             is not broken
> >             >                 by the pom
> >             >                 >>>>    changes and runtime has no known
> >             regression
> >             >                 compared to the 2.4.0
> >             >                 >>>>
> >             >                 >>>>    (side note: kudo to JB for this
> >             build tool
> >             >                 change release, I know
> >             >                 >>>>    how it can hurt ;))
> >             >                 >>>>
> >             >                 >>>>    Romain Manni-Bucau
> >             >                 >>>>    @rmannibucau
> >             <https://twitter.com/rmannibucau>
> >             >                 |  Blog
> >             >                 >>>>
> >             <https://rmannibucau.metawerx.net/> | Old Blog
> >             >                 >>>>
> >             <http://rmannibucau.wordpress.com/> | Github
> >             >                 >>>>    <https://github.com/rmannibucau> |
> >             LinkedIn
> >             >                 >>>>
> >             <https://www.linkedin.com/in/rmannibucau> | Book
> >             >                 >>>>
> >             >
> >              <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> >             >                 >>>>
> >             >                 >>>>
> >             >                 >>>>    Le jeu. 7 juin 2018 à 16:17,
> >             Jean-Baptiste Onofré
> >             >                 >>>>    <j...@nanthrax.net
> >             <mailto:j...@nanthrax.net> <mailto:j...@nanthrax.net
> >             <mailto:j...@nanthrax.net>>
> >             >                 <mailto:j...@nanthrax.net
> >             <mailto:j...@nanthrax.net> <mailto:j...@nanthrax.net
> >             <mailto:j...@nanthrax.net>>>> a écrit :
> >             >                 >>>>>    Thanks for the details Etienne !
> >             >                 >>>>>
> >             >                 >>>>>    The good news is that the
> >             artifacts seem OK and
> >             >                 the overall Nexmark
> >             >                 >>>>>    results are consistent with the
> >             2.4.0 release ones.
> >             >                 >>>>>
> >             >                 >>>>>    I'm starting a complete review
> >             using the
> >             >                 beam-samples as well.
> >             >                 >>>>>
> >             >                 >>>>>    Regards
> >             >                 >>>>>    JB
> >             >                 >>>>>
> >             >                 >>>>>>    On 07/06/2018 16:14, Etienne
> >             Chauchot wrote:
> >             >                 >>>>>> Hi,
> >             >                 >>>>>> I've just run the nexmark queries
> >             on v2.5.0-RC1 tag
> >             >                 >>>>>> What we can notice:
> >             >                 >>>>>> - query 3 (exercises CoGroupByKey,
> >             state and
> >             >                 timer) shows
> >             >                 >>>>>    different
> >             >                 >>>>>> output with DR between batch and
> >             streaming and
> >             >                 with the other
> >             >                 >>>>>    runners =>
> >             >                 >>>>>> I compared with v2.4 there were
> >             still these
> >             >                 differences but with
> >             >                 >>>>>> different output size numbers
> >             >                 >>>>>>
> >             >                 >>>>>> - query 6 (exercises specialized
> >             combiner) shows
> >             >                 different output
> >             >                 >>>>>> between the runners => the correct
> >             output is 401.
> >             >                 strange that
> >             >                 >>>>>    in batch
> >             >                 >>>>>> mode some runners output les
> >             Sellers. I compared
> >             >                 with v2.4
> >             >                 >>>>>    same output
> >             >                 >>>>>>
> >             >                 >>>>>> - response time of query 7
> >             (exercices Max
> >             >                 transform, fanout
> >             >                 >>>>>    and side
> >             >                 >>>>>> input) is very slow on DR => I
> >             compared with v2.4
> >             >                 , comparable
> >             >                 >>>>>    execution
> >             >                 >>>>>> times
> >             >                 >>>>>>
> >             >                 >>>>>> I'm not comparing q10 because it is
> >             a write to
> >             >                 GCS so it is
> >             >                 >>>>>    very specific.
> >             >                 >>>>>>
> >             >                 >>>>>> => Basically no regression
> >             comparing to v2.4
> >             >                 >>>>>>
> >             >                 >>>>>> For the record here is the output
> >             (waiting for
> >             >                 ongoing perfkit
> >             >                 >>>>>    integration):
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>> 1. DR batch
> >             >                 >>>>>>
> >             >                 >>>>>> Performance:
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    Conf  Runtime(sec)    (Baseline)
> >             Events(/sec)
> >             >                   (Baseline)       Results    (Baseline)
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0000           5,8
> >                  17283,1
> >             >                                     100000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0001           3,2
> >                  31104,2
> >             >                                      92000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0002           1,2
> >                  82918,7
> >             >                                        351
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0003           2,2
> >                  46210,7
> >             >                                        458
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0004           1,2
> >                   8503,4
> >             >                                         40
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0005           4,0
> >                  25220,7
> >             >                                         12
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0006           0,9
> >                  11148,3
> >             >                                        401
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0007          13,2
> >                   7580,9
> >             >                                          1
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0008           1,5
> >                  67340,1
> >             >                                       6000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0009           0,7
> >                  14025,2
> >             >                                        298
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0010          12,8
> >                   7793,0
> >             >                                          1
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0011           2,4
> >                  42319,1
> >             >                                       1919
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0012           1,6
> >                  61462,8
> >             >                                       1919
> >             >                 >>>>>>
> >             >                 >>>>>
> >             >
> >
>   
> ==========================================================================================
> >             >                 >>>>>>
> >             >                 >>>>>> 2. DR streaming
> >             >                 >>>>>>
> >             >                 >>>>>> Performance:
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    Conf  Runtime(sec)    (Baseline)
> >             Events(/sec)
> >             >                   (Baseline)       Results    (Baseline)
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0000           6,5
> >                  15285,8
> >             >                                     100000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0001           3,7
> >                  27397,3
> >             >                                      92000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0002           1,4
> >                  69108,5
> >             >                                        351
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0003           3,2
> >                  31181,8
> >             >                                        447
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0004           1,2
> >                   8361,2
> >             >                                         40
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0005           5,3
> >                  18903,6
> >             >                                         12
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0006           0,9
> >                  11111,1
> >             >                                        401
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0007          82,5
> >                   1212,2
> >             >                                          1
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0008           2,0
> >                  51072,5
> >             >                                       6000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0009           0,8
> >                  12903,2
> >             >                                        298
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0010          49,5
> >                   2021,8
> >             >                                          1
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0011           3,9
> >                  25667,4
> >             >                                       1919
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0012           2,4
> >                  41067,8
> >             >                                       1919
> >             >                 >>>>>>
> >             >                 >>>>>
> >             >
> >
>   
> ==========================================================================================
> >             >                 >>>>>>
> >             >                 >>>>>> 3. Flink batch
> >             >                 >>>>>> Performance:
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    Conf  Runtime(sec)    (Baseline)
> >             Events(/sec)
> >             >                   (Baseline)       Results    (Baseline)
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0000           1,0
> >                  97656,3
> >             >                                     100000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0001           0,7
> >                 141643,1
> >             >                                      92000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0002           0,4
> >                 228310,5
> >             >                                        351
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0003           1,6
> >                  64020,5
> >             >                                        580
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0004           0,7
> >                  13831,3
> >             >                                         40
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0005           1,4
> >                  72939,5
> >             >                                         12
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0006           0,5
> >                  20491,8
> >             >                                        103
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0007           1,3
> >                  74239,0
> >             >                                          1
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0008           0,8
> >                 121506,7
> >             >                                       6000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0009           0,6
> >                  17953,3
> >             >                                        298
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0010           1,3
> >                  74682,6
> >             >                                          1
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0011           1,1
> >                  92936,8
> >             >                                       1919
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0012           0,8
> >                 123001,2
> >             >                                       1919
> >             >                 >>>>>>
> >             >                 >>>>>
> >             >
> >
>   
> ==========================================================================================
> >             >                 >>>>>>
> >             >                 >>>>>> 4. Flink streaming
> >             >                 >>>>>> Performance:
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    Conf  Runtime(sec)    (Baseline)
> >             Events(/sec)
> >             >                   (Baseline)       Results    (Baseline)
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0000           5,4
> >                  18677,6
> >             >                                     100000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0001           2,8
> >                  35511,4
> >             >                                      92000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0002           1,8
> >                  54318,3
> >             >                                        351
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0003           2,4
> >                  41614,6
> >             >                                        580
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0004           1,0
> >                  10341,3
> >             >                                         40
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0005           3,4
> >                  29568,3
> >             >                                         12
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0006           0,7
> >                  13369,0
> >             >                                        401
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0007           2,8
> >                  36192,5
> >             >                                          1
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0008           1,8
> >                  54854,6
> >             >                                       6000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0009           0,7
> >                  13369,0
> >             >                                        298
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0010           3,4
> >                  29841,8
> >             >                                          2
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0011           5,0
> >                  19932,2
> >             >                                       1919
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0012           2,6
> >                  38835,0
> >             >                                       1919
> >             >                 >>>>>>
> >             >                 >>>>>
> >             >
> >
>   
> ==========================================================================================
> >             >                 >>>>>>
> >             >                 >>>>>> 5. Spark batch
> >             >                 >>>>>> Performance:
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    Conf  Runtime(sec)    (Baseline)
> >             Events(/sec)
> >             >                   (Baseline)       Results    (Baseline)
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0000           1,5
> >                  65445,0
> >             >                                     100000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0001           1,3
> >                  79491,3
> >             >                                      92000
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0002           0,9
> >                 112107,6
> >             >                                        351
> >             >                 >>>>>>
> >             >                 >>>>>>
> >             >                 >>>>>    0003           2,0
> >                  48804,3
> >             >                                        580
> >             >                 >>

Reply via email to