Add https://issues.apache.org/jira/browse/BEAM-2833 which is a blocker to merge DSL_SQL. There may be something wrong in the back-end(maybe RunnerApi) to handle parametered CustomCoder in TestPipeline.
On Thu, Aug 31, 2017 at 10:38 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Fair enough. > > That's fine for me. > > Regards > JB > > On Aug 31, 2017, 19:03, at 19:03, Steve Niemitz <sniem...@apache.org> > wrote: > >I'll chime in as a user who would love to see 2.2.0 sooner than later, > >specifically for the file IO Eugene mentioned. We're using the AvroIO > >enhancements extensively, but I am hesitant to run from HEAD in master > >in > >production. > > > >On Thu, Aug 31, 2017 at 12:42 PM, Eugene Kirpichov < > >kirpic...@google.com.invalid> wrote: > > > >> There are a lot of users including very large production customers > >who have > >> been asking specifically for the features that are in 2.2.0 (most of > >them > >> accumulated while 2.1.0 was being iterated on) - mostly I'm referring > >to > >> the vastly improved file IO - and they have been hesitant to use Beam > >at > >> HEAD in production. I think the slight unusualness of having a > >release > >> published soon after the previous release is a small price to pay for > >> helping those users :) > >> > >> On Wed, Aug 30, 2017, 11:30 PM Jean-Baptiste Onofré <j...@nanthrax.net> > >> wrote: > >> > >> > As we released 2.1.0 couple of weeks ago, it could sound weird to > >the > >> > users to > >> > do a 2.2.0 so fast. If we have a blocking issue, we can do a 2.1.1 > >If > >> it's > >> > new > >> > features, why not having a release pace in October (2.2.0) ? > >> > > >> > Thoughts ? > >> > > >> > Regards > >> > JB > >> > > >> > On 08/31/2017 08:27 AM, Eugene Kirpichov wrote: > >> > > I'd suggest to do 2.2.0 as quickly as possible, and target 2.3.0 > >for > >> > > October. I don't see a reason to delay 2.2.0 until October: > >there's a > >> > huge > >> > > amount of features worth releasing between when 2.1.0 was cut and > >the > >> > > current HEAD. > >> > > > >> > > On Wed, Aug 30, 2017 at 10:18 PM Jean-Baptiste Onofré > ><j...@nanthrax.net > >> > > >> > > wrote: > >> > > > >> > >> With a 2.2.0 in October, I think we can try to move forward on > >> RedisIO. > >> > >> > >> > >> I'm now back from vacation and I will resume the work on this > >IO. > >> > >> > >> > >> Regards > >> > >> JB > >> > >> > >> > >> On 08/30/2017 11:27 PM, Eugene Kirpichov wrote: > >> > >>> RedisIO in 2.2.0 is very unlikely. There's still a lot of > >review > >> > >> remaining > >> > >>> last time I checked on the PR. > >> > >>> > >> > >>> On Wed, Aug 30, 2017 at 2:24 PM Vilhelm von Ehrenheim < > >> > >>> vonehrenh...@gmail.com> wrote: > >> > >>> > >> > >>>> Any chance to get the RedisIO in this release? > >> > >>>> [BEAM-1017] Add RedisIO #1687 > >> > >>>> > >> > >>>> Its not my PR but ll be happy to assist if there is anything I > >can > >> do > >> > to > >> > >>>> help. > >> > >>>> > >> > >>>> On 30 Aug 2017 22:46, "Daniel Ribeiro" > >> <danie...@squareup.com.invalid > >> > > > >> > >>>> wrote: > >> > >>>> > >> > >>>>> It would be great to get a bump on pubsub > >> > >>>>> <https://github.com/apache/beam/blob/master/pom.xml#L145> > >> > dependency. > >> > >> It > >> > >>>>> is > >> > >>>>> currently very outdated (v1-rev10-1.22.0, which was released > >over a > >> > >> year > >> > >>>>> ago > >> > >>>>> <http://repo1.maven.org/maven2/com/google/apis/google- > >> > >>>>> api-services-pubsub/v1-rev10-1.22.0/> > >> > >>>>> ). > >> > >>>>> > >> > >>>>> > >> > >>>>> On Wed, Aug 30, 2017 at 1:27 PM, Eugene Kirpichov < > >> > >>>>> kirpic...@google.com.invalid> wrote: > >> > >>>>> > >> > >>>>>> Thanks Ismael. I've marked these two issues for fix in > >2.2.0. > >> > >>>> Definitely > >> > >>>>>> agree that at least the first one must be fixed. > >> > >>>>>> > >> > >>>>>> Here's the current burndown list > >> > >>>>>> > >https://issues.apache.org/jira/projects/BEAM/versions/12341044 - > >> we > >> > >>>>> should > >> > >>>>>> clean it up. > >> > >>>>>> > >> > >>>>>> On Wed, Aug 30, 2017 at 1:20 PM Ismaël Mejía > ><ieme...@gmail.com> > >> > >>>> wrote: > >> > >>>>>> > >> > >>>>>>> The current master has accumulated a good amount of nice > >features > >> > >>>>>>> since 2.1.0 so a new release is welcomed. I have two > >JIRAs/PR > >> that > >> > I > >> > >>>>>>> think are important to check/solve before the cut: > >> > >>>>>>> > >> > >>>>>>> BEAM-2516 (this is a regression on the performance of > >Direct > >> runner > >> > >>>> on > >> > >>>>>>> Java). We had never really defined if a performance > >regression is > >> > >>>>>>> critical to be a blocker. I executed WordCount with the > >> > kinglear.txt > >> > >>>>>>> (170KB) file in version 2.1.0 vs the current 2.2.0-SNAPSHOT > >and I > >> > >>>>>>> found that the execution time passed from 5s to 126s. So > >maybe we > >> > >>>> need > >> > >>>>>>> to review this one before the release. I can understand if > >others > >> > >>>>>>> consider this a minor issue because the Direct runner is > >not > >> > supposed > >> > >>>>>>> to be used for production, but this performance regression > >can > >> > cause > >> > >>>> a > >> > >>>>>>> bad impression for a casual user starting with Beam. > >> > >>>>>>> > >> > >>>>>>> BEAM-2790 (fix reading from Amazon S3 via > >HadoopFileSystem). I > >> > think > >> > >>>>>>> this one is a nice to have. I am not sure that I can tackle > >it > >> for > >> > >>>> the > >> > >>>>>>> wednesday cut. I’m OOO until the beginning of next week, > >but > >> maybe > >> > >>>>>>> someone else can take a look. In the worst case this is not > >a > >> > release > >> > >>>>>>> blocker but definitely a really nice fix to include. > >> > >>>>>>> > >> > >>>>>>> > >> > >>>>>>> On Wed, Aug 30, 2017 at 8:49 PM, Eugene Kirpichov > >> > >>>>>>> <kirpic...@google.com.invalid> wrote: > >> > >>>>>>>> I'd like to get the following PRs into 2.2.0: > >> > >>>>>>>> > >> > >>>>>>>> #3765 <https://github.com/apache/beam/pull/3765> > >[BEAM-2753 > >> > >>>>>>>> <https://issues.apache.org/jira/browse/BEAM-2753>] Fixes > >> > >>>> translation > >> > >>>>>> of > >> > >>>>>>>> WriteFiles side inputs (important bugfix for > >DynamicDestinations > >> > in > >> > >>>>>>> files) > >> > >>>>>>>> #3725 <https://github.com/apache/beam/pull/3725> > >[BEAM-2827 > >> > >>>>>>>> <https://issues.apache.org/jira/browse/BEAM-2827>] > >Introduces > >> > >>>>>>>> AvroIO.watchForNewFiles (parity for AvroIO with TextIO in > >a few > >> > >>>>>> important > >> > >>>>>>>> features) > >> > >>>>>>>> #3759 <https://github.com/apache/beam/pull/3759> > >[BEAM-2828 > >> > >>>>>>>> <https://issues.apache.org/jira/browse/BEAM-2828>] Moves > >Match > >> > >>>> into > >> > >>>>>>>> FileIO.match()/matchAll() (to prevent releasing current > >> > >>>>>>>> Match.filepatterns() into 2.2.0 and then having to keep it > >under > >> > >>>> that > >> > >>>>>>> name) > >> > >>>>>>>> > >> > >>>>>>>> On Wed, Aug 30, 2017 at 11:31 AM Mingmin Xu > ><mingm...@gmail.com > >> > > >> > >>>>>> wrote: > >> > >>>>>>>> > >> > >>>>>>>>> Glad to see that 2.2.0 is coming. Can we include SQL > >feature in > >> > >>>> next > >> > >>>>>>>>> release? We're in the final stage and expect to merge > >back to > >> > >>>> master > >> > >>>>>>> this > >> > >>>>>>>>> week. > >> > >>>>>>>>> > >> > >>>>>>>>> On Wed, Aug 30, 2017 at 11:27 AM, Reuven Lax > >> > >>>>> <re...@google.com.invalid > >> > >>>>>>> > >> > >>>>>>>>> wrote: > >> > >>>>>>>>> > >> > >>>>>>>>>> Now that Beam 2.1.0 has finally completed, I think we > >should > >> cut > >> > >>>>>> Beam > >> > >>>>>>>>> 2.2.0 > >> > >>>>>>>>>> soon. I volunteer to coordinate this release. > >> > >>>>>>>>>> > >> > >>>>>>>>>> Are there any pending pull requests that people think > >should > >> be > >> > >>>>>> merged > >> > >>>>>>>>>> before we cut 2.2.0? If so, please let me know soon, as > >I > >> would > >> > >>>>> like > >> > >>>>>>> to > >> > >>>>>>>>> cut > >> > >>>>>>>>>> by Wednesday of next week. > >> > >>>>>>>>>> > >> > >>>>>>>>>> Thanks, > >> > >>>>>>>>>> > >> > >>>>>>>>>> Reuven > >> > >>>>>>>>>> > >> > >>>>>>>>> > >> > >>>>>>>>> > >> > >>>>>>>>> > >> > >>>>>>>>> -- > >> > >>>>>>>>> ---- > >> > >>>>>>>>> Mingmin > >> > >>>>>>>>> > >> > >>>>>>> > >> > >>>>>> > >> > >>>>> > >> > >>>> > >> > >>> > >> > >> > >> > >> -- > >> > >> Jean-Baptiste Onofré > >> > >> jbono...@apache.org > >> > >> http://blog.nanthrax.net > >> > >> Talend - http://www.talend.com > >> > >> > >> > > > >> > > >> > -- > >> > Jean-Baptiste Onofré > >> > jbono...@apache.org > >> > http://blog.nanthrax.net > >> > Talend - http://www.talend.com > >> > > >> > -- ---- Mingmin
