Re: Slack Channel Request
Georg, Invite sent to you. On Fri, Apr 14, 2017 at 11:06 AM, Georg Heiler wrote: > Please add me toi > > Mingmin Xu schrieb am Fr. 14. Apr. 2017 um 07:06: > >> both sent. >> >> On Thu, Apr 13, 2017 at 10:00 PM, Anant Bhandarkar > impactanalytics.co> wrote: >> >>> Would love to be part of beam group on slack. >>> Also please add anil.b...@impactanaytics.co >>> Thanks, >>> Anant >>> >>> On 14-Apr-2017 9:15 AM, "Mingmin Xu" wrote: >>> @James, @Jingsong, @Tom, invite sent. On Thu, Apr 13, 2017 at 8:34 PM, Tom Pollard < tpoll...@flashpoint-intel.com> wrote: > If it's not inconvenient, I'd also like an invitation to the Slack > channel. > > Tom > > > On Apr 13, 2017, at 11:31 PM, JingsongLee > wrote: > > Please add me too. > > Best, > > JingsongLee > > > -- > From:James > Time:2017 Apr 14 (Fri) 11:00 > To:user > Subject:Re: Slack Channel Request > > Could I also have an invite please? > > On 2017-03-28 08:28 (+0800), Davor Bonaci wrote: > > Invite sent. > > > > On Sat, Mar 25, 2017 at 2:48 AM, Prabeesh K. > wrote: > > > > > Hi Jean, > > > > > > Thank you for your reply. I am eagerly waiting for the > other options. > > > > > > Regards, > > > Prabeesh K. > > > > > > On 25 March 2017 at 10:08, Jean-Baptiste Onofré > wrote: > > > > > >> Unfortunately we reached the max number of people on Slack (90). > > >> > > >> Let me see what we can do. > > >> > > >> Regards > > >> JB > > >> > > >> > > >> On 03/24/2017 09:49 PM, Prabeesh K. wrote: > > >> > > >>> Hi, > > >>> > > >>> Can someone please add me to the Apache Beam slack channel? > > >>> > > >>> Regards, > > >>> > > >>> Prabeesh K. > > >>> > > >>> > > >> -- > > >> Jean-Baptiste Onofré > > >> jbono...@apache.org > > >> http://blog.nanthrax.net > > >> Talend - http://www.talend.com > > >> > > > > > > > > > > > -- Mingmin >>> >> >> >> -- >> >> Mingmin >> >
Re: Slack Channel Request
Please add me toi Mingmin Xu schrieb am Fr. 14. Apr. 2017 um 07:06: > both sent. > > On Thu, Apr 13, 2017 at 10:00 PM, Anant Bhandarkar < > anant.bhandar...@impactanalytics.co> wrote: > >> Would love to be part of beam group on slack. >> Also please add anil.b...@impactanaytics.co >> Thanks, >> Anant >> >> On 14-Apr-2017 9:15 AM, "Mingmin Xu" wrote: >> >>> @James, @Jingsong, @Tom, invite sent. >>> >>> On Thu, Apr 13, 2017 at 8:34 PM, Tom Pollard < >>> tpoll...@flashpoint-intel.com> wrote: >>> If it's not inconvenient, I'd also like an invitation to the Slack channel. Tom On Apr 13, 2017, at 11:31 PM, JingsongLee wrote: Please add me too. Best, JingsongLee -- From:James Time:2017 Apr 14 (Fri) 11:00 To:user Subject:Re: Slack Channel Request Could I also have an invite please? On 2017-03-28 08:28 (+0800), Davor Bonaci wrote: > Invite sent. > > On Sat, Mar 25, 2017 at 2:48 AM, Prabeesh K. >>> > wrote: > > > Hi Jean, > > > > Thank you for your reply. I am eagerly waiting for the other options. > > > > Regards, > > Prabeesh K. > > > > On 25 March 2017 at 10:08, Jean-Baptiste Onofré >>> > wrote: > > > >> Unfortunately we reached the max number of people on Slack (90). > >> > >> Let me see what we can do. > >> > >> Regards > >> JB > >> > >> > >> On 03/24/2017 09:49 PM, Prabeesh K. wrote: > >> > >>> Hi, > >>> > >>> Can someone please add me to the Apache Beam slack channel? > >>> > >>> Regards, > >>> > >>> Prabeesh K. > >>> > >>> > >> -- > >> Jean-Baptiste Onofré > >> jbono...@apache.org > >> http://blog.nanthrax.net > >> Talend - http://www.talend.com > >> > > > > > >>> >>> >>> -- >>> >>> Mingmin >>> >> > > > -- > > Mingmin >
Re: Slack Channel Request
both sent. On Thu, Apr 13, 2017 at 10:00 PM, Anant Bhandarkar < anant.bhandar...@impactanalytics.co> wrote: > Would love to be part of beam group on slack. > Also please add anil.b...@impactanaytics.co > Thanks, > Anant > > On 14-Apr-2017 9:15 AM, "Mingmin Xu" wrote: > >> @James, @Jingsong, @Tom, invite sent. >> >> On Thu, Apr 13, 2017 at 8:34 PM, Tom Pollard < >> tpoll...@flashpoint-intel.com> wrote: >> >>> If it's not inconvenient, I'd also like an invitation to the Slack >>> channel. >>> >>> Tom >>> >>> >>> On Apr 13, 2017, at 11:31 PM, JingsongLee >>> wrote: >>> >>> Please add me too. >>> >>> Best, >>> >>> JingsongLee >>> >>> >>> -- >>> From:James >>> Time:2017 Apr 14 (Fri) 11:00 >>> To:user >>> Subject:Re: Slack Channel Request >>> >>> Could I also have an invite please? >>> >>> On 2017-03-28 08:28 (+0800), Davor Bonaci wrote: >>> > Invite sent. >>> > >>> > On Sat, Mar 25, 2017 at 2:48 AM, Prabeesh K. >> > wrote: >>> > >>> > > Hi Jean, >>> > > >>> > > Thank you for your reply. I am eagerly waiting for the o >>> ther options. >>> > > >>> > > Regards, >>> > > Prabeesh K. >>> > > >>> > > On 25 March 2017 at 10:08, Jean-Baptiste Onofré >> > wrote: >>> > > >>> > >> Unfortunately we reached the max number of people on Slack (90). >>> > >> >>> > >> Let me see what we can do. >>> > >> >>> > >> Regards >>> > >> JB >>> > >> >>> > >> >>> > >> On 03/24/2017 09:49 PM, Prabeesh K. wrote: >>> > >> >>> > >>> Hi, >>> > >>> >>> > >>> Can someone please add me to the Apache Beam slack channel? >>> > >>> >>> > >>> Regards, >>> > >>> >>> > >>> Prabeesh K. >>> > >>> >>> > >>> >>> > >> -- >>> > >> Jean-Baptiste Onofré >>> > >> jbono...@apache.org >>> > >> http://blog.nanthrax.net >>> > >> Talend - http://www.talend.com >>> > >> >>> > > >>> > > >>> > >>> >>> >>> >> >> >> -- >> >> Mingmin >> > -- Mingmin
Re: Slack Channel Request
Would love to be part of beam group on slack. Also please add anil.b...@impactanaytics.co Thanks, Anant On 14-Apr-2017 9:15 AM, "Mingmin Xu" wrote: > @James, @Jingsong, @Tom, invite sent. > > On Thu, Apr 13, 2017 at 8:34 PM, Tom Pollard < > tpoll...@flashpoint-intel.com> wrote: > >> If it's not inconvenient, I'd also like an invitation to the Slack >> channel. >> >> Tom >> >> >> On Apr 13, 2017, at 11:31 PM, JingsongLee >> wrote: >> >> Please add me too. >> >> Best, >> >> JingsongLee >> >> >> -- >> From:James >> Time:2017 Apr 14 (Fri) 11:00 >> To:user >> Subject:Re: Slack Channel Request >> >> Could I also have an invite please? >> >> On 2017-03-28 08:28 (+0800), Davor Bonaci wrote: >> > Invite sent. >> > >> > On Sat, Mar 25, 2017 at 2:48 AM, Prabeesh K. > > wrote: >> > >> > > Hi Jean, >> > > >> > > Thank you for your reply. I am eagerly waiting for the other options. >> > > >> > > Regards, >> > > Prabeesh K. >> > > >> > > On 25 March 2017 at 10:08, Jean-Baptiste Onofré > > wrote: >> > > >> > >> Unfortunately we reached the max number of people on Slack (90). >> > >> >> > >> Let me see what we can do. >> > >> >> > >> Regards >> > >> JB >> > >> >> > >> >> > >> On 03/24/2017 09:49 PM, Prabeesh K. wrote: >> > >> >> > >>> Hi, >> > >>> >> > >>> Can someone please add me to the Apache Beam slack channel? >> > >>> >> > >>> Regards, >> > >>> >> > >>> Prabeesh K. >> > >>> >> > >>> >> > >> -- >> > >> Jean-Baptiste Onofré >> > >> jbono...@apache.org >> > >> http://blog.nanthrax.net >> > >> Talend - http://www.talend.com >> > >> >> > > >> > > >> > >> >> >> > > > -- > > Mingmin >
Re: Slack Channel Request
@James, @Jingsong, @Tom, invite sent. On Thu, Apr 13, 2017 at 8:34 PM, Tom Pollard wrote: > If it's not inconvenient, I'd also like an invitation to the Slack channel. > > Tom > > > On Apr 13, 2017, at 11:31 PM, JingsongLee wrote: > > Please add me too. > > Best, > > JingsongLee > > > -- > From:James > Time:2017 Apr 14 (Fri) 11:00 > To:user > Subject:Re: Slack Channel Request > > Could I also have an invite please? > > On 2017-03-28 08:28 (+0800), Davor Bonaci wrote: > > Invite sent. > > > > On Sat, Mar 25, 2017 at 2:48 AM, Prabeesh K. > wrote: > > > > > Hi Jean, > > > > > > Thank you for your reply. I am eagerly waiting for the other options. > > > > > > Regards, > > > Prabeesh K. > > > > > > On 25 March 2017 at 10:08, Jean-Baptiste Onofré > wrote: > > > > > >> Unfortunately we reached the max number of people on Slack (90). > > >> > > >> Let me see what we can do. > > >> > > >> Regards > > >> JB > > >> > > >> > > >> On 03/24/2017 09:49 PM, Prabeesh K. wrote: > > >> > > >>> Hi, > > >>> > > >>> Can someone please add me to the Apache Beam slack channel? > > >>> > > >>> Regards, > > >>> > > >>> Prabeesh K. > > >>> > > >>> > > >> -- > > >> Jean-Baptiste Onofré > > >> jbono...@apache.org > > >> http://blog.nanthrax.net > > >> Talend - http://www.talend.com > > >> > > > > > > > > > > > -- Mingmin
Re: Slack Channel Request
If it's not inconvenient, I'd also like an invitation to the Slack channel. Tom > On Apr 13, 2017, at 11:31 PM, JingsongLee wrote: > > Please add me too. > > Best, > > JingsongLee > > > -- > From:James > Time:2017 Apr 14 (Fri) 11:00 > To:user > Subject:Re: Slack Channel Request > > Could I also have an invite please? > > On 2017-03-28 08:28 (+0800), Davor Bonaci wrote: > > Invite sent. > > > > On Sat, Mar 25, 2017 at 2:48 AM, Prabeesh K. wrote: > > > > > Hi Jean, > > > > > > Thank you for your reply. I am eagerly waiting for the other options. > > > > > > Regards, > > > Prabeesh K. > > > > > > On 25 March 2017 at 10:08, Jean-Baptiste Onofré > > > wrote: > > > > > >> Unfortunately we reached the max number of people on Slack (90). > > >> > > >> Let me see what we can do. > > >> > > >> Regards > > >> JB > > >> > > >> > > >> On 03/24/2017 09:49 PM, Prabeesh K. wrote: > > >> > > >>> Hi, > > >>> > > >>> Can someone please add me to the Apache Beam slack channel? > > >>> > > >>> Regards, > > >>> > > >>> Prabeesh K. > > >>> > > >>> > > >> -- > > >> Jean-Baptiste Onofré > > >> jbono...@apache.org > > >> http://blog.nanthrax.net > > >> Talend - http://www.talend.com > > >> > > > > > > > > smime.p7s Description: S/MIME cryptographic signature
Re: Slack Channel Request
Please add me too. Best, JingsongLee --From:James Time:2017 Apr 14 (Fri) 11:00To:user Subject:Re: Slack Channel Request Could I also have an invite please? On 2017-03-28 08:28 (+0800), Davor Bonaci wrote: > Invite sent. > > On Sat, Mar 25, 2017 at 2:48 AM, Prabeesh K. wrote: > > > Hi Jean, > > > > Thank you for your reply. I am eagerly waiting for the other options. > > > > Regards, > > Prabeesh K. > > > > On 25 March 2017 at 10:08, Jean-Baptiste Onofré wrote: > > > >> Unfortunately we reached the max number of people on Slack (90). > >> > >> Let me see what we can do. > >> > >> Regards > >> JB > >> > >> > >> On 03/24/2017 09:49 PM, Prabeesh K. wrote: > >> > >>> Hi, > >>> > >>> Can someone please add me to the Apache Beam slack channel? > >>> > >>> Regards, > >>> > >>> Prabeesh K. > >>> > >>> > >> -- > >> Jean-Baptiste Onofré > >> jbono...@apache.org > >> http://blog.nanthrax.net > >> Talend - http://www.talend.com > >> > > > > >
Re: Slack Channel Request
Could I also have an invite please? On 2017-03-28 08:28 (+0800), Davor Bonaci wrote: > Invite sent. > > On Sat, Mar 25, 2017 at 2:48 AM, Prabeesh K. wrote: > > > Hi Jean, > > > > Thank you for your reply. I am eagerly waiting for the other options. > > > > Regards, > > Prabeesh K. > > > > On 25 March 2017 at 10:08, Jean-Baptiste Onofré wrote: > > > >> Unfortunately we reached the max number of people on Slack (90). > >> > >> Let me see what we can do. > >> > >> Regards > >> JB > >> > >> > >> On 03/24/2017 09:49 PM, Prabeesh K. wrote: > >> > >>> Hi, > >>> > >>> Can someone please add me to the Apache Beam slack channel? > >>> > >>> Regards, > >>> > >>> Prabeesh K. > >>> > >>> > >> -- > >> Jean-Baptiste Onofré > >> jbono...@apache.org > >> http://blog.nanthrax.net > >> Talend - http://www.talend.com > >> > > > > >
Re: beam + scala + streamline
Hi Georg -- Great to see you are evaluating Beam for your scenario. > > someone told me that e.g. the flink runner for beam seems to be slower >> than a >> > native flink job. Is this true? Did you observe such characteristics >> for several >> > runners? >> > This should not be true in a general sense -- the performance should be ~equivalent. The Flink runner in Beam constructs a "native" Flink pipeline; the overhead of invoking user-defined functions is often to set a few fields and invoke a function, which is negligible. The actual performance of a pipeline tend to depend on other factors -- stragglers, how fast the system can adopt to changing load, etc. (If there's a gap somewhere, it is likely a bug -- and we'd like to know about it and fix it.) > in case I want to use some low level functionality (specific to a runner) >> like >> > ML, graph processing or sql-tables api, is it possible to just drop >> from the >> > beam API one level deeper to the actual runner and sort of mesh beam >> with runner >> > native code to integrate these features? > > The Beam API, in a general sense, doesn't provide such hooks, as that would break portability. I wouldn't advise this, but technically, it wouldn't be hard -- you'd create a PTransform in Beam, and modify the runner to replace it with their own specific implementation. Instead, I'd suggest using Beam's abstractions and, in the case of a missing pattern or a feature, to work with us to augment the Beam model accordingly. Hope this helps -- and that you find Beam fitting for your case. Please let us know if we can assist any further -- thanks! Davor
Re: Unhelpful ExceptionInChainedStubException errors with Flink runner
Sorry for the slow response. I've had this marked in my inbox but due to the preparations for Flink Forward I didn't have time to look at it yet. I'll definitely still do that. On Thu, Apr 13, 2017, at 05:15, Davor Bonaci wrote: > Aljoscha, any ideas perhaps? > > On Wed, Apr 5, 2017 at 12:52 PM, peay wrote: >> Hello, >> >> I've been having some trouble with debugging exceptions in user code >> when using the Flink runner. Here's an example from a >> window/DoFn/GroupByKey pipeline. >> >> ERROR o.a.f.runtime.operators.BatchTask - Error in task code: CHAIN >> MapPartition (MapPartition at ParDo(MyDoFn)) -> FlatMap >> (Transform/Windowing/Window.Assign.out) -> Map (Key Extractor) -> >> GroupCombine (GroupCombine at GroupCombine: GroupByKey) -> Map (Key >> Extractor) (1/8) >> org.apache.beam.sdk.util.UserCodeException: org.apache.flink.runtime- >> .operators.chaining.ExceptionInChainedStubException >> at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeE- >> xception.java:36) ~[beam-sdks-java-core-0.6.0.jar:0.6.0] >> at org.org.my.pipelines.MyDoFn$auxiliary$s09rfuPj.invokeProc- >> essElement(Unknown Source) ~[na:na] >> at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProce- >> ssElement(SimpleDoFnRunner.java:198) ~[beam-runners-core-java- >> 0.6.0.jar:0.6.0] >> at org.apache.beam.runners.core.SimpleDoFnRunner.processElem- >> ent(SimpleDoFnRunner.java:156) ~[beam-runners-core-java- >> 0.6.0.jar:0.6.0] >> at org.apache.beam.runners.flink.translation.functions.Flink- >> DoFnFunction.mapPartition(FlinkDoFnFunction.java:109) >> ~[beam-runners-flink_2.10- >> 0.6.0.jar:0.6.0] >> at org.apache.flink.runtime.operators.MapPartitionDriver.run- >> (MapPartitionDriver.java:103) ~[flink-runtime_2.10- >> 1.2.0.jar:1.2.0] >> at org.apache.flink.runtime.operators.BatchTask.run(BatchTas- >> k.java:490) [flink-runtime_2.10-1.2.0.jar:1.2.0] >> at org.apache.flink.runtime.operators.BatchTask.invoke(Batch- >> Task.java:355) [flink-runtime_2.10-1.2.0.jar:1.2.0] >> at >> org.apache.flink.runtime.taskmanager.Task.run(Task.java:655) >> [flink-runtime_2.10-1.2.0.jar:1.2.0] >> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] >> org.apache.flink.runtime.operators.chaining.ExceptionInChainedStubEx- >> ception: null >> at org.apache.flink.runtime.operators.chaining.ChainedFlatMa- >> pDriver.collect(ChainedFlatMapDriver.java:82) ~[flink-runtime_2.10- >> 1.2.0.jar:1.2.0] >> at org.apache.flink.runtime.operators.util.metrics.CountingC- >> ollector.collect(CountingCollector.java:35) ~[flink-runtime_2.10- >> 1.2.0.jar:1.2.0] >> at org.apache.beam.runners.flink.translation.functions.Flink- >> DoFnFunction$DoFnOutputManager.output(FlinkDoFnFunction.java- >> :138) ~[beam-runners-flink_2.10-0.6.0.jar:0.6.0] >> at org.apache.beam.runners.core.SimpleDoFnRunner$DoFnContext- >> .outputWindowedValue(SimpleDoFnRunner.java:351) >> ~[beam-runners-core-java- >> 0.6.0.jar:0.6.0] >> at org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcess- >> Context.output(SimpleDoFnRunner.java:545) ~[beam-runners-core-java- >> 0.6.0.jar:0.6.0] >> at org.my.pipelines.MyDoFn.processElement(MyDoFn.java:49) >> ~[pipelines- >> 0.1.jar:na] >> >> The top stacktrace references some kind of anonymous >> `invokeProcessElement(Unknown Source)` which is not really >> informative. The bottom stacktrace references my call to >> `context.output()`, which is even more confusing. I've gone through >> fixing a couple issue by manually try/catching and logging directly >> from within `processElement`, but this is far from ideal. Any advice >> on how to interpret those and possibly set things up in order to get >> more helpful error messages would be really helpful. >> >> Running Beam 0.6, Flink 1.2. >> >> Thanks! >> >>