Re: Strata Conference this March 6-8
Folks, Just wanted to confirm if we've finalized on meeting for the Strata Conference this March 6-8? I remember something about a calendar getting put somewhere but I can't seem to find that in beam.apache.org... Thanks,Ron On Monday, January 22, 2018, 2:13:01 PM CST, Griselda Cuevas <g...@google.com> wrote: Hi Everyone, +1 to the BoF. I'd suggest meeting at 3:30 p.m. on 3/7/2018 at the Philz Coffee Holden suggested. It takes about 10min to walk from the convention center there. re:Organizing a Meetup near the dates of Strata SJ. I can organize something for Wednesday night @ Google's campus. Would anyone have a use case or talk they'd like to share with the community? re:Strata London -- @Matthias & Victor, happy to help on that as well. Cheers, G On 19 January 2018 at 11:00, Tyler Akidau <taki...@google.com> wrote: I'll be talking about streaming SQL at Strata SJC. It's related to the SQL work happening in Beam that Mingmin, James, and Anton are working on, but the talk itself is relatively conceptual, hence no Beam in the title. Is the BoF thing happening? If so, is Wed 3:20pm the confirmed time? I'll be flying back from QCon London that week, so will try to book flights that get me back in time for the BoF if I can. -Tyler On Thu, Jan 18, 2018 at 9:39 AM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: I think Matthias has already some plan for London meetup later this year (it's what he said to me). Stay tuned ! Regards JB On 01/18/2018 06:29 PM, Ismaël Mejía wrote: > My excuses I somehow misread the dates and thought you referred to the > London conference, but well in the end this becomes two good ideas :) > > - A meetup in London for the week of May 21 > - A meetup in San Jose if someone can organize it for the March dates. > > > On Thu, Jan 18, 2018 at 12:00 AM, Holden Karau <hol...@pigscanfly.ca> wrote: >> So doing a streaming BoF join in would probably require meeting somewhere >> other than a coffee shop so as not to be jerks in the coffee shop. >> >> On Wed, Jan 17, 2018 at 2:53 PM, Matthias Baetens >> <matthias.baet...@datatonic.co m> wrote: >>> >>> Sure, I'd be very happy to organise something. This is about Strata San >>> Jose though right? Maybe we can organise a remote session in which we can >>> join (depending on when you would organise the BoF) or have a channel set-up >>> if the talks would be broadcasted? >>> >>> Also: will there be any Beam talks on Strata London or is this not known >>> yet? Keen to get involved and set things up around that date as well. >>> >>> On Wed, Jan 17, 2018 at 8:37 AM, Jean-Baptiste Onofré <j...@nanthrax.net> >>> wrote: >>>> >>>> That's a great idea ! I'm sure that Matthias (organizer of the Beam >>>> London Meetup) can help us to plan something. >>>> >>>> Regards >>>> JB >>>> >>>> >>>> On 01/17/2018 08:57 AM, Ismaël Mejía wrote: >>>>> >>>>> Maybe a good idea to try to organize a Beam meetup in london in the >>>>> same dates in case some of the people around can jump in and talk too. >>>>> >>>>> On Wed, Jan 17, 2018 at 2:51 AM, Ron Gonzalez <zlgonza...@yahoo.com> >>>>> wrote: >>>>>> >>>>>> Works for me... >>>>>> >>>>>> On Tuesday, January 16, 2018, 5:45:33 PM PST, Holden Karau >>>>>> <hol...@pigscanfly.ca> wrote: >>>>>> >>>>>> >>>>>> How would folks feel about during the afternoon break (3:20-4:20) on >>>>>> the >>>>>> Wednesday (same day as Eugene's talk)? We could do the Philz which is a >>>>>> bit >>>>>> of a walk but gets us away from the big crowd and also lets folks not >>>>>> attending the conference but in the area join us. >>>>>> >>>>>> On Tue, Jan 16, 2018 at 5:29 PM, Ron Gonzalez <zlgonza...@yahoo.com> >>>>>> wrote: >>>>>> >>>>>> Cool, let me know if you guys finally schedule it. I will definitely >>>>>> try to >>>>>> make it to Eugene's talk but having an informal BoF in the area would >>>>>> be >>>>>> nice... >>>>>> >>>>>> Thanks, >>>>>> Ron >>>>>> >>>>>> On Tuesday, January 16, 2018, 5:06:53 PM PST, Boris Lublinsky >>>>>> <boris.lublin...@lightbend.com > wrote: >>>>>> >>>&g
Re: Some interesting use case
Yes you're right. I believe this is the use case that I'm after. So if I understand correctly, transforms that do aggregations just assume that the batch of data being aggregated is passed as part of a tensor column. Is it possible to hook up a lookup call to another Tensorflow Serving servable for a join in batch mode? Will a saved model when loaded into a tensorflow serving model actually have the definitions of the metadata when retrieved using the tensorflow serving metadata api? Thanks,Ron On Tuesday, January 16, 2018, 6:16:01 PM PST, Charles Chen <c...@google.com> wrote: This sounds similar to the use case for tf.Transform, a library that depends on Beam: https://github.com/tensorflow/transform On Tue, Jan 16, 2018 at 5:51 PM Ron Gonzalez <zlgonza...@yahoo.com> wrote: Hi, I was wondering if anyone has encountered or used Beam in the following manner: 1. During machine learning training, use Beam to create the event table. The flow may consist of some joins, aggregations, row-based transformations, etc... 2. Once the model is created, deploy the model to some scoring service via PMML (or some other scoring service). 3. Enable the SAME transformations used in #1 by using a separate engine but thereby guaranteeing that it will transform the data identically as the engine used in #1. I think this is a pretty interesting use case where Beam is used to guarantee portability across engines and deployment (batch to true streaming, not micro-batch). What's not clear to me is with respect to how batch joins would translate during one-by-one scoring (probably lookups) or how aggregations given that some kind of history would need to be stored (and how much is kept is configurable too). Thoughts? Thanks,Ron
Re: Strata Conference this March 6-8
Works for me... On Tuesday, January 16, 2018, 5:45:33 PM PST, Holden Karau <hol...@pigscanfly.ca> wrote: How would folks feel about during the afternoon break (3:20-4:20) on the Wednesday (same day as Eugene's talk)? We could do the Philz which is a bit of a walk but gets us away from the big crowd and also lets folks not attending the conference but in the area join us. On Tue, Jan 16, 2018 at 5:29 PM, Ron Gonzalez <zlgonza...@yahoo.com> wrote: Cool, let me know if you guys finally schedule it. I will definitely try to make it to Eugene's talk but having an informal BoF in the area would be nice... Thanks,Ron On Tuesday, January 16, 2018, 5:06:53 PM PST, Boris Lublinsky <boris.lublin...@lightbend.com > wrote: All for it Boris Lublinsky FDP Architect boris.lublin...@lightbend.com https://www.lightbend.com/ On Jan 16, 2018, at 7:01 PM, Ted Yu <yuzhih...@gmail.com> wrote: +1 to BoF On Tue, Jan 16, 2018 at 5:00 PM, Dmitry Demeshchuk <dmi...@postmates.com> wrote: Probably won't be attending the conference, but totally down for a BoF. On Tue, Jan 16, 2018 at 4:58 PM, Holden Karau <hol...@pigscanfly.ca> wrote: Do interested folks have any timing constraints around a BoF? On Tue, Jan 16, 2018 at 4:30 PM, Jesse Anderson <je...@bigdatainstitute.io> wrote: +1 to BoF. I don't know if any Beam talks will be on the schedule. > We could do an informal BoF at the Philz nearby or similar? -- Twitter: https://twitter.com/h oldenkarau -- Best regards,Dmitry Demeshchuk. -- Twitter: https://twitter.com/holdenkarau
Some interesting use case
Hi, I was wondering if anyone has encountered or used Beam in the following manner: 1. During machine learning training, use Beam to create the event table. The flow may consist of some joins, aggregations, row-based transformations, etc... 2. Once the model is created, deploy the model to some scoring service via PMML (or some other scoring service). 3. Enable the SAME transformations used in #1 by using a separate engine but thereby guaranteeing that it will transform the data identically as the engine used in #1. I think this is a pretty interesting use case where Beam is used to guarantee portability across engines and deployment (batch to true streaming, not micro-batch). What's not clear to me is with respect to how batch joins would translate during one-by-one scoring (probably lookups) or how aggregations given that some kind of history would need to be stored (and how much is kept is configurable too). Thoughts? Thanks,Ron
Re: Strata Conference this March 6-8
Cool, let me know if you guys finally schedule it. I will definitely try to make it to Eugene's talk but having an informal BoF in the area would be nice... Thanks,Ron On Tuesday, January 16, 2018, 5:06:53 PM PST, Boris Lublinskywrote: All for it Boris Lublinsky FDP Architect boris.lublin...@lightbend.com https://www.lightbend.com/ On Jan 16, 2018, at 7:01 PM, Ted Yu wrote: +1 to BoF On Tue, Jan 16, 2018 at 5:00 PM, Dmitry Demeshchuk wrote: Probably won't be attending the conference, but totally down for a BoF. On Tue, Jan 16, 2018 at 4:58 PM, Holden Karau wrote: Do interested folks have any timing constraints around a BoF? On Tue, Jan 16, 2018 at 4:30 PM, Jesse Anderson wrote: +1 to BoF. I don't know if any Beam talks will be on the schedule. > We could do an informal BoF at the Philz nearby or similar? -- Twitter: https://twitter.com/h oldenkarau -- Best regards,Dmitry Demeshchuk.
Strata Conference this March 6-8
Hi, Will there be some talks or representation of Apache Beam at the coming Strata Conference this March 6-8? Would be great to hear someone talk about how Beam's been used at their company as their core data integration platform. Thanks,Ron
Spark runner maven shade plugin
Hi, I added the maven build plugin in the Spark runner page: org.apache.maven.plugins maven-shade-plugin false *:* META-INF/*.SF META-INF/*.DSA META-INF/*.RSA package shade true shaded but I'm getting the following error: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:2.4.3:shade (default) on project transform: Unable to parse configuration of mojo org.apache.maven.plugins:maven-shade-plugin:2.4.3:shade for parameter resource: Cannot find 'resource' in class org.apache.maven.plugins.shade.resource.ServicesResourceTransformer -> [Help 1] Please advise... Thanks,Ron
Question on basic version changes
Hi, I'd like to contribute a way to track metadata lineage and impact analysis in beam. Whom can I speak with to discuss details? Thanks,Ron