Re: Strata Conference this March 6-8

2018-02-14 Thread Ron Gonzalez
 Folks,  Just wanted to confirm if we've finalized on meeting for the Strata 
Conference this March 6-8? I remember something about a calendar getting put 
somewhere but I can't seem to find that in beam.apache.org...
Thanks,Ron
On Monday, January 22, 2018, 2:13:01 PM CST, Griselda Cuevas 
<g...@google.com> wrote:  
 
 Hi Everyone, 
+1 to the BoF. I'd suggest meeting at 3:30 p.m. on 3/7/2018 at the Philz Coffee 
Holden suggested. It takes about 10min to walk from the convention center 
there. 
re:Organizing a Meetup near the dates of Strata SJ. I can organize something 
for Wednesday night @ Google's campus. Would anyone have a use case or talk 
they'd like to share with the community?
re:Strata London -- @Matthias & Victor, happy to help on that as well. 
Cheers, G



On 19 January 2018 at 11:00, Tyler Akidau <taki...@google.com> wrote:

I'll be talking about streaming SQL at Strata SJC. It's related to the SQL work 
happening in Beam that Mingmin, James, and Anton are working on, but the talk 
itself is relatively conceptual, hence no Beam in the title. Is the BoF thing 
happening? If so, is Wed 3:20pm the confirmed time? I'll be flying back from 
QCon London that week, so will try to book flights that get me back in time for 
the BoF if I can.

-Tyler

On Thu, Jan 18, 2018 at 9:39 AM Jean-Baptiste Onofré <j...@nanthrax.net> wrote:

I think Matthias has already some plan for London meetup later this year (it's
what he said to me).

Stay tuned !

Regards
JB

On 01/18/2018 06:29 PM, Ismaël Mejía wrote:
> My excuses I somehow misread the dates and thought you referred to the
> London conference, but well in the end this becomes two good ideas :)
>
> - A meetup in London for the week of May 21
> - A meetup in San Jose if someone can organize it for the March dates.
>
>
> On Thu, Jan 18, 2018 at 12:00 AM, Holden Karau <hol...@pigscanfly.ca> wrote:
>> So doing a streaming BoF join in would probably require meeting somewhere
>> other than a coffee shop so as not to be jerks in the coffee shop.
>>
>> On Wed, Jan 17, 2018 at 2:53 PM, Matthias Baetens
>> <matthias.baet...@datatonic.co m> wrote:
>>>
>>> Sure, I'd be very happy to organise something. This is about Strata San
>>> Jose though right? Maybe we can organise a remote session in which we can
>>> join (depending on when you would organise the BoF) or have a channel set-up
>>> if the talks would be broadcasted?
>>>
>>> Also: will there be any Beam talks on Strata London or is this not known
>>> yet? Keen to get involved and set things up around that date as well.
>>>
>>> On Wed, Jan 17, 2018 at 8:37 AM, Jean-Baptiste Onofré <j...@nanthrax.net>
>>> wrote:
>>>>
>>>> That's a great idea ! I'm sure that Matthias (organizer of the Beam
>>>> London Meetup) can help us to plan something.
>>>>
>>>> Regards
>>>> JB
>>>>
>>>>
>>>> On 01/17/2018 08:57 AM, Ismaël Mejía wrote:
>>>>>
>>>>> Maybe a good idea to try to organize a Beam meetup in london in the
>>>>> same dates in case some of the people around can jump in and talk too.
>>>>>
>>>>> On Wed, Jan 17, 2018 at 2:51 AM, Ron Gonzalez <zlgonza...@yahoo.com>
>>>>> wrote:
>>>>>>
>>>>>> Works for me...
>>>>>>
>>>>>> On Tuesday, January 16, 2018, 5:45:33 PM PST, Holden Karau
>>>>>> <hol...@pigscanfly.ca> wrote:
>>>>>>
>>>>>>
>>>>>> How would folks feel about during the afternoon break (3:20-4:20) on
>>>>>> the
>>>>>> Wednesday (same day as Eugene's talk)? We could do the Philz which is a
>>>>>> bit
>>>>>> of a walk but gets us away from the big crowd and also lets folks not
>>>>>> attending the conference but in the area join us.
>>>>>>
>>>>>> On Tue, Jan 16, 2018 at 5:29 PM, Ron Gonzalez <zlgonza...@yahoo.com>
>>>>>> wrote:
>>>>>>
>>>>>> Cool, let me know if you guys finally schedule it. I will definitely
>>>>>> try to
>>>>>> make it to Eugene's talk but having an informal BoF in the area would
>>>>>> be
>>>>>> nice...
>>>>>>
>>>>>> Thanks,
>>>>>> Ron
>>>>>>
>>>>>> On Tuesday, January 16, 2018, 5:06:53 PM PST, Boris Lublinsky
>>>>>> <boris.lublin...@lightbend.com > wrote:
>>>>>>
>>>&g

Re: Some interesting use case

2018-01-16 Thread Ron Gonzalez
 Yes you're right. I believe this is the use case that I'm after. So if I 
understand correctly, transforms that do aggregations just assume that the 
batch of data being aggregated is passed as part of a tensor column. Is it 
possible to hook up a lookup call to another Tensorflow Serving servable for a 
join in batch mode?
Will a saved model when loaded into a tensorflow serving model actually have 
the definitions of the metadata when retrieved using the tensorflow serving 
metadata api?
Thanks,Ron
On Tuesday, January 16, 2018, 6:16:01 PM PST, Charles Chen 
<c...@google.com> wrote:  
 
 This sounds similar to the use case for tf.Transform, a library that depends 
on Beam: https://github.com/tensorflow/transform
On Tue, Jan 16, 2018 at 5:51 PM Ron Gonzalez <zlgonza...@yahoo.com> wrote:

Hi,  I was wondering if anyone has encountered or used Beam in the following 
manner:   1. During machine learning training, use Beam to create the event 
table. The flow may consist of some joins, aggregations, row-based 
transformations, etc...  2. Once the model is created, deploy the model to some 
scoring service via PMML (or some other scoring service).  3. Enable the SAME 
transformations used in #1 by using a separate engine but thereby guaranteeing 
that it will transform the data identically as the engine used in #1.
  I think this is a pretty interesting use case where Beam is used to guarantee 
portability across engines and deployment (batch to true streaming, not 
micro-batch). What's not clear to me is with respect to how batch joins would 
translate during one-by-one scoring (probably lookups) or how aggregations 
given that some kind of history would need to be stored (and how much is kept 
is configurable too).
  Thoughts?
Thanks,Ron
  

Re: Strata Conference this March 6-8

2018-01-16 Thread Ron Gonzalez
 Works for me...
On Tuesday, January 16, 2018, 5:45:33 PM PST, Holden Karau 
<hol...@pigscanfly.ca> wrote:  
 
 How would folks feel about during the afternoon break (3:20-4:20) on the 
Wednesday (same day as Eugene's talk)? We could do the Philz which is a bit of 
a walk but gets us away from the big crowd and also lets folks not attending 
the conference but in the area join us.
On Tue, Jan 16, 2018 at 5:29 PM, Ron Gonzalez <zlgonza...@yahoo.com> wrote:

 Cool, let me know if you guys finally schedule it. I will definitely try to 
make it to Eugene's talk but having an informal BoF in the area would be nice...
Thanks,Ron
On Tuesday, January 16, 2018, 5:06:53 PM PST, Boris Lublinsky 
<boris.lublin...@lightbend.com > wrote:  
 
 All for it
Boris Lublinsky
FDP Architect
boris.lublin...@lightbend.com
https://www.lightbend.com/

On Jan 16, 2018, at 7:01 PM, Ted Yu <yuzhih...@gmail.com> wrote:
+1 to BoF
On Tue, Jan 16, 2018 at 5:00 PM, Dmitry Demeshchuk <dmi...@postmates.com> wrote:

Probably won't be attending the conference, but totally down for a BoF.
On Tue, Jan 16, 2018 at 4:58 PM, Holden Karau <hol...@pigscanfly.ca> wrote:

Do interested folks have any timing constraints around a BoF?
On Tue, Jan 16, 2018 at 4:30 PM, Jesse Anderson <je...@bigdatainstitute.io> 
wrote:

+1 to BoF. I don't know if any Beam talks will be on the schedule.

> We could do an informal BoF at the Philz nearby or similar?




-- 
Twitter: https://twitter.com/h oldenkarau




-- 
Best regards,Dmitry Demeshchuk.



  



-- 
Twitter: https://twitter.com/holdenkarau
  

Some interesting use case

2018-01-16 Thread Ron Gonzalez
Hi,  I was wondering if anyone has encountered or used Beam in the following 
manner:   1. During machine learning training, use Beam to create the event 
table. The flow may consist of some joins, aggregations, row-based 
transformations, etc...  2. Once the model is created, deploy the model to some 
scoring service via PMML (or some other scoring service).  3. Enable the SAME 
transformations used in #1 by using a separate engine but thereby guaranteeing 
that it will transform the data identically as the engine used in #1.
  I think this is a pretty interesting use case where Beam is used to guarantee 
portability across engines and deployment (batch to true streaming, not 
micro-batch). What's not clear to me is with respect to how batch joins would 
translate during one-by-one scoring (probably lookups) or how aggregations 
given that some kind of history would need to be stored (and how much is kept 
is configurable too).
  Thoughts?
Thanks,Ron

Re: Strata Conference this March 6-8

2018-01-16 Thread Ron Gonzalez
 Cool, let me know if you guys finally schedule it. I will definitely try to 
make it to Eugene's talk but having an informal BoF in the area would be nice...
Thanks,Ron
On Tuesday, January 16, 2018, 5:06:53 PM PST, Boris Lublinsky 
 wrote:  
 
 All for it
Boris Lublinsky
FDP Architect
boris.lublin...@lightbend.com
https://www.lightbend.com/

On Jan 16, 2018, at 7:01 PM, Ted Yu  wrote:
+1 to BoF
On Tue, Jan 16, 2018 at 5:00 PM, Dmitry Demeshchuk  wrote:

Probably won't be attending the conference, but totally down for a BoF.
On Tue, Jan 16, 2018 at 4:58 PM, Holden Karau  wrote:

Do interested folks have any timing constraints around a BoF?
On Tue, Jan 16, 2018 at 4:30 PM, Jesse Anderson  
wrote:

+1 to BoF. I don't know if any Beam talks will be on the schedule.

> We could do an informal BoF at the Philz nearby or similar?




-- 
Twitter: https://twitter.com/h oldenkarau




-- 
Best regards,Dmitry Demeshchuk.



  

Strata Conference this March 6-8

2018-01-16 Thread Ron Gonzalez
Hi,  Will there be some talks or representation of Apache Beam at the coming 
Strata Conference this March 6-8?  Would be great to hear someone talk about 
how Beam's been used at their company as their core data integration platform.
Thanks,Ron  

Spark runner maven shade plugin

2018-01-14 Thread Ron Gonzalez
Hi,  I added the maven build plugin in the Spark runner page:

  org.apache.maven.plugins
  maven-shade-plugin
  
false

  
*:*

  META-INF/*.SF
  META-INF/*.DSA
  META-INF/*.RSA

  

  
  

  package
  
shade
  
  
true
shaded

  

  

  

but I'm getting the following error:

[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-shade-plugin:2.4.3:shade (default) on project 
transform: Unable to parse configuration of mojo 
org.apache.maven.plugins:maven-shade-plugin:2.4.3:shade for parameter resource: 
Cannot find 'resource' in class 
org.apache.maven.plugins.shade.resource.ServicesResourceTransformer -> [Help 1]
Please advise...
Thanks,Ron


Question on basic version changes

2018-01-14 Thread Ron Gonzalez
Hi,  I'd like to contribute a way to track metadata lineage and impact analysis 
in beam. Whom can I speak with to discuss details?
Thanks,Ron