Re: Slack Channel Request

2017-04-13 Thread Georg Heiler 
Please add me toi
Mingmin Xu  schrieb am Fr. 14. Apr. 2017 um 07:06:

> both sent.
>
> On Thu, Apr 13, 2017 at 10:00 PM, Anant Bhandarkar <
> anant.bhandar...@impactanalytics.co> wrote:
>
>> Would love to be part of beam group on slack.
>> Also please add anil.b...@impactanaytics.co
>> Thanks,
>> Anant
>>
>> On 14-Apr-2017 9:15 AM, "Mingmin Xu"  wrote:
>>
>>> @James, @Jingsong, @Tom, invite sent.
>>>
>>> On Thu, Apr 13, 2017 at 8:34 PM, Tom Pollard <
>>> tpoll...@flashpoint-intel.com> wrote:
>>>
 If it's not inconvenient, I'd also like an invitation to the Slack
 channel.

 Tom


 On Apr 13, 2017, at 11:31 PM, JingsongLee 
 wrote:

 Please add me too.

 Best,

 JingsongLee


 --
 From:James 
 Time:2017 Apr 14 (Fri) 11:00
 To:user 
 Subject:Re: Slack Channel Request

 Could I also have an invite please?

 On 2017-03-28 08:28 (+0800), Davor Bonaci  wrote:
 > Invite sent.
 >
 > On Sat, Mar 25, 2017 at 2:48 AM, Prabeesh K.  wrote:
 >
 > > Hi Jean,
 > >

 > > Thank you for your reply. I am eagerly waiting for the other options.
 > >
 > > Regards,
 > > Prabeesh K.
 > >
 > > On 25 March 2017 at 10:08, Jean-Baptiste Onofré  wrote:
 > >
 > >> Unfortunately we reached the max number of people on Slack (90).
 > >>
 > >> Let me see what we can do.
 > >>
 > >> Regards
 > >> JB
 > >>
 > >>
 > >> On 03/24/2017 09:49 PM, Prabeesh K. wrote:
 > >>
 > >>> Hi,
 > >>>
 > >>> Can someone please add me to the Apache Beam slack channel?
 > >>>
 > >>> Regards,
 > >>>
 > >>> Prabeesh K.
 > >>>
 > >>>
 > >> --
 > >> Jean-Baptiste Onofré
 > >> jbono...@apache.org
 > >> http://blog.nanthrax.net
 > >> Talend - http://www.talend.com
 > >>
 > >
 > >
 >



>>>
>>>
>>> --
>>> 
>>> Mingmin
>>>
>>
>
>
> --
> 
> Mingmin
>

Re: beam + scala + streamline

2017-04-12 Thread Georg Heiler 
Hi JB ,
Indeed I would like to get access to this lower level features as some are
still missing from beam.

So in a DoFn I would not be able to let's say access spark session and
convert to a spark native data frame.

How is / should such things currently be handled in beam?

Regards
Georg
Jean-Baptiste Onofré <j...@nanthrax.net> schrieb am Mi. 12. Apr. 2017 um
21:45:

> Hi Georg,
>
> Personally, I didn't have feedback about performance difference between
> flink
> native and beam/flink runner.
>
> For low level runner access, you mean you want to access Spark ML
> function, etc
> from a Beam pipeline ? Even if you can always use a DoFn, you won't have
> access
> to all internal objects/API. That's what you mean ?
>
> Regards
> JB
>
> On 04/12/2017 08:33 PM, Georg Heiler wrote:
> > HI JB,
> >
> > The overall look and feel of streamline (Kafka, storm, hbase and the
> simplicity
> > of the UI) were really compelling. As we are currently starting to build
> a
> > streaming platform we are looking forward to such a tight integration,
> but would
> > rather use flink or spark (maybe via beam) and not storm integrated into
> such a
> > platform.
> >
> > Regarding streamline performance:
> > someone told me that e.g. the flink runner for beam seems to be slower
> than a
> > native flink job. Is this true? Did you observe such characteristics for
> several
> > runners?
> >
> > Regarding streamline low level runner access:
> > in case I want to use some low level functionality (specific to a
> runner) like
> > ML, graph processing or sql-tables api, is it possible to just drop from
> the
> > beam API one level deeper to the actual runner and sort of mesh beam
> with runner
> > native code to integrate these features?
> >
> > Regards,
> > Georg
> >
> > Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> schrieb
> am Mi.,
> > 12. Apr. 2017 um 20:04 Uhr:
> >
> > Hi Georg,
> >
> > You can use Java API via Scala, or you can use the Scio Scala DSL
> (this DSL use
> > the Beam Java SDK).
> >
> > For Streamline, can you explain a bit ? Streamline contains
> different parts:
> > HBase, Kafka, the web frontend, ...
> >
> > Using provided IO, it should be possible to store the data and use
> streamline on
> > top of this data for analytics.
> >
> > Regards
> > JB
> >
> > On 04/12/2017 07:57 PM, Georg Heiler wrote:
> > > Hi,
> > > new to beam I wonder what API is recommended for using beam from
> scala.
> > > Would you recommend simply using the java API from scala or
> > > https://github.com/spotify/scio?
> > >
> > > Are there any plans to support beam in
> > https://github.com/hortonworks/streamline?
> > >
> > > regards,
> > > Georg
> >
> > --
> > Jean-Baptiste Onofré
> > jbono...@apache.org <mailto:jbono...@apache.org>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: beam + scala + streamline

2017-04-12 Thread Georg Heiler 
HI JB,

The overall look and feel of streamline (Kafka, storm, hbase and the
simplicity of the UI) were really compelling. As we are currently starting
to build a streaming platform we are looking forward to such a tight
integration, but would rather use flink or spark (maybe via beam) and not
storm integrated into such a platform.

Regarding streamline performance:
someone told me that e.g. the flink runner for beam seems to be slower than
a native flink job. Is this true? Did you observe such characteristics for
several runners?

Regarding streamline low level runner access:
in case I want to use some low level functionality (specific to a runner)
like ML, graph processing or sql-tables api, is it possible to just drop
from the beam API one level deeper to the actual runner and sort of mesh
beam with runner native code to integrate these features?

Regards,
Georg

Jean-Baptiste Onofré <j...@nanthrax.net> schrieb am Mi., 12. Apr. 2017 um
20:04 Uhr:

> Hi Georg,
>
> You can use Java API via Scala, or you can use the Scio Scala DSL (this
> DSL use
> the Beam Java SDK).
>
> For Streamline, can you explain a bit ? Streamline contains different
> parts:
> HBase, Kafka, the web frontend, ...
>
> Using provided IO, it should be possible to store the data and use
> streamline on
> top of this data for analytics.
>
> Regards
> JB
>
> On 04/12/2017 07:57 PM, Georg Heiler wrote:
> > Hi,
> > new to beam I wonder what API is recommended for using beam from scala.
> > Would you recommend simply using the java API from scala or
> > https://github.com/spotify/scio?
> >
> > Are there any plans to support beam in
> https://github.com/hortonworks/streamline?
> >
> > regards,
> > Georg
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

beam + scala + streamline

2017-04-12 Thread Georg Heiler 
Hi,
new to beam I wonder what API is recommended for using beam from scala.
Would you recommend simply using the java API from scala or
https://github.com/spotify/scio?

Are there any plans to support beam in
https://github.com/hortonworks/streamline?

regards,
Georg