Who's hiring, December 2016

2016-12-16 Thread Kostas Tzoumas
Hi folks,

As promised, here is the first thread for Flink-related job positions. If
your organization is hiring people on Flink-related positions do reply to
this thread with a link for applications.

data Artisans is hiring on multiple technical positions. Help us build
Flink, and help our customers be successful in their Flink projects:

- Senior distributed systems engineer:
https://data-artisans.workable.com/jobs/396284

- Software engineer (Java/Scala and/or Python):
https://data-artisans.workable.com/jobs/396286

- QA/DevOps engineer: https://data-artisans.workable.com/jobs/396288

- UI/UX engineer: https://data-artisans.workable.com/jobs/396287

- Senior data engineer (EU and USA):
https://data-artisans.workable.com/jobs/325667

Best regards,
Kostas

PS: As mentioned in the original DISCUSS thread, I am cc'ing the dev and
user lists in the first few emails to remind folks to subscribe to the new
commun...@flink.apache.org mailing list

Instructions to subscribe are here:
http://flink.apache.org/community.html#mailing-lists


[ANNOUNCE] New Flink community mailing list

2016-12-14 Thread Kostas Tzoumas
Hi everyone,

We have created a new Flink mailing lists, commun...@flink.apache.org where
we can post everything related to the broader Flink community including job
offers, upcoming meetups and conferences, exciting reads, and everything
else that is deemed worthy for the greater Flink community. We will be
running a monthly "Who's hiring" thread there for job positions on Flink.

The scope of this list is restricted to Flink-related content, i.e., please
do not post content or job offers on Big Data in general.

You can subscribe to this list as usual by sending an email to

community-subscr...@flink.apache.org

We will be adding this list to the Flink website soon.

Best,
Kostas


Re: [DISCUSS] "Who's hiring on Flink" monthly thread in the mailing lists?

2016-12-09 Thread Kostas Tzoumas
I appreciate the concern Kanstantsin!

We do have a news@ mailing list, but it has been under-utilized so far.
Perhaps revamping that one would do it?

My only concern is that subscribing to a new mailing list is an overhead.
As a temp solution, we could cc the dev and user list in the first few
(say, 3) threads and encourage folks in these threads to sign up for the
news@ list.

On Thu, Dec 8, 2016 at 10:07 AM, Robert Metzger <rmetz...@apache.org> wrote:

> Thank you for speaking up Kanstantsin. I really don't want to downgrade
> the experience on the user@ list.
>
> I wonder if jobs@flink would be a too narrowly-scoped mailing list.
> Maybe we could also start a community@flink (alternatively also general@)
> mailing list for everything relating to the broader Flink community,
> including job offers, meetups, conferences and everything else that is
> important for the community to grow.
>
> On Thu, Dec 8, 2016 at 3:10 AM, Radu Tudoran <radu.tudo...@huawei.com>
> wrote:
>
>> Hi,
>>
>>
>>
>> I think the idea of having such a monthly thread is very good and it
>> might even help to further attract new people in the community.
>>
>> In the same time I do not think that 1 extra mail per month is necessary
>> a spam J
>>
>> In the same time – we can also consider a jobs@flink mailing list
>>
>>
>>
>>
>>
>> Dr. Radu Tudoran
>>
>> Senior Research Engineer - Big Data Expert
>>
>> IT R Division
>>
>>
>>
>> [image: cid:image007.jpg@01CD52EB.AD060EE0]
>>
>> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>>
>> European Research Center
>>
>> Riesstrasse 25, 80992 München
>>
>>
>>
>> E-mail: *radu.tudo...@huawei.com <radu.tudo...@huawei.com>*
>>
>> Mobile: +49 15209084330 <01520%209084330>
>>
>> Telephone: +49 891588344173 <089%201588344173>
>>
>>
>>
>> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>> Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
>> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
>> Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
>> Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
>> Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
>>
>> This e-mail and its attachments contain confidential information from
>> HUAWEI, which is intended only for the person or entity whose address is
>> listed above. Any use of the information contained herein in any way
>> (including, but not limited to, total or partial disclosure, reproduction,
>> or dissemination) by persons other than the intended recipient(s) is
>> prohibited. If you receive this e-mail in error, please notify the sender
>> by phone or email immediately and delete it!
>>
>>
>>
>> *From:* Kanstantsin Kamkou [mailto:kkam...@gmail.com]
>> *Sent:* Wednesday, December 07, 2016 9:57 PM
>> *To:* user@flink.apache.org
>> *Subject:* Re: [DISCUSS] "Who's hiring on Flink" monthly thread in the
>> mailing lists?
>>
>>
>>
>> Is it possible to avoid such a spam here? If I need a new job, I could
>> search it. The same way I might want to subscribe to a different thread,
>> like jobs@flink. * The idea itself is great.
>>
>>
>>
>> On Tue, 6 Dec 2016 at 14:04, Kostas Tzoumas <ktzou...@apache.org> wrote:
>>
>> yes, of course!
>>
>>
>>
>> On Tue, Dec 6, 2016 at 12:54 PM, Márton Balassi <balassi.mar...@gmail.com
>> > wrote:
>>
>> +1. It keeps it both organized and to a reasonable minimum overhead.
>>
>>
>>
>> Would you volunteer for starting the mail thread each month then, Kostas?
>>
>>
>>
>> Best,
>>
>>
>>
>> Marton
>>
>>
>>
>> On Tue, Dec 6, 2016 at 6:42 AM, Kostas Tzoumas <ktzou...@apache.org>
>> wrote:
>>
>> Hi folks,
>>
>>
>>
>>
>>
>> I'd like to see how the community feels about a monthly "Who is hiring on
>>
>>
>> Flink" email thread on the dev@ and user@ mailing lists where folks can
>>
>>
>> post job positions related to Flink.
>>
>>
>>
>>
>>
>> I personally think that posting individual job offerings in the mailing
>>
>>
>> list is off-topic (hence I have refrained to do that wearing my company
>>
>>
>> hat, and I have discouraged others when they asked for my opinion on
>> this),
>>
>>
>> but I thought that a monthly thread like this would be both helpful to the
>>
>>
>> community and not cause overhead.
>>
>>
>>
>>
>>
>> Cheers,
>>
>>
>> Kostas
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>


Re: [DISCUSS] "Who's hiring on Flink" monthly thread in the mailing lists?

2016-12-06 Thread Kostas Tzoumas
yes, of course!

On Tue, Dec 6, 2016 at 12:54 PM, Márton Balassi <balassi.mar...@gmail.com>
wrote:

> +1. It keeps it both organized and to a reasonable minimum overhead.
>
> Would you volunteer for starting the mail thread each month then, Kostas?
>
> Best,
>
> Marton
>
> On Tue, Dec 6, 2016 at 6:42 AM, Kostas Tzoumas <ktzou...@apache.org>
> wrote:
>
>> Hi folks,
>>
>> I'd like to see how the community feels about a monthly "Who is hiring on
>> Flink" email thread on the dev@ and user@ mailing lists where folks can
>> post job positions related to Flink.
>>
>> I personally think that posting individual job offerings in the mailing
>> list is off-topic (hence I have refrained to do that wearing my company
>> hat, and I have discouraged others when they asked for my opinion on
>> this),
>> but I thought that a monthly thread like this would be both helpful to the
>> community and not cause overhead.
>>
>> Cheers,
>> Kostas
>>
>
>


[DISCUSS] "Who's hiring on Flink" monthly thread in the mailing lists?

2016-12-06 Thread Kostas Tzoumas
Hi folks,

I'd like to see how the community feels about a monthly "Who is hiring on
Flink" email thread on the dev@ and user@ mailing lists where folks can
post job positions related to Flink.

I personally think that posting individual job offerings in the mailing
list is off-topic (hence I have refrained to do that wearing my company
hat, and I have discouraged others when they asked for my opinion on this),
but I thought that a monthly thread like this would be both helpful to the
community and not cause overhead.

Cheers,
Kostas


[ANNOUNCE] Flink Forward 2016: First round of speakers and sessions is out

2016-07-25 Thread Kostas Tzoumas
Hi everyone,

I wanted to share this with the community: we have announced the first
round of speakers and sessions of Flink Forward 2016, and it looks amazing!

Check it out here: http://flink-forward.org/program/sessions/

This year we have a great mix of use case talks (e.g., by Netflix, Alibaba,
Intel, Cisco, King, Zalando, etc), in-depth developer-oriented talks on
Flink existing and upcoming features by committers and contributors, and
several talks on the wider stream processing landscape, including Apache
Beam (incubating), streaming SQL, and more.

As a reminder, the last day for early bird tickets is this Sunday, July 31.
I'm really looking forward to seeing as many of us there as possible!

Best,
Kostas


Re: Flink on Azure HDInsight

2016-05-04 Thread Kostas Tzoumas
As far as I know, Azure HDInsight is based on Hortonworks HDP, on top of
which Flink has been used extensively.

On Mon, May 2, 2016 at 10:42 AM, Brig Lamoreaux <
brig.lamore...@microsoft.com> wrote:

> Thanks Stephan,
>
>
>
> Turns out Azure Table is slightly different than Azure HDInsight. Both use
> Azure Storage however, HDInsight allows HDFS over Azure Storage.
>
>
>
> I’d be curious if anyone has tried to use Flink on top of Azure HDInsight.
>
>
>
> Thanks,
>
> Brig
>
>
>
> *From:* ewenstep...@gmail.com [mailto:ewenstep...@gmail.com] *On Behalf
> Of *Stephan Ewen
> *Sent:* Saturday, April 30, 2016 9:36 PM
> *To:* user@flink.apache.org
> *Subject:* Re: Flink on Azure HDInsight
>
>
>
> Hi!
>
>
>
> As far as I know, some people have been using Flink together with Azure,
> and we try and do some release validation on Azure as well.
>
>
>
> There is even a section in the docs that describes how to use Hadoop's
> Azure Table formats with Flink
>
>
> https://ci.apache.org/projects/flink/flink-docs-master/apis/batch/connectors.html#access-microsoft-azure-table-storage
> 
>
>
>
> I am not aware of any Azure specific issues at this point...
>
>
>
> Greetings,
>
> Stephan
>
>
>
>
>
>
>
> On Fri, Apr 29, 2016 at 11:18 AM, Brig Lamoreaux <
> brig.lamore...@microsoft.com> wrote:
>
> Hi All,
>
>
>
> Are there any issues with Flink on Azure HDInsight?
>
>
>
> Thanks,
>
> Brig Lamoreaux
>
> Data Solution Architect
>
> US Desert/Mountain Tempe
>
>
>
>
>
> [image: MSFT_logo_Gray DE sized SIG1.png]
>
>
>
>
>
>
>
>
>


[ANNOUNCE] Flink 1.0.0 has been released

2016-03-08 Thread Kostas Tzoumas
Hi everyone!

As you might have noticed, Apache Flink 1.0.0 has been released and
announced!

You can read more about the release at the ASF blog and the Flink blog
-
https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces88
- http://flink.apache.org/news/2016/03/08/release-1.0.0.html

Don't forget to retweet and spread the news :-)
- https://twitter.com/TheASF/status/707174116969857024
- https://twitter.com/ApacheFlink/status/707175973482012672

Check out the changelog and the migration guide, download the release, and
check out the documentation
- http://flink.apache.org/blog/release_1.0.0-changelog_known_issues.html
-
https://cwiki.apache.org/confluence/display/FLINK/Migration+Guide%3A+0.10.x+to+1.0.x
- https://cwiki.apache.org/confluence/display/FLINK/Stability+Annotations
- http://flink.apache.org/downloads.html
- https://ci.apache.org/projects/flink/flink-docs-release-1.0/

Many congratulations to the Flink community for making this happen!

Best,
Kostas


Community choice for Hadoop Summit Europe 2016

2015-12-09 Thread Kostas Tzoumas
Hi everyone,

Just a reminder, the community vote for the Hadoop Summit Europe 2016 talks
in Dublin is still open until December 15.

There is a very good number of talks around Flink submitted, here are the
ones that mention "flink" in their abstract:
https://hadoopsummit.uservoice.com/search?filter=merged=flink

Vote away :-)

Best,
Kostas


Re: Fold vs Reduce in DataStream API

2015-11-18 Thread Kostas Tzoumas
Granted, both are presented with the same example in the docs. They are
modeled after reduce and fold in functional programming. Perhaps we should
have a bit more enlightening examples.

On Wed, Nov 18, 2015 at 6:39 PM, Fabian Hueske  wrote:

> Hi Ron,
>
> Have you checked:
> https://ci.apache.org/projects/flink/flink-docs-release-0.10/apis/streaming_guide.html#transformations
> ?
>
> Fold is like reduce, except that you define a start element (of a
> different type than the input type) and the result type is the type of the
> initial value. In reduce, the result type must be identical to the input
> type.
>
> Best, Fabian
>
> 2015-11-18 18:32 GMT+01:00 Ron Crocker :
>
>> Is there a succinct description of the distinction between these
>> transforms?
>>
>> Ron
>> —
>> Ron Crocker
>> Principal Engineer & Architect
>> ( ( •)) New Relic
>> rcroc...@newrelic.com
>> M: +1 630 363 8835
>>
>>
>


Powered by Flink

2015-10-19 Thread Kostas Tzoumas
Hi everyone,

I started a "Powered by Flink" wiki page, listing some of the organizations
that are using Flink:

https://cwiki.apache.org/confluence/display/FLINK/Powered+by+Flink

If you would like to be added to the list, just send me a short email with
your organization's name and a description and I will add you to the wiki
page.

Best,
Kostas


Re: Powered by Flink

2015-10-19 Thread Kostas Tzoumas
yes, definitely. How about a link under the Community drop-down that points
to the wiki page?

On Mon, Oct 19, 2015 at 2:53 PM, Fabian Hueske <fhue...@gmail.com> wrote:

> Thanks for starting this Kostas.
>
> I think the list is quite hidden in the wiki. Should we link from
> flink.apache.org to that page?
>
> Cheers, Fabian
>
> 2015-10-19 14:50 GMT+02:00 Kostas Tzoumas <ktzou...@apache.org>:
>
>> Hi everyone,
>>
>> I started a "Powered by Flink" wiki page, listing some of the
>> organizations that are using Flink:
>>
>> https://cwiki.apache.org/confluence/display/FLINK/Powered+by+Flink
>>
>> If you would like to be added to the list, just send me a short email
>> with your organization's name and a description and I will add you to the
>> wiki page.
>>
>> Best,
>> Kostas
>>
>
>


Volunteers needed for Flink Forward 2015 (and they get a free ticket)

2015-09-07 Thread Kostas Tzoumas
Hi folks,

The Flink Forward 2015 organizers are looking for volunteers (and they are
offering free tickets in exchange).

Sign up here if you are interested (or send me an email):
http://flink-forward.org/?page_id=495

Best,
Kostas


Re: About exactly once question?

2015-08-27 Thread Kostas Tzoumas
Note that the definition of exactly-once means that records are
guaranteed to be processed exactly once by Flink operators, and thus state
updates to operator state happen exactly once (e.g., if C had a counter
that x1, x2, and x3 incremented, the counter would have a value of 3 and
not a value of 6). This is not specific to Flink, but the most accepted
definition, and applicable to all stream processing systems. The reason is
that the stream processor cannot by itself guarantee what happens to the
outside world (the outside world is in this case the data sink).

See the docs (
https://ci.apache.org/projects/flink/flink-docs-master/internals/stream_checkpointing.html
):

Apache Flink offers a fault tolerance mechanism to consistently recover
the state of data streaming applications. The mechanism ensures that even
in the presence of failures, the program’s state will eventually reflect
every record from the data stream exactly once.

Guaranteeing exactly once delivery to the sink is possible, as Marton above
suggests, but the sink implementation needs to be aware and take part in
the checkpointing mechanism.


On Thu, Aug 27, 2015 at 1:14 PM, Márton Balassi balassi.mar...@gmail.com
wrote:

 Dear Zhangrucong,

 From your explanation it seems that you have a good general understanding
 of Flink's checkpointing algorithm. Your concern is valid, by default a
 sink C with emits tuples to the outside world potentially multiple times.
 A neat trick to solve this issue for your user defined sinks is to use the
 CheckpointNotifier interface to output records only after the corresponding
 checkpoint has been totally processed by the system, so sinks can also
 provid exactly once guarantees in Flink.

 This would mean that your SinkFunction has to implement both the
 Checkpointed and the CheckpointNotifier interfaces. The idea is to mark the
 output tuples with the correspoding checkpoint id, so then they can be
 emitted in a consistent manner when the checkpoint is globally
 acknowledged by the system. You buffer your output records in a collection
 of your choice and whenever a snapshotState of the Checkpointed interface
 is invoked you mark your fresh output records with the current
 checkpointID. Whenever the notifyCheckpointComplete is invoked you emit
 records with the corresponding ID.

 Note that this adds latency to your processing and as you potentially need
 to checkpoint a lot of data in the sinks I would recommend to use a HDFS as
 a state backend instead of the default solution.

 Best,

 Marton

 On Thu, Aug 27, 2015 at 12:32 PM, Zhangrucong zhangruc...@huawei.com
 wrote:

 Hi:

   The document said Flink can guarantee processing each tuple
 exactly-once, but I can not understand how it works.

For example, In Fig 1, C is running between snapshot n-1 and snapshot
 n(snapshot n hasn’t been generated). After snapshot n-1, C has processed
 tuple x1, x2, x3 and already outputted to user,  then C failed and it
 recoveries from snapshot n-1. In my opinion, x1, x2, x3 will be processed
 and outputted to user again. My question is how Flink guarantee x1,x2,x3
 are processed and outputted to user only once?





 Fig 1.

 Thanks for answing.





Re: About exactly once question?

2015-08-27 Thread Kostas Tzoumas
Oops, seems that Stephan's email covers my answer plus the plans to provide
transactional sinks :-)

On Thu, Aug 27, 2015 at 1:25 PM, Kostas Tzoumas ktzou...@apache.org wrote:

 Note that the definition of exactly-once means that records are
 guaranteed to be processed exactly once by Flink operators, and thus state
 updates to operator state happen exactly once (e.g., if C had a counter
 that x1, x2, and x3 incremented, the counter would have a value of 3 and
 not a value of 6). This is not specific to Flink, but the most accepted
 definition, and applicable to all stream processing systems. The reason is
 that the stream processor cannot by itself guarantee what happens to the
 outside world (the outside world is in this case the data sink).

 See the docs (
 https://ci.apache.org/projects/flink/flink-docs-master/internals/stream_checkpointing.html
 ):

 Apache Flink offers a fault tolerance mechanism to consistently recover
 the state of data streaming applications. The mechanism ensures that even
 in the presence of failures, the program’s state will eventually reflect
 every record from the data stream exactly once.

 Guaranteeing exactly once delivery to the sink is possible, as Marton
 above suggests, but the sink implementation needs to be aware and take part
 in the checkpointing mechanism.


 On Thu, Aug 27, 2015 at 1:14 PM, Márton Balassi balassi.mar...@gmail.com
 wrote:

 Dear Zhangrucong,

 From your explanation it seems that you have a good general understanding
 of Flink's checkpointing algorithm. Your concern is valid, by default a
 sink C with emits tuples to the outside world potentially multiple times.
 A neat trick to solve this issue for your user defined sinks is to use the
 CheckpointNotifier interface to output records only after the corresponding
 checkpoint has been totally processed by the system, so sinks can also
 provid exactly once guarantees in Flink.

 This would mean that your SinkFunction has to implement both the
 Checkpointed and the CheckpointNotifier interfaces. The idea is to mark the
 output tuples with the correspoding checkpoint id, so then they can be
 emitted in a consistent manner when the checkpoint is globally
 acknowledged by the system. You buffer your output records in a collection
 of your choice and whenever a snapshotState of the Checkpointed interface
 is invoked you mark your fresh output records with the current
 checkpointID. Whenever the notifyCheckpointComplete is invoked you emit
 records with the corresponding ID.

 Note that this adds latency to your processing and as you potentially
 need to checkpoint a lot of data in the sinks I would recommend to use a
 HDFS as a state backend instead of the default solution.

 Best,

 Marton

 On Thu, Aug 27, 2015 at 12:32 PM, Zhangrucong zhangruc...@huawei.com
 wrote:

 Hi:

   The document said Flink can guarantee processing each tuple
 exactly-once, but I can not understand how it works.

For example, In Fig 1, C is running between snapshot n-1 and snapshot
 n(snapshot n hasn’t been generated). After snapshot n-1, C has processed
 tuple x1, x2, x3 and already outputted to user,  then C failed and it
 recoveries from snapshot n-1. In my opinion, x1, x2, x3 will be processed
 and outputted to user again. My question is how Flink guarantee x1,x2,x3
 are processed and outputted to user only once?





 Fig 1.

 Thanks for answing.






[ANNOUNCE] Flink Forward 2015 program is online

2015-08-25 Thread Kostas Tzoumas
Hi everyone,

Just a shoutout that we have posted the program of Flink Forward 2015 here:
http://flink-forward.org/?post_type=day

You can expect few changes here and there, but the majority of the talks is
in.

Thanks again to the speakers and the reviewers!

If you have not registered yet, now is the time to do it :-) (here:
http://flink-forward.org/?page_id=96)

Kostas


Re: Flink 0.9 built with Scala 2.11

2015-06-10 Thread Kostas Tzoumas
Please do ping this list if you encounter any problems with Flink during
your project (you have done so already :-), but also if you find that the
Flink API needs additions to map Pig well to Flink

On Wed, Jun 10, 2015 at 3:47 PM, Philipp Goetze 
philipp.goe...@tu-ilmenau.de wrote:

 Done. Can be found here: https://issues.apache.org/jira/browse/FLINK-2200

 Best Regards,
 Philipp



 On 10.06.2015 15:29, Chiwan Park wrote:

 But I think uploading Flink API with scala 2.11 to maven repository is
 nice idea.
 Could you create a JIRA issue?

 Regards,
 Chiwan Park

  On Jun 10, 2015, at 10:23 PM, Chiwan Park chiwanp...@icloud.com wrote:

 No. Currently, there are no Flink binaries with scala 2.11 which are
 downloadable.

 Regards,
 Chiwan Park

  On Jun 10, 2015, at 10:18 PM, Philipp Goetze 
 philipp.goe...@tu-ilmenau.de wrote:

 Thank you Chiwan!

 I did not know the master has a 2.11 profile.

 But there is no pre-built Flink with 2.11, which I could refer to in
 sbt or maven, is it?

 Best Regards,
 Philipp

 On 10.06.2015 15:03, Chiwan Park wrote:

 Hi. You can build Flink with Scala 2.11 with scala-2.11 profile in
 master branch.
 `mvn clean install -DskipTests -P \!scala-2.10,scala-2.11` command
 builds Flink with Scala 2.11.

 Regards,
 Chiwan Park

  On Jun 10, 2015, at 9:56 PM, Flavio Pompermaier pomperma...@okkam.it
 wrote:

 Nice!

 On 10 Jun 2015 14:49, Philipp Goetze philipp.goe...@tu-ilmenau.de
 wrote:
 Hi community!

 We started a new project called Piglet (
 https://github.com/ksattler/piglet).
 For that we use i.a. Flink as a backend. The project is based on
 Scala 2.11. Thus we need a 2.11 build of Flink.

 Until now we used the 2.11 branch of the stratosphere project and
 built Flink ourselves. Unfortunately this branch is not up-to-date.

 Do you have an official repository for Flink 0.9 (built with Scala
 2.11)?

 Best Regards,
 Philipp














Re: CoGgroup Operator Data Sink

2015-04-14 Thread Kostas Tzoumas
Each operator has only one output (which can be consumed by multiple
downstream operators), so you cannot branch out to two different directions
from inside the user code with many collectors. The reasoning is that you
can have the same effect with what Robert suggested.

But perhaps your use case is different; can you not achieve the same result
with branching out to two different DataSets as per Robert's suggestion? If
this is the case, posting some details on the function would be helpful.

On Tue, Apr 14, 2015 at 11:37 AM, Mustafa Elbehery 
elbeherymust...@gmail.com wrote:

 Thanks for prompt reply.

 Maybe the expression Sink is not suitable to what I need. What if I want
 to *Collect* two data sets directly from the coGroup operator. Is there
 anyway to do so ?!!

 As I might know, the operator has only Collector Object, but I wonder if
 there is another feature in Flink that supports what I need .

 Thanks.

 On Tue, Apr 14, 2015 at 11:27 AM, Robert Metzger rmetz...@apache.org
 wrote:

 Hi,

 you can write the output of a coGroup operator to two sinks:

 --\   /Sink1
\ /
 (CoGroup)
/\
 --/  \--Sink2

 You can actually write to as many sinks as you want.
 Note that the data written to Sink1 and Sink2 will be identical.
 If you want to write different data to S1 and S2, you can use a Tuple2
 where the first field contains a tag, and the second field contains your
 data.
 Then, you use a filter in front of your Sinks to select the data based on
 the tag.

 --\   /---(Filter)--Sink1
\ /
 (CoGroup)
/\
 --/  \(Filter)--Sink2

 So the output of CoGroup could be Tuple2Integer,YourPojo, when the
 integer is 1, it is only written by Sink1, when the integer is 2, its only
 written by Sink2.




 On Tue, Apr 14, 2015 at 10:20 AM, Mustafa Elbehery 
 elbeherymust...@gmail.com wrote:

 Hi all,

 I wonder if the coGroup operator have the ability to sink two output
 simultaneously. I am trying to mock it by calling a function inside the
 operator, in which I sink the first output, and get the second output
 myself.

 I am not sure if this is the best way, and I would like to hear your
 suggestions,

 Regards.

 --
 Mustafa Elbehery
 EIT ICT Labs Master School http://www.masterschool.eitictlabs.eu/home/
 +49(0)15750363097
 skype: mustafaelbehery87





 --
 Mustafa Elbehery
 EIT ICT Labs Master School http://www.masterschool.eitictlabs.eu/home/
 +49(0)15750363097
 skype: mustafaelbehery87




Re: Flink meetup group in Stockholm

2015-04-08 Thread Kostas Tzoumas
Super cool!!! I'm sure you will make it a huge success :-)

On Wed, Apr 8, 2015 at 5:44 PM, Till Rohrmann trohrm...@apache.org wrote:

 Really cool :-)

 On Wed, Apr 8, 2015 at 5:09 PM, Maximilian Michels m...@apache.org wrote:

 Love the purple. Have fun! :)

 On Wed, Apr 8, 2015 at 5:05 PM, Henry Saputra henry.sapu...@gmail.com
 wrote:

 Nice, congrats!

 On Wed, Apr 8, 2015 at 7:39 AM, Gyula Fóra gyf...@apache.org wrote:
  Hey Everyone!
 
  We our proud to announce the first Apache Flink meetup group in
 Stockholm.
 
  Join us at http://www.meetup.com/Apache-Flink-Stockholm/
 
  We are looking forward to organise our first event in May!
 
  Cheers,
  Gyula






Flink Forward 2015

2015-04-07 Thread Kostas Tzoumas
Hi everyone,

The folks at data Artisans and the Berlin Big Data Center are organizing
the first physical conference all about Apache Flink in Berlin the coming
October:

http://flink-forward.org

The conference will be held in a beautiful spot an old brewery turned event
space (the same space that Berlin Buzzwords took place last year). We are
soliciting technical talks on Flink, talks on how you are using Flink to
solve real world problems, as well as talks on Big Data technology in
general that relate to Apache Flink's general direction. And of course,
there will be enough social and networking events to get the community
together :-)

The website and the call for abstracts are live, but the ticket
registration is not yet open.

At this point, I would like to ask the community to mark your calendars if
you'd like to attend, submit an abstract, and forward the event to your
friends and family. If you can help us market the event, help in any other
way, or have any other inquiries, please get in touch with me!

I will also announce this via our social media channels this week.

I am looking forward to gathering the community in a great conference!

Best,
Kostas


Re: Flink Forward 2015

2015-04-07 Thread Kostas Tzoumas
Ah, thanks Sebastian! :-)

On Tue, Apr 7, 2015 at 2:33 PM, Sebastian ssc.o...@googlemail.com wrote:

 There are still some Berlin Buzzwords snippets in your texts ;)

 http://flink-forward.org/?page_id=294


 On 07.04.2015 14:24, Kostas Tzoumas wrote:

 Hi everyone,

 The folks at data Artisans and the Berlin Big Data Center are organizing
 the first physical conference all about Apache Flink in Berlin the
 coming October:

 http://flink-forward.org

 The conference will be held in a beautiful spot an old brewery turned
 event space (the same space that Berlin Buzzwords took place last year).
 We are soliciting technical talks on Flink, talks on how you are using
 Flink to solve real world problems, as well as talks on Big Data
 technology in general that relate to Apache Flink's general direction.
 And of course, there will be enough social and networking events to get
 the community together :-)

 The website and the call for abstracts are live, but the ticket
 registration is not yet open.

 At this point, I would like to ask the community to mark your calendars
 if you'd like to attend, submit an abstract, and forward the event to
 your friends and family. If you can help us market the event, help in
 any other way, or have any other inquiries, please get in touch with me!

 I will also announce this via our social media channels this week.

 I am looking forward to gathering the community in a great conference!

 Best,
 Kostas