Re: returns method in scala api

Note that you should use `createTypeInfomation[T]` method in `org.apache.flink.api.scala` package object to create `TypeInformation` for Scala specific types such as case classes or tuples. Regards, Chiwan Park > On Apr 5, 2016, at 1:53 AM, Stephan Ewen wrote: > > Hi! > >

Re: Apache Flink <=> Apache Ignite integration

On Mon, Apr 4, 2016 at 3:54 PM, Stephan Ewen wrote: > > - Sounds like the having Ignite for snapshots should work pretty much > out > of the box (via the IGFS) > - The source and sink connector sounds like the next logical step. Does > Ignite have a notion of stream

Re: Apache Flink <=> Apache Ignite integration

On Mon, Apr 4, 2016 at 3:35 PM, Robert Metzger wrote: > thanks a lot for reaching out to the Flink community. > I'm really excited to see a Flink connector in Ignite. If you feel that > the > connector would be more suitable for our "connector library" feel free to > open a

[jira] [Created] (FLINK-3698) Scala Option can't be used as a key

Timur Fayruzov created FLINK-3698: - Summary: Scala Option can't be used as a key Key: FLINK-3698 URL: https://issues.apache.org/jira/browse/FLINK-3698 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-3697) keyBy() with nested POJO computes invalid field position indexes

Ron Crocker created FLINK-3697: -- Summary: keyBy() with nested POJO computes invalid field position indexes Key: FLINK-3697 URL: https://issues.apache.org/jira/browse/FLINK-3697 Project: Flink

Re: FYI: Added Documentation on Basic Concepts

Great overview of the distributed dataflow concepts in Flink. This stuff is not easy to break down. On Mon, Apr 4, 2016 at 6:13 PM, Ufuk Celebi wrote: > Hey Tianju, > > thanks for spotting that. It was an error and I've removed the > JavaScript. The website will be updated soon

Re: FYI: Added Documentation on Basic Concepts

Hey Tianju, thanks for spotting that. It was an error and I've removed the JavaScript. The website will be updated soon by the buildbot. – Ufuk On Mon, Apr 4, 2016 at 3:48 PM, Tianji Li wrote: > Hi Ufuk, > > Not sure if it is blocked by our firewall, but jQuery seems not

[jira] [Created] (FLINK-3696) Some Union tests fail for TableConfigMode.EFFICIENT

Vasia Kalavri created FLINK-3696: Summary: Some Union tests fail for TableConfigMode.EFFICIENT Key: FLINK-3696 URL: https://issues.apache.org/jira/browse/FLINK-3696 Project: Flink Issue

[jira] [Created] (FLINK-3695) ValueArray types

Greg Hogan created FLINK-3695: - Summary: ValueArray types Key: FLINK-3695 URL: https://issues.apache.org/jira/browse/FLINK-3695 Project: Flink Issue Type: New Feature Components: Core

Re: Broken links after doc resturcturing

Hi, I just stepped through the whole blog. Some stuff can get fixed easily, more links should just be removed, and for some I am not sure what to do about (quite old stuff). I put my though about each broken link (or nothing if I have no idea how to deal with it). Please give feedback. **

Re: Apache Flink <=> Apache Ignite integration

Hi! - Sounds like the having Ignite for snapshots should work pretty much out of the box (via the IGFS) - The source and sink connector sounds like the next logical step. Does Ignite have a notion of stream partitions and offsets, to build a consistent replay around? This should probably have

Re: Apache Flink <=> Apache Ignite integration

Hi Raul, thanks a lot for reaching out to the Flink community. I'm really excited to see a Flink connector in Ignite. If you feel that the connector would be more suitable for our "connector library" feel free to open a JIRA and open a pull request. Were there requests in the Ignite community to

[jira] [Created] (FLINK-3694) Document stoppable function interface for custom sources

Till Rohrmann created FLINK-3694: Summary: Document stoppable function interface for custom sources Key: FLINK-3694 URL: https://issues.apache.org/jira/browse/FLINK-3694 Project: Flink Issue

Re: Dynamic Scaling: State Sharding

Sorry, my bad. Comments should work now. On Mon, Apr 4, 2016 at 3:51 PM, Aljoscha Krettek wrote: > Comments are not enabled. > > On Mon, 4 Apr 2016 at 13:58 Till Rohrmann wrote: > > > Hi Flink community, > > > > I recently started working on dynamic

Re: Dynamic Scaling: State Sharding

Comments are not enabled. On Mon, 4 Apr 2016 at 13:58 Till Rohrmann wrote: > Hi Flink community, > > I recently started working on dynamic scaling. As a first step we want to > introduce state sharding which is a requirement for partitioned state to be > re-distributable.

Re: FYI: Added Documentation on Basic Concepts

Hi Ufuk, Not sure if it is blocked by our firewall, but jQuery seems not included in the concepts.html for some reason. Chrome throws me this error: Uncaught ReferenceError: $ is not defined(anonymous function) @ concepts.html:221 Thanks Tianji On Mon, Apr 4, 2016 at 5:39 AM, Ufuk Celebi

Re: Broken links after doc resturcturing

Hey Matthias, could you update the links in the posts or share a list of the posts with broken links so I can update them? We can also provide redirects for the broken links. With 1.0, we now should be more careful to not break/redirect links... On Mon, Apr 4, 2016 at 2:53 PM, Matthias J. Sax

Re: CEP blog post

Thanks a lot to all for the valuable feedback. I've incorporated your suggestions and will publish the article, once Flink 1.0.1 has been released (we need 1.0.1 to run the example code). Cheers, Till On Mon, Apr 4, 2016 at 10:29 AM, gen tang wrote: > It is really a good

Re: [VOTE] Release Apache Flink 1.0.1 (RC1)

+1 I ran "mvn clean verify" on the source. Dowloaded binary, ran example programs in cluster mode. On Mon, 4 Apr 2016 at 15:08 Maximilian Michels wrote: > On Mon, Apr 4, 2016 at 12:16 PM, Robert Metzger > wrote: > > We can probably move the force shading

Re: FYI: Updated Slides Section

Dear community, Just in case it is useful, please find below the links to the slides from the 1st Flink Madrid Meetup talks given by Fabien Hueske [1] and myself [2] (in spanish). Best [1] http://www.slideshare.net/fhueske/data-stream-processing-with-apache-flink [2]

Re: [VOTE] Release Apache Flink 1.0.1 (RC1)

On Mon, Apr 4, 2016 at 12:16 PM, Robert Metzger wrote: > We can probably move the force shading module into a (deactivated) build > profile and set the dependency to 1.0.0, so that it just looks like a > regular dependency. That's what I suggested. It's not a blocker for the

returns method in scala api

Hi, I'm writing a custom serializer so I need to call the returns(TypeInformation) method after a reduce method, but it isn't available in the scala api, only in the java api. Do you have any suggestions how to go around this and still call the correct deserializer? Thanks! Judit

Re: Aggregation Design Questions

Hi! Sorry for taking so long to react. I am looking through this now as well... Stephan On Wed, Mar 23, 2016 at 6:59 PM, Lisonbee, Todd wrote: > I wrote another design for a summarize() function on DataSet. > https://issues.apache.org/jira/browse/FLINK-3664 > > I

Broken links after doc resturcturing

Hi, I just realized, that the restructuring of the programming guide in our documentation https://github.com/apache/flink/commit/ad267a4b199979536dd8a5572628eefc77d7e0f4) broke a couple of links from the Flink blog. Should we update those older blog post? JIRA for it? For the future: would it

[jira] [Created] (FLINK-3693) JobManagerHAJobGraphRecoveryITCase.testClientNonDetachedListeningBehaviour is unstable

Robert Metzger created FLINK-3693: - Summary: JobManagerHAJobGraphRecoveryITCase.testClientNonDetachedListeningBehaviour is unstable Key: FLINK-3693 URL: https://issues.apache.org/jira/browse/FLINK-3693

Re: [VOTE] Release Apache Flink 1.0.1 (RC1)

+1 for the 1.0.1 release: - I checked the staged files for maven. The quickstarts look good, other modules seem to have correct scala versions - I ran the Kafka examples (locally, against Kafka 0.8) - Started a yarn session on a CDH 5.4.4 cluster (6 nodes) & ran a job with rocksdb state,

Dynamic Scaling: State Sharding

Hi Flink community, I recently started working on dynamic scaling. As a first step we want to introduce state sharding which is a requirement for partitioned state to be re-distributable. The following design doc [1] explains the ideas behind it. Any feedback is highly appreciated :-) [1]

Re: FYI: Updated Slides Section

Hi Ufuk, Thanks for updating the page. The "latest documentation" points to the page itself and not the documentation. I've fixed that and added the slides from Big Data Warsaw. Cheers, Max On Mon, Apr 4, 2016 at 12:09 PM, Ufuk Celebi wrote: > @Paris: Just added it. Thanks for

Re: [VOTE] Release Apache Flink 1.0.1 (RC1)

The scripts in master and release-1.0 had different behaviour and I went with the behaviour in master (e.g. version X-SNAPSHOT for force-shading, which gets updated by the change version script). I think that we can decide this independently of this release. No one has this as a dependency

Re: [VOTE] Release Apache Flink 1.0.1 (RC1)

We can probably move the force shading module into a (deactivated) build profile and set the dependency to 1.0.0, so that it just looks like a regular dependency. On Mon, Apr 4, 2016 at 12:12 PM, Maximilian Michels wrote: > The version of force-shading had to be a snapshot

Re: [VOTE] Release Apache Flink 1.0.1 (RC1)

The version of force-shading had to be a snapshot version, otherwise the nightly deployment of snapshots wouldn't work. However, since we have 1.0.0 already in place, we could revert the version to 1.0.0 again and skip the deployment from now on. I don't think it makes much of a difference but it

Re: FYI: Updated Slides Section

@Paris: Just added it. Thanks for the pointer. Great slides!

Re: FYI: Updated Slides Section

Some people might find my slides on the FT fundamentals from last summer interesting. If you like it feel free to include it. http://www.slideshare.net/ParisCarbone/tech-talk-google-on-flink-fault-tolerance-and-ha Paris On 04 Apr 2016, at 11:33, Ufuk Celebi

FYI: Added Documentation on Basic Concepts

Dear Flink community, I'm happy to announce that we have added a long overdue section on general Flink concepts to the documentation. You can find it here: https://ci.apache.org/projects/flink/flink-docs-release-1.0/concepts/concepts.html Thanks to Stephan Ewen who wrote this great overview. I

FYI: Updated Slides Section

Dear Flink community, I have updated the Material section on the Flink project page and moved the slides section to a separate page. You can find links to slides and talks here now: http://flink.apache.org/slides.html I've added slides for talks from this year by Till Rohrmann, Vasia Kalavri,

Re: [VOTE] Release Apache Flink 1.0.1 (RC1)

One quick question regarding the release: Are you intentionally releasing a 1.0.1 version of the force-shading module? Since it is really a work around, I wounder if we should not release the force-shading anymore and always depend on force-shading:1.0.0. On Sat, Apr 2, 2016 at 9:25 PM,

Re: CEP blog post

Made a few suggestions. Reads well, Till! On Mon, Apr 4, 2016 at 10:10 AM, Ufuk Celebi wrote: > Same here. > > +1 to publish > > On Mon, Apr 4, 2016 at 10:04 AM, Aljoscha Krettek wrote: >> Hi, >> I like it. Very dense and very focused on the example but I

Re: CEP blog post

Same here. +1 to publish On Mon, Apr 4, 2016 at 10:04 AM, Aljoscha Krettek wrote: > Hi, > I like it. Very dense and very focused on the example but I think it should > be good for the Flink Blog. > > -- > aljoscha > > On Fri, 1 Apr 2016 at 15:56 Till Rohrmann

Re: CEP blog post

Hi, I like it. Very dense and very focused on the example but I think it should be good for the Flink Blog. -- aljoscha On Fri, 1 Apr 2016 at 15:56 Till Rohrmann wrote: > Hi Flink community, > > I've written a short blog [1] post about Flink's new CEP library which >