date:20151017

[jira] [Commented] (FLINK-2411) Add basic graph summarization algorithm

2015-10-17 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/FLINK-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962006#comment-14962006
 ] 

ASF GitHub Bot commented on FLINK-2411:
---

GitHub user s1ck opened a pull request:

https://github.com/apache/flink/pull/1264

[FLINK-2411] Add graph summarization algorithm

* added algorithm
* added integration tests

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/s1ck/flink FLINK-2411

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/1264.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1264


commit 3ac42a6c23fc900b6451174b5b463cdba60951da
Author: Martin Junghanns 
Date:   2015-10-17T14:59:32Z

[FLINK-2411] Add graph summarization algorithm

* added algorithm
* added integration tests




> Add basic graph summarization algorithm
> ---
>
> Key: FLINK-2411
> URL: https://issues.apache.org/jira/browse/FLINK-2411
> Project: Flink
>  Issue Type: New Feature
>  Components: Gelly
>Affects Versions: 0.10
>Reporter: Martin Junghanns
>Assignee: Martin Junghanns
>Priority: Minor
>
> Graph summarization determines a structural grouping of similar vertices and 
> edges to condense a graph and thus helps to uncover insights about patterns 
> hidden in the graph. It can be used in OLAP-style operations on the graph and 
> is similar to group by in SQL but on the graph structure instead of rows.
>  
> The graph summarization operator represents every vertex group by a single 
> vertex in the summarized graph; edges between vertices in the summary graph 
> represent a group of edges between the vertex group members of the original 
> graph. Summarization is defined by specifying grouping keys for vertices and 
> edges, respectively.
> One publication that presents a Map/Reduce based approach is "Pagrol: 
> Parallel graph olap over large-scale attributed graphs", however they 
> pre-compute the graph-cube before it can be analyzed. With Flink, we can give 
> the user an interactive way of summarizing the graph and do not need to 
> compute the  cube beforehand.
> A more complex approach focuses on summarization on graph patterns  
> "SynopSys: Large Graph Analytics in the SAP HANA Database Through 
> Summarization".
> However, I want to start with a simple algorithm that summarizes the graph on 
> vertex and optionally edge values and additionally stores a count aggregate 
> at summarized vertices/edges.
> Consider the following two examples (e.g., social network with users from 
> cities and friendships with timestamp):
>  
> h4. Input graph:
>  
> Vertices (id, value):
> (0, Leipzig)
> (1, Leipzig)
> (2, Dresden)
> (3, Dresden)
> (4, Dresden)
> (5, Berlin)
> Edges (source, target, value):
> (0, 1, 2014)
> (1, 0, 2014)
> (1, 2, 2013)
> (2, 1, 2013)
> (2, 3, 2014)
> (3, 2, 2014)
> (4, 0, 2013)
> (4, 1, 2015)
> (5, 2, 2015)
> (5, 3, 2015)
> h4. Output graph (summarized on vertex value):
> Vertices (id, value, count)
> (0, Leipzig, 2) // "2 users from Leipzig"
> (2, Dresden, 3) // "3 users from Dresden"
> (5, Berlin, 1) // "1 user from Berlin"
> Edges (source, target, count) 
> (0, 0, 2) // "2 edges between users in Leipzig"
> (0, 2, 1) // "1 edge from users in Leipzig to users in Dresden"
> (2, 0, 3) // "3 edges from users in Dresden to users in Leipzig"
> (2, 2, 2) // "2 edges between users in Dresden"
> (5, 2, 2) // "2 edges from users in Berlin to users in Dresden"
> h4. Output graph (summarized on vertex and edge value):
> Vertices (id, value, count)
> (0, Leipzig, 2)
> (2, Dresden, 3)
> (5, Berlin, 1)
> Edges (source, target, value, count) 
> (0, 0, 2014, 2) // ...
> (0, 2, 2013, 1) // ...
> (2, 0, 2013, 2) // "2 edges from users in Dresden to users in Leipzig with 
> timestamp 2013"
> (2, 0, 2015, 1) // "1 edge from users in Dresden to users in Leipzig with 
> timestamp 2015"
> (2, 2, 2014, 2) // ...
> (5, 2, 2015, 2) // ...
> I've already implemented two versions of the summarization algorithm in our 
> own project [Gradoop|https://github.com/dbs-leipzig/gradoop], which is a 
> graph analytics stack on top of Hadoop + Gelly/Flink with a fixed data model. 
> You can see the current WIP here: 
> 1 [Abstract 
> summarization|https://github.com/dbs-leipzig/gradoop/blob/%2345_gradoop_flink/gradoop-flink/src/main/java/org/gradoop/model/impl/operators/Summarization.java]
> 2 [Implementation using 
> cross|https://github.com/dbs-leipzig/gradoop/blob/%2345_gradoop_flink/gradoop-flink/src/main/java/org/gradoop/model/impl/operators/SummarizationCross.java]
> 3 [Implementation using 
>

[GitHub] flink pull request: [FLINK-2411] Add graph summarization algorithm

2015-10-17 Thread s1ck

GitHub user s1ck opened a pull request:

https://github.com/apache/flink/pull/1264

[FLINK-2411] Add graph summarization algorithm

* added algorithm
* added integration tests

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/s1ck/flink FLINK-2411

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/1264.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1264


commit 3ac42a6c23fc900b6451174b5b463cdba60951da
Author: Martin Junghanns 
Date:   2015-10-17T14:59:32Z

[FLINK-2411] Add graph summarization algorithm

* added algorithm
* added integration tests




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-2858) Cannot build Flink Scala 2.11 with IntelliJ

2015-10-17 Thread Alexander Alexandrov (JIRA)


[ 
https://issues.apache.org/jira/browse/FLINK-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961846#comment-14961846
 ] 

Alexander Alexandrov commented on FLINK-2858:
-

I like the idea for a marker-based activation suggested here:

http://stackoverflow.com/a/8391313

What do you think?

> Cannot build Flink Scala 2.11 with IntelliJ
> ---
>
> Key: FLINK-2858
> URL: https://issues.apache.org/jira/browse/FLINK-2858
> Project: Flink
>  Issue Type: Bug
>  Components: Build System
>Affects Versions: 0.10
>Reporter: Till Rohrmann
>
> If I activate the scala-2.11 profile from within IntelliJ (and thus 
> deactivate the scala-2.10 profile) in order to build Flink with Scala 2.11, 
> then Flink cannot be built. The problem is that some Scala macros cannot be 
> expanded because they were compiled with the wrong version (I assume 2.10).
> This makes debugging tests with Scala 2.11 in IntelliJ impossible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] flink pull request: [FLINK-2809] [scala-api] Added UnitTypeInfo an...

2015-10-17 Thread aalexandrov

Github user aalexandrov commented on the pull request:

https://github.com/apache/flink/pull/1217#issuecomment-148901986
  
> Not all methods without paremeters should translate to methods without 
parenthesis...

@StephanEwen I agree with that, but I cannot understand how the 
`UnitTypeInfo` might cause a confusion here.

The typeInformation macros are synthesized by the macro based on the 
inferred collection type, which means that the meaning of `()` is resolved 
before that. Consider the following example:

```scala
// in the Scala REPL

case class Foo(answer: Int)
// defined class Foo

def f1(): Foo = Foo(42)
// f1: ()Foo

def f2: Foo = Foo(42)
// f2: Foo

val xs = Seq(f1(), f2) // how a literate person would write it
// xs: Seq[Foo] = List(Foo(42), Foo(42))

val xs = Seq(f1, f2) // how a dazed & confused person would write it, but 
still compiles  
// xs: Seq[Foo] = List(Foo(42), Foo(42))

val xs = Seq(f1, f2()) // even worse, but this breaks with a compiler 
exception
// error: Foo does not take parameters
//   val xs = Seq(f1, f2())

val xs = Seq((), ()) // typing '()' without syntactic context resolves to 
Unit
// xs: Seq[Unit] = List((), ())
```

In all of the above situations `env.fromCollection(xs)` is (1) either going 
to typecheck and trigger `TypeInformation` synthesis or (2) fail with the above.

Can you point to StackOverflow conversation or something similar where the 
issue you mention is explained with an example?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-2692) Untangle CsvInputFormat into PojoTypeCsvInputFormat and TupleTypeCsvInputFormat

2015-10-17 Thread Chesnay Schepler (JIRA)


[ 
https://issues.apache.org/jira/browse/FLINK-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961847#comment-14961847
 ] 

Chesnay Schepler commented on FLINK-2692:
-

[~aljoscha] Great, that will work.

Another question: the CommonCsvInputFormat contains a method setFields(int[] 
sourceFieldIndices, Class[] fieldTypes) . This method is not exposed in the 
CsvReader and only used in tests. Can it be removed? (together with 
setFieldsGeneric(int[] ...) in GenericCsvInputFormat)

> Untangle CsvInputFormat into PojoTypeCsvInputFormat and 
> TupleTypeCsvInputFormat 
> 
>
> Key: FLINK-2692
> URL: https://issues.apache.org/jira/browse/FLINK-2692
> Project: Flink
>  Issue Type: Improvement
>Reporter: Till Rohrmann
>Assignee: Chesnay Schepler
>Priority: Minor
>
> The {{CsvInputFormat}} currently allows to return values as a {{Tuple}} or a 
> {{Pojo}} type. As a consequence, the processing logic, which has to work for 
> both types, is overly complex. For example, the {{CsvInputFormat}} contains 
> fields which are only used when a Pojo is returned. Moreover, the pojo field 
> information are constructed by calling setter methods which have to be called 
> in a very specific order, otherwise they fail. E.g. one first has to call 
> {{setFieldTypes}} before calling {{setOrderOfPOJOFields}}, otherwise the 
> number of fields might be different. Furthermore, some of the methods can 
> only be called if the return type is a {{Pojo}} type, because they expect 
> that a {{PojoTypeInfo}} is present.
> I think the {{CsvInputFormat}} should be refactored to make the code more 
> easily maintainable. I propose to split it up into a 
> {{PojoTypeCsvInputFormat}} and a {{TupleTypeCsvInputFormat}} which take all 
> the required information via their constructors instead of using the 
> {{setFields}} and {{setOrderOfPOJOFields}} approach.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2809) DataSet[Unit] doesn't work

2015-10-17 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/FLINK-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961835#comment-14961835
 ] 

ASF GitHub Bot commented on FLINK-2809:
---

Github user aalexandrov commented on the pull request:

https://github.com/apache/flink/pull/1217#issuecomment-148901986
  
> Not all methods without paremeters should translate to methods without 
parenthesis...

@StephanEwen I agree with that, but I cannot understand how the 
`UnitTypeInfo` might cause a confusion here.

The typeInformation macros are synthesized by the macro based on the 
inferred collection type, which means that the meaning of `()` is resolved 
before that. Consider the following example:

```scala
// in the Scala REPL

case class Foo(answer: Int)
// defined class Foo

def f1(): Foo = Foo(42)
// f1: ()Foo

def f2: Foo = Foo(42)
// f2: Foo

val xs = Seq(f1(), f2) // how a literate person would write it
// xs: Seq[Foo] = List(Foo(42), Foo(42))

val xs = Seq(f1, f2) // how a dazed & confused person would write it, but 
still compiles  
// xs: Seq[Foo] = List(Foo(42), Foo(42))

val xs = Seq(f1, f2()) // even worse, but this breaks with a compiler 
exception
// error: Foo does not take parameters
//   val xs = Seq(f1, f2())

val xs = Seq((), ()) // typing '()' without syntactic context resolves to 
Unit
// xs: Seq[Unit] = List((), ())
```

In all of the above situations `env.fromCollection(xs)` is (1) either going 
to typecheck and trigger `TypeInformation` synthesis or (2) fail with the above.

Can you point to StackOverflow conversation or something similar where the 
issue you mention is explained with an example?


> DataSet[Unit] doesn't work
> --
>
> Key: FLINK-2809
> URL: https://issues.apache.org/jira/browse/FLINK-2809
> Project: Flink
>  Issue Type: Bug
>  Components: Scala API
>Reporter: Gabor Gevay
>Assignee: Gabor Gevay
>Priority: Minor
>
> The following code creates a DataSet\[Unit\]:
> val env = ExecutionEnvironment.createLocalEnvironment()
> val a = env.fromElements(1,2,3)
> val b = a.map (_ => ())
> b.writeAsText("/tmp/xxx")
> env.execute()
> This doesn't work, because a VoidSerializer is created, which can't cope with 
> a BoxedUnit. See exception below.
> I'm now thinking about creating a UnitSerializer class.
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
>   at 
> org.apache.flink.runtime.jobmanager.JobManager$$anonfun$receiveWithLogMessages$1.applyOrElse(JobManager.scala:314)
>   at 
> scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
>   at 
> org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:36)
>   at 
> org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:29)
>   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
>   at 
> org.apache.flink.runtime.ActorLogMessages$$anon$1.applyOrElse(ActorLogMessages.scala:29)
>   at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
>   at 
> org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:92)
>   at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
>   at akka.actor.ActorCell.invoke(ActorCell.scala:487)
>   at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
>   at akka.dispatch.Mailbox.run(Mailbox.scala:221)
>   at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
>   at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>   at 
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>   at 
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>   at 
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> Caused by: java.lang.ClassCastException: scala.runtime.BoxedUnit cannot be 
> cast to java.lang.Void
>   at 
> org.apache.flink.api.common.typeutils.base.VoidSerializer.serialize(VoidSerializer.java:26)
>   at 
> org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:51)
>   at 
> org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializer.addRecord(SpanningRecordSerializer.java:76)
>   at 
> org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:83)
>   at 
> org.apache.flink.runtime.operators.shipping.OutputCollector.collect(OutputCollector.java:65)
>   at 
> org.apache.flink.runtime.operators.chaining.ChainedMapDriver.collect(ChainedMapDriver.java:78)
>   at 
> org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:177)
>   at

[jira] [Commented] (FLINK-2692) Untangle CsvInputFormat into PojoTypeCsvInputFormat and TupleTypeCsvInputFormat

2015-10-17 Thread Aljoscha Krettek (JIRA)


[ 
https://issues.apache.org/jira/browse/FLINK-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961848#comment-14961848
 ] 

Aljoscha Krettek commented on FLINK-2692:
-

If it is only used in tests, I would think so.

> Untangle CsvInputFormat into PojoTypeCsvInputFormat and 
> TupleTypeCsvInputFormat 
> 
>
> Key: FLINK-2692
> URL: https://issues.apache.org/jira/browse/FLINK-2692
> Project: Flink
>  Issue Type: Improvement
>Reporter: Till Rohrmann
>Assignee: Chesnay Schepler
>Priority: Minor
>
> The {{CsvInputFormat}} currently allows to return values as a {{Tuple}} or a 
> {{Pojo}} type. As a consequence, the processing logic, which has to work for 
> both types, is overly complex. For example, the {{CsvInputFormat}} contains 
> fields which are only used when a Pojo is returned. Moreover, the pojo field 
> information are constructed by calling setter methods which have to be called 
> in a very specific order, otherwise they fail. E.g. one first has to call 
> {{setFieldTypes}} before calling {{setOrderOfPOJOFields}}, otherwise the 
> number of fields might be different. Furthermore, some of the methods can 
> only be called if the return type is a {{Pojo}} type, because they expect 
> that a {{PojoTypeInfo}} is present.
> I think the {{CsvInputFormat}} should be refactored to make the code more 
> easily maintainable. I propose to split it up into a 
> {{PojoTypeCsvInputFormat}} and a {{TupleTypeCsvInputFormat}} which take all 
> the required information via their constructors instead of using the 
> {{setFields}} and {{setOrderOfPOJOFields}} approach.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2692) Untangle CsvInputFormat into PojoTypeCsvInputFormat and TupleTypeCsvInputFormat

2015-10-17 Thread Chesnay Schepler (JIRA)


[ 
https://issues.apache.org/jira/browse/FLINK-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961858#comment-14961858
 ] 

Chesnay Schepler commented on FLINK-2692:
-

ah my bad, its available through the Scala API's ExecutionEnvironment, so it's 
staying then...

> Untangle CsvInputFormat into PojoTypeCsvInputFormat and 
> TupleTypeCsvInputFormat 
> 
>
> Key: FLINK-2692
> URL: https://issues.apache.org/jira/browse/FLINK-2692
> Project: Flink
>  Issue Type: Improvement
>Reporter: Till Rohrmann
>Assignee: Chesnay Schepler
>Priority: Minor
>
> The {{CsvInputFormat}} currently allows to return values as a {{Tuple}} or a 
> {{Pojo}} type. As a consequence, the processing logic, which has to work for 
> both types, is overly complex. For example, the {{CsvInputFormat}} contains 
> fields which are only used when a Pojo is returned. Moreover, the pojo field 
> information are constructed by calling setter methods which have to be called 
> in a very specific order, otherwise they fail. E.g. one first has to call 
> {{setFieldTypes}} before calling {{setOrderOfPOJOFields}}, otherwise the 
> number of fields might be different. Furthermore, some of the methods can 
> only be called if the return type is a {{Pojo}} type, because they expect 
> that a {{PojoTypeInfo}} is present.
> I think the {{CsvInputFormat}} should be refactored to make the code more 
> easily maintainable. I propose to split it up into a 
> {{PojoTypeCsvInputFormat}} and a {{TupleTypeCsvInputFormat}} which take all 
> the required information via their constructors instead of using the 
> {{setFields}} and {{setOrderOfPOJOFields}} approach.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (FLINK-2863) Kafka producer does not fail in case of write failure

2015-10-17 Thread Stephan Ewen (JIRA)


 [ 
https://issues.apache.org/jira/browse/FLINK-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stephan Ewen resolved FLINK-2863.
-
Resolution: Fixed

Done in a8eeb3bb1314d6c6cc19001181539d74bc25f419

> Kafka producer does not fail in case of write failure
> -
>
> Key: FLINK-2863
> URL: https://issues.apache.org/jira/browse/FLINK-2863
> Project: Flink
>  Issue Type: Bug
>  Components: Kafka Connector
>Affects Versions: 0.10
>Reporter: Stephan Ewen
>Assignee: Stephan Ewen
> Fix For: 0.10
>
>
> The async producer used in the Kafka connector only logs errors, but does not 
> fail the program in case of an error.
> I will change it such that it fails by default on error and add a flag for 
> the "lenient" mode that only logs failures.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLINK-2844) Remove old web interface and default to the new one

2015-10-17 Thread Stephan Ewen (JIRA)


 [ 
https://issues.apache.org/jira/browse/FLINK-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stephan Ewen closed FLINK-2844.
---

> Remove old web interface and default to the new one
> ---
>
> Key: FLINK-2844
> URL: https://issues.apache.org/jira/browse/FLINK-2844
> Project: Flink
>  Issue Type: New Feature
>  Components: JobManager
>Affects Versions: 0.10
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
> Fix For: 0.10
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (FLINK-2841) Broken roadmap link in FlinkML contribution guide

2015-10-17 Thread Stephan Ewen (JIRA)


 [ 
https://issues.apache.org/jira/browse/FLINK-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stephan Ewen resolved FLINK-2841.
-
   Resolution: Fixed
Fix Version/s: 010

Fixed via 580768c3075e794cec248c8bf8de97b754a41021

Thank you for the contribution!

> Broken roadmap link in FlinkML contribution guide
> -
>
> Key: FLINK-2841
> URL: https://issues.apache.org/jira/browse/FLINK-2841
> Project: Flink
>  Issue Type: Bug
>  Components: Documentation, Machine Learning Library
>Affects Versions: 0.10
>Reporter: Chiwan Park
>Assignee: Saumitra Shahapure
>  Labels: starter
> Fix For: 010
>
>
> Because the roadmap of FlinkML is moved to wiki, we need to update roadmap 
> link in [FlinkML contribution 
> guide|https://ci.apache.org/projects/flink/flink-docs-master/libs/ml/contribution_guide.html].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLINK-2841) Broken roadmap link in FlinkML contribution guide

2015-10-17 Thread Stephan Ewen (JIRA)


 [ 
https://issues.apache.org/jira/browse/FLINK-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stephan Ewen closed FLINK-2841.
---

> Broken roadmap link in FlinkML contribution guide
> -
>
> Key: FLINK-2841
> URL: https://issues.apache.org/jira/browse/FLINK-2841
> Project: Flink
>  Issue Type: Bug
>  Components: Documentation, Machine Learning Library
>Affects Versions: 0.10
>Reporter: Chiwan Park
>Assignee: Saumitra Shahapure
>  Labels: starter
> Fix For: 010
>
>
> Because the roadmap of FlinkML is moved to wiki, we need to update roadmap 
> link in [FlinkML contribution 
> guide|https://ci.apache.org/projects/flink/flink-docs-master/libs/ml/contribution_guide.html].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (FLINK-2844) Remove old web interface and default to the new one

2015-10-17 Thread Stephan Ewen (JIRA)


 [ 
https://issues.apache.org/jira/browse/FLINK-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stephan Ewen resolved FLINK-2844.
-
Resolution: Fixed

Done in 45931095180f4fee9426bdcfb8b713ff5da0b0d2

> Remove old web interface and default to the new one
> ---
>
> Key: FLINK-2844
> URL: https://issues.apache.org/jira/browse/FLINK-2844
> Project: Flink
>  Issue Type: New Feature
>  Components: JobManager
>Affects Versions: 0.10
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
> Fix For: 0.10
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (FLINK-2863) Kafka producer does not fail in case of write failure

2015-10-17 Thread Stephan Ewen (JIRA)


 [ 
https://issues.apache.org/jira/browse/FLINK-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stephan Ewen closed FLINK-2863.
---

> Kafka producer does not fail in case of write failure
> -
>
> Key: FLINK-2863
> URL: https://issues.apache.org/jira/browse/FLINK-2863
> Project: Flink
>  Issue Type: Bug
>  Components: Kafka Connector
>Affects Versions: 0.10
>Reporter: Stephan Ewen
>Assignee: Stephan Ewen
> Fix For: 0.10
>
>
> The async producer used in the Kafka connector only logs errors, but does not 
> fail the program in case of an error.
> I will change it such that it fails by default on error and add a flag for 
> the "lenient" mode that only logs failures.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] flink pull request: Removed broken dependency to flink-spargel.

2015-10-17 Thread hsaputra

Github user hsaputra commented on the pull request:

https://github.com/apache/flink/pull/1259#issuecomment-148939135
  
@StephanEwen has this been merged as part of your other commit?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2841] Correcting roadmap link to point ...

2015-10-17 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/1254


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-2844) Remove old web interface and default to the new one

2015-10-17 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/FLINK-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962047#comment-14962047
 ] 

ASF GitHub Bot commented on FLINK-2844:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/1246


> Remove old web interface and default to the new one
> ---
>
> Key: FLINK-2844
> URL: https://issues.apache.org/jira/browse/FLINK-2844
> Project: Flink
>  Issue Type: New Feature
>  Components: JobManager
>Affects Versions: 0.10
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
> Fix For: 0.10
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] flink pull request: [FLINK-2844][jobmanager] remove old web interf...

2015-10-17 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/1246


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-2841) Broken roadmap link in FlinkML contribution guide

2015-10-17 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/FLINK-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962046#comment-14962046
 ] 

ASF GitHub Bot commented on FLINK-2841:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/1254


> Broken roadmap link in FlinkML contribution guide
> -
>
> Key: FLINK-2841
> URL: https://issues.apache.org/jira/browse/FLINK-2841
> Project: Flink
>  Issue Type: Bug
>  Components: Documentation, Machine Learning Library
>Affects Versions: 0.10
>Reporter: Chiwan Park
>Assignee: Saumitra Shahapure
>  Labels: starter
>
> Because the roadmap of FlinkML is moved to wiki, we need to update roadmap 
> link in [FlinkML contribution 
> guide|https://ci.apache.org/projects/flink/flink-docs-master/libs/ml/contribution_guide.html].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] flink pull request: Removed broken dependency to flink-spargel.

2015-10-17 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/1259


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2864] Make State of General-Purpose Win...

2015-10-17 Thread aljoscha

GitHub user aljoscha opened a pull request:

https://github.com/apache/flink/pull/1265

[FLINK-2864] Make State of General-Purpose Window Operators Fault-Tolerant

This adds method state() on Trigger context that should be used to
create an OperatorState to deal with fault-tolerant state.

WindowAssigner now has a method getWindowSerializer() that is used to
get a TypeSerializer for the Windows that it assigns. The Serializer for
the Key is retrieved from the input KeyedStream and the serializer for
the input elements is already available.

During checkpointing all currently in-flight windows (per key, per
window) are serialized using the TypeSerializers. The state that is
accessible in Triggers using state() is kept in a
HashMap, this is serialized using java
serialization.

This introduces the restriction that the element must be Serializable when 
using DeltaTrigger. I did not yet take the step of integrating triggers with 
the operator-provided key-value state since this would require state to be very 
dynamic and also allow deletion of state.

@StephanEwen could you please have a look at how the state 
checkpoint/restore is implemented.

Also the triggers have a state interface that differs from the state 
interface that user functions have, for now.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aljoscha/flink window-state

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/1265.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1265


commit fb5733f52df8eabfd88f51fb39f83930f27befbd
Author: Aljoscha Krettek 
Date:   2015-10-11T09:37:29Z

[FLINK-2864] Make State of General-Purpose Window Operators Fault-Tolerant

This adds method state() on Trigger context that should be used to
create an OperatorState to deal with fault-tolerant state.

WindowAssigner now has a method getWindowSerializer() that is used to
get a TypeSerializer for the Windows that it assigns. The Serializer for
the Key is retrieved from the input KeyedStream and the serializer for
the input elements is already available.

During checkpointing all currently in-fligh windows (per key, per
window) are serialized using the TypeSerializers. The state that is
accessible in Triggers using state() is kept in a
HashMap, this is serialized using java
serialization.

commit 07d96f5cc41ae70006699c3f0a6986252565e3df
Author: Aljoscha Krettek 
Date:   2015-10-17T11:35:24Z

Replace Trigger.onTime by Trigger.onProcessingTime/onEventTime

This also renames WatermarkTrigger to EventTimeTrigger and
ContinuousWatermarkTrigger to ContinuousEventTimeTrigger.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-2864) Make State of General-Purpose Window Operators Fault-Tolerant

2015-10-17 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/FLINK-2864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962073#comment-14962073
 ] 

ASF GitHub Bot commented on FLINK-2864:
---

GitHub user aljoscha opened a pull request:

https://github.com/apache/flink/pull/1265

[FLINK-2864] Make State of General-Purpose Window Operators Fault-Tolerant

This adds method state() on Trigger context that should be used to
create an OperatorState to deal with fault-tolerant state.

WindowAssigner now has a method getWindowSerializer() that is used to
get a TypeSerializer for the Windows that it assigns. The Serializer for
the Key is retrieved from the input KeyedStream and the serializer for
the input elements is already available.

During checkpointing all currently in-flight windows (per key, per
window) are serialized using the TypeSerializers. The state that is
accessible in Triggers using state() is kept in a
HashMap, this is serialized using java
serialization.

This introduces the restriction that the element must be Serializable when 
using DeltaTrigger. I did not yet take the step of integrating triggers with 
the operator-provided key-value state since this would require state to be very 
dynamic and also allow deletion of state.

@StephanEwen could you please have a look at how the state 
checkpoint/restore is implemented.

Also the triggers have a state interface that differs from the state 
interface that user functions have, for now.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aljoscha/flink window-state

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/1265.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1265


commit fb5733f52df8eabfd88f51fb39f83930f27befbd
Author: Aljoscha Krettek 
Date:   2015-10-11T09:37:29Z

[FLINK-2864] Make State of General-Purpose Window Operators Fault-Tolerant

This adds method state() on Trigger context that should be used to
create an OperatorState to deal with fault-tolerant state.

WindowAssigner now has a method getWindowSerializer() that is used to
get a TypeSerializer for the Windows that it assigns. The Serializer for
the Key is retrieved from the input KeyedStream and the serializer for
the input elements is already available.

During checkpointing all currently in-fligh windows (per key, per
window) are serialized using the TypeSerializers. The state that is
accessible in Triggers using state() is kept in a
HashMap, this is serialized using java
serialization.

commit 07d96f5cc41ae70006699c3f0a6986252565e3df
Author: Aljoscha Krettek 
Date:   2015-10-17T11:35:24Z

Replace Trigger.onTime by Trigger.onProcessingTime/onEventTime

This also renames WatermarkTrigger to EventTimeTrigger and
ContinuousWatermarkTrigger to ContinuousEventTimeTrigger.




> Make State of General-Purpose Window Operators Fault-Tolerant
> -
>
> Key: FLINK-2864
> URL: https://issues.apache.org/jira/browse/FLINK-2864
> Project: Flink
>  Issue Type: Sub-task
>  Components: Streaming
>Reporter: Aljoscha Krettek
>Assignee: Aljoscha Krettek
> Fix For: 0.10
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] flink pull request: Fix Avro getter/setter recognition

2015-10-17 Thread ukarlsson

Github user ukarlsson commented on the pull request:

https://github.com/apache/flink/pull/1252#issuecomment-148949103
  
Hello, thanks for your feedback, yes I will add these things.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-2411) Add basic graph summarization algorithm

[GitHub] flink pull request: [FLINK-2411] Add graph summarization algorithm

[jira] [Commented] (FLINK-2858) Cannot build Flink Scala 2.11 with IntelliJ

[GitHub] flink pull request: [FLINK-2809] [scala-api] Added UnitTypeInfo an...

[jira] [Commented] (FLINK-2692) Untangle CsvInputFormat into PojoTypeCsvInputFormat and TupleTypeCsvInputFormat

[jira] [Commented] (FLINK-2809) DataSet[Unit] doesn't work

[jira] [Commented] (FLINK-2692) Untangle CsvInputFormat into PojoTypeCsvInputFormat and TupleTypeCsvInputFormat

[jira] [Commented] (FLINK-2692) Untangle CsvInputFormat into PojoTypeCsvInputFormat and TupleTypeCsvInputFormat

[jira] [Resolved] (FLINK-2863) Kafka producer does not fail in case of write failure

[jira] [Closed] (FLINK-2844) Remove old web interface and default to the new one

[jira] [Resolved] (FLINK-2841) Broken roadmap link in FlinkML contribution guide

[jira] [Closed] (FLINK-2841) Broken roadmap link in FlinkML contribution guide

[jira] [Resolved] (FLINK-2844) Remove old web interface and default to the new one

[jira] [Closed] (FLINK-2863) Kafka producer does not fail in case of write failure

[GitHub] flink pull request: Removed broken dependency to flink-spargel.

[GitHub] flink pull request: [FLINK-2841] Correcting roadmap link to point ...

[jira] [Commented] (FLINK-2844) Remove old web interface and default to the new one

[GitHub] flink pull request: [FLINK-2844][jobmanager] remove old web interf...

[jira] [Commented] (FLINK-2841) Broken roadmap link in FlinkML contribution guide

[GitHub] flink pull request: Removed broken dependency to flink-spargel.

[GitHub] flink pull request: [FLINK-2864] Make State of General-Purpose Win...

[jira] [Commented] (FLINK-2864) Make State of General-Purpose Window Operators Fault-Tolerant

[GitHub] flink pull request: Fix Avro getter/setter recognition

23 matches

Site Navigation

Mail list logo

Footer information