[GitHub] flink pull request #6141: flink-metrics-datadog: beautify metric name by exc...
Github user DmitryBe closed the pull request at: https://github.com/apache/flink/pull/6141 ---
[jira] [Commented] (FLINK-9367) Truncate() in BucketingSink is only allowed after hadoop2.7
[ https://issues.apache.org/jira/browse/FLINK-9367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16507641#comment-16507641 ] ASF GitHub Bot commented on FLINK-9367: --- Github user zhangxinyu1 commented on the issue: https://github.com/apache/flink/pull/6108 @kl0u Thanks. Would you please consider to implement a BucketingSource which we can use it to read data from FileSystems? Besides we also care about the limit of bandwidth. > Truncate() in BucketingSink is only allowed after hadoop2.7 > --- > > Key: FLINK-9367 > URL: https://issues.apache.org/jira/browse/FLINK-9367 > Project: Flink > Issue Type: Improvement > Components: Streaming Connectors >Affects Versions: 1.5.0 >Reporter: zhangxinyu >Priority: Major > > When output to HDFS using BucketingSink, truncate() is only allowed after > hadoop2.7. > If some tasks failed, the ".valid-length" file is created for the lower > version hadoop. > The problem is, if other people want to use the data in HDFS, they must know > how to deal with the ".valid-length" file, otherwise, the data may be not > exactly-once. > I think it's not convenient for other people to use the data. Why not just > read the in-progress file and write a new file when restoring instead of > writing a ".valid-length" file. > In this way, others who use the data in HDFS don't need to know how to deal > with the ".valid-length" file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] flink issue #6108: [FLINK-9367] [Streaming Connectors] Allow to do truncate(...
Github user zhangxinyu1 commented on the issue: https://github.com/apache/flink/pull/6108 @kl0u Thanks. Would you please consider to implement a BucketingSource which we can use it to read data from FileSystems? Besides we also care about the limit of bandwidth. ---
[jira] [Closed] (FLINK-8256) Cannot use Scala functions to filter in 1.4 - java.lang.ClassCastException
[
https://issues.apache.org/jira/browse/FLINK-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabian Hueske closed FLINK-8256.
Resolution: Not A Bug
> Cannot use Scala functions to filter in 1.4 - java.lang.ClassCastException
> --
>
> Key: FLINK-8256
> URL: https://issues.apache.org/jira/browse/FLINK-8256
> Project: Flink
> Issue Type: Bug
> Components: DataStream API
>Affects Versions: 1.4.0
> Environment: macOS, Local Flink v1.4.0, Scala 2.11
>Reporter: Ryan Brideau
>Priority: Major
>
> I built the newest release locally today, but when I try to filter a stream
> using an anonymous or named function, I get an error. Here's a simple example:
> {code:java}
> import org.apache.flink.api.java.utils.ParameterTool
> import org.apache.flink.streaming.api.scala._
> object TestFunction {
> def main(args: Array[String]): Unit = {
> val env = StreamExecutionEnvironment.getExecutionEnvironment
> val params = ParameterTool.fromArgs(args)
> env.getConfig.setGlobalJobParameters(params)
> val someArray = Array(1,2,3)
> val stream = env.fromCollection(someArray).filter(_ => true)
> stream.print().setParallelism(1)
> env.execute("Testing Function")
> }
> }
> {code}
> This results in:
> {code:java}
> Job execution switched to status FAILING.
> org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot
> instantiate user function.
> at
> org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:235)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain.createChainedOperator(OperatorChain.java:355)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain.createOutputCollector(OperatorChain.java:282)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:126)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:231)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassCastException: cannot assign instance of
> org.peopleinmotion.TestFunction$$anonfun$1 to field
> org.apache.flink.streaming.api.scala.DataStream$$anon$7.cleanFun$6 of type
> scala.Function1 in instance of
> org.apache.flink.streaming.api.scala.DataStream$$anon$7
> at
> java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2233)
> at
> java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1405)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2288)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2206)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2064)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1568)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2282)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2206)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2064)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1568)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:428)
> at
> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:290)
> at
> org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:248)
> at
> org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:220)
> ... 6 more
> 12/13/2017 15:10:01 Job execution switched to status FAILED.
>
> The program finished with the following exception:
> org.apache.flink.client.program.ProgramInvocationException: The program
> execution failed: Job execution failed.
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:492)
> at
> org.apache.flink.client.program.StandaloneClusterClient.submitJob(StandaloneClusterClient.java:105)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:456)
> at
> org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:66)
> at
> org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.scala:638)
> at org.peopleinmotion.TestFunction$.main(TestFunction.scala:20)
> at org.peopleinmotion.TestFunction.main(TestFunction.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
[jira] [Closed] (FLINK-6977) Add MD5/SHA1/SHA2 supported in TableAPI
[ https://issues.apache.org/jira/browse/FLINK-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabian Hueske closed FLINK-6977. Resolution: Implemented Fixed in 1.6.0: 5544ab5df5d805511a40f2e4007b3afee06d15af & d5de2bcf122fd931080ac7a9edc704c93dbe15ee > Add MD5/SHA1/SHA2 supported in TableAPI > --- > > Key: FLINK-6977 > URL: https://issues.apache.org/jira/browse/FLINK-6977 > Project: Flink > Issue Type: Sub-task > Components: Table API SQL >Affects Versions: 1.4.0 >Reporter: sunjincheng >Assignee: Alexandr Arkhipov >Priority: Major > Labels: starter > > See FLINK-6926 for detail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-6977) Add MD5/SHA1/SHA2 supported in TableAPI
[ https://issues.apache.org/jira/browse/FLINK-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16507474#comment-16507474 ] Fabian Hueske commented on FLINK-6977: -- [~alex.arkhipov], you're right. Thanks for the notice! > Add MD5/SHA1/SHA2 supported in TableAPI > --- > > Key: FLINK-6977 > URL: https://issues.apache.org/jira/browse/FLINK-6977 > Project: Flink > Issue Type: Sub-task > Components: Table API SQL >Affects Versions: 1.4.0 >Reporter: sunjincheng >Assignee: Alexandr Arkhipov >Priority: Major > Labels: starter > > See FLINK-6926 for detail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8861) Add support for batch queries in SQL Client
[
https://issues.apache.org/jira/browse/FLINK-8861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16507318#comment-16507318
]
ASF GitHub Bot commented on FLINK-8861:
---
Github user xccui commented on the issue:
https://github.com/apache/flink/pull/5660
Hi @twalthr, the PR has been reworked. Take a look when you are convenient.
Thanks!
> Add support for batch queries in SQL Client
> ---
>
> Key: FLINK-8861
> URL: https://issues.apache.org/jira/browse/FLINK-8861
> Project: Flink
> Issue Type: Sub-task
> Components: Table API SQL
>Reporter: Timo Walther
>Assignee: Xingcan Cui
>Priority: Major
>
> This issue is a subtask of part two "Full Embedded SQL Client" of the
> implementation plan mentioned in
> [FLIP-24|https://cwiki.apache.org/confluence/display/FLINK/FLIP-24+-+SQL+Client].
> Similar to streaming queries, it should be possible to execute batch queries
> in the SQL Client and collect the results using {{DataSet.collect()}} for
> debugging purposes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] flink issue #5660: [FLINK-8861] [table] Add support for batch queries in SQL...
Github user xccui commented on the issue: https://github.com/apache/flink/pull/5660 Hi @twalthr, the PR has been reworked. Take a look when you are convenient. Thanks! ---
[jira] [Commented] (FLINK-8861) Add support for batch queries in SQL Client
[
https://issues.apache.org/jira/browse/FLINK-8861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16507317#comment-16507317
]
ASF GitHub Bot commented on FLINK-8861:
---
Github user xccui commented on a diff in the pull request:
https://github.com/apache/flink/pull/5660#discussion_r194250975
--- Diff:
flink-libraries/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/CliTableResultView.java
---
@@ -159,6 +160,9 @@ protected void evaluate(ResultTableOperation operation,
String binding) {
case PREV:
gotoPreviousPage();
break;
+ case FIRST:
--- End diff --
From the function's point of view, that's true. I added it just for
symmetry .
> Add support for batch queries in SQL Client
> ---
>
> Key: FLINK-8861
> URL: https://issues.apache.org/jira/browse/FLINK-8861
> Project: Flink
> Issue Type: Sub-task
> Components: Table API SQL
>Reporter: Timo Walther
>Assignee: Xingcan Cui
>Priority: Major
>
> This issue is a subtask of part two "Full Embedded SQL Client" of the
> implementation plan mentioned in
> [FLIP-24|https://cwiki.apache.org/confluence/display/FLINK/FLIP-24+-+SQL+Client].
> Similar to streaming queries, it should be possible to execute batch queries
> in the SQL Client and collect the results using {{DataSet.collect()}} for
> debugging purposes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] flink pull request #5660: [FLINK-8861] [table] Add support for batch queries...
Github user xccui commented on a diff in the pull request:
https://github.com/apache/flink/pull/5660#discussion_r194250975
--- Diff:
flink-libraries/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/CliTableResultView.java
---
@@ -159,6 +160,9 @@ protected void evaluate(ResultTableOperation operation,
String binding) {
case PREV:
gotoPreviousPage();
break;
+ case FIRST:
--- End diff --
From the function's point of view, that's true. I added it just for
symmetry ð.
---
