[GitHub] flink pull request:
Github user zentol commented on the pull request: https://github.com/apache/flink/commit/707606ac40dbbbd497fcbbb5442870fec5468bf3#commitcomment-17574494 In flink-core/src/main/java/org/apache/flink/metrics/groups/JobMetricGroup.java: In flink-core/src/main/java/org/apache/flink/metrics/groups/JobMetricGroup.java on line 97: this can produce wrong metrics for the Batch API. Metrics directly associated with a job will be reset if at any point a given TM has no tasks for a job. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-3939) Prevent distinct aggregates and grouping sets from being translated
[
https://issues.apache.org/jira/browse/FLINK-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295814#comment-15295814
]
ASF GitHub Bot commented on FLINK-3939:
---
Github user yjshen commented on a diff in the pull request:
https://github.com/apache/flink/pull/2014#discussion_r64160758
--- Diff:
flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/plan/rules/dataSet/DataSetAggregateRule.scala
---
@@ -33,20 +33,32 @@ class DataSetAggregateRule
"DataSetAggregateRule")
{
-def convert(rel: RelNode): RelNode = {
- val agg: LogicalAggregate = rel.asInstanceOf[LogicalAggregate]
- val traitSet: RelTraitSet =
rel.getTraitSet.replace(DataSetConvention.INSTANCE)
- val convInput: RelNode = RelOptRule.convert(agg.getInput,
DataSetConvention.INSTANCE)
-
- new DataSetAggregate(
-rel.getCluster,
-traitSet,
-convInput,
-agg.getNamedAggCalls,
-rel.getRowType,
-agg.getInput.getRowType,
-agg.getGroupSet.toArray)
- }
+ override def matches(call: RelOptRuleCall): Boolean = {
--- End diff --
Is that possible we add an `UnsupportedOperatorRule` to match unsupported
plan and throws an `TableException` on `convert` method?
> Prevent distinct aggregates and grouping sets from being translated
> ---
>
> Key: FLINK-3939
> URL: https://issues.apache.org/jira/browse/FLINK-3939
> Project: Flink
> Issue Type: Bug
> Components: Table API
>Reporter: Fabian Hueske
>Assignee: Fabian Hueske
> Fix For: 1.1.0
>
>
> Flink's SQL interface is currently not capable of executing distinct
> aggregates and grouping sets.
> We need to prevent that queries with these operations are translated by
> adapting the DataSetAggregateRule.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request: [FLINK-3939] [tableAPI] Prevent translation of...
Github user yjshen commented on a diff in the pull request:
https://github.com/apache/flink/pull/2014#discussion_r64160758
--- Diff:
flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/plan/rules/dataSet/DataSetAggregateRule.scala
---
@@ -33,20 +33,32 @@ class DataSetAggregateRule
"DataSetAggregateRule")
{
-def convert(rel: RelNode): RelNode = {
- val agg: LogicalAggregate = rel.asInstanceOf[LogicalAggregate]
- val traitSet: RelTraitSet =
rel.getTraitSet.replace(DataSetConvention.INSTANCE)
- val convInput: RelNode = RelOptRule.convert(agg.getInput,
DataSetConvention.INSTANCE)
-
- new DataSetAggregate(
-rel.getCluster,
-traitSet,
-convInput,
-agg.getNamedAggCalls,
-rel.getRowType,
-agg.getInput.getRowType,
-agg.getGroupSet.toArray)
- }
+ override def matches(call: RelOptRuleCall): Boolean = {
--- End diff --
Is that possible we add an `UnsupportedOperatorRule` to match unsupported
plan and throws an `TableException` on `convert` method?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (FLINK-3586) Risk of data overflow while use sum/count to calculate AVG value
[
https://issues.apache.org/jira/browse/FLINK-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295804#comment-15295804
]
Chengxiang Li commented on FLINK-3586:
--
Not at all, feel free to take over it, Fabian.
> Risk of data overflow while use sum/count to calculate AVG value
>
>
> Key: FLINK-3586
> URL: https://issues.apache.org/jira/browse/FLINK-3586
> Project: Flink
> Issue Type: Sub-task
> Components: Table API
>Reporter: Chengxiang Li
>Assignee: Fabian Hueske
>Priority: Minor
>
> Now, we use {{(sum: Long, count: Long}} to store AVG partial aggregate data,
> which may have data overflow risk, we should use unbounded data type(such as
> BigInteger) to store them for necessary data types.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLINK-3952) Bump Netty to 4.1
[ https://issues.apache.org/jira/browse/FLINK-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rektide de la fey updated FLINK-3952: - Description: Netty 4.1 is about to release final. This release has [a number of significant enhancements|http://netty.io/wiki/new-and-noteworthy-in-4.1.html], and in particular I find HTTP/2 codecs to be incredibly desirable to have. Additionally, hopefully, the [Hadoop patches for Netty 4.1|https://issues.apache.org/jira/browse/HADOOP-11716] get some tests and get merged, & I believe if/when that happens it'll be important for Flink to also be using the new Netty minor version. was: Netty 4.1 is about to release final. This release has [a number of significant enhancements|http://netty.io/wiki/new-and-noteworthy-in-4.1.html], and in particular I find HTTP/2 codecs to be incredibly desirable to have. Additionally, hopefully, the [Hadoop patches for Netty 4.1](https://issues.apache.org/jira/browse/HADOOP-11716) get some tests and get merged, & I believe if/when that happens it'll be important for Flink to also be using the new Netty minor version. > Bump Netty to 4.1 > - > > Key: FLINK-3952 > URL: https://issues.apache.org/jira/browse/FLINK-3952 > Project: Flink > Issue Type: Improvement > Components: Core, Hadoop Compatibility >Reporter: rektide de la fey > Labels: netty > > Netty 4.1 is about to release final. This release has [a number of > significant > enhancements|http://netty.io/wiki/new-and-noteworthy-in-4.1.html], and in > particular I find HTTP/2 codecs to be incredibly desirable to have. > Additionally, hopefully, the [Hadoop patches for Netty > 4.1|https://issues.apache.org/jira/browse/HADOOP-11716] get some tests and > get merged, & I believe if/when that happens it'll be important for Flink to > also be using the new Netty minor version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-3952) Bump Netty to 4.1
rektide de la fey created FLINK-3952: Summary: Bump Netty to 4.1 Key: FLINK-3952 URL: https://issues.apache.org/jira/browse/FLINK-3952 Project: Flink Issue Type: Improvement Components: Core, Hadoop Compatibility Reporter: rektide de la fey Netty 4.1 is about to release final. This release has [a number of significant enhancements|http://netty.io/wiki/new-and-noteworthy-in-4.1.html], and in particular I find HTTP/2 codecs to be incredibly desirable to have. Additionally, hopefully, the [Hadoop patches for Netty 4.1](https://issues.apache.org/jira/browse/HADOOP-11716) get some tests and get merged, & I believe if/when that happens it'll be important for Flink to also be using the new Netty minor version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-3702) DataStream API PojoFieldAccessor doesn't support nested POJOs
[
https://issues.apache.org/jira/browse/FLINK-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295698#comment-15295698
]
Gabor Gevay commented on FLINK-3702:
This turned out to be more complicated than I thought, but I'm 80% done. I'll
try to wrap it up in the next few days.
I'm refactoring FieldAccessor, so that it will be created from a method of the
TypeInfos, instead of the static method FieldAccessor.create. This makes it
possible to process the outermost part of a field expression and then recurse
into the specified field's TypeInfo with the rest of the field expression.
> DataStream API PojoFieldAccessor doesn't support nested POJOs
> -
>
> Key: FLINK-3702
> URL: https://issues.apache.org/jira/browse/FLINK-3702
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
>Affects Versions: 1.0.0
>Reporter: Robert Metzger
>Assignee: Gabor Gevay
>
> The {{PojoFieldAccessor}} (which is used by {{.sum(String)}} and similar
> methods) doesn't support nested POJOs right now.
> As part of FLINK-3697 I'll add a check for a nested POJO and fail with an
> exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-3660) Measure latency of elements and expose it through web interface
[
https://issues.apache.org/jira/browse/FLINK-3660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295688#comment-15295688
]
Stephan Ewen commented on FLINK-3660:
-
I would break this into two tasks:
- measure latency and exposes them via a metric
- visualize that metric in the Web Dashboard
> Measure latency of elements and expose it through web interface
> ---
>
> Key: FLINK-3660
> URL: https://issues.apache.org/jira/browse/FLINK-3660
> Project: Flink
> Issue Type: Sub-task
> Components: Streaming
>Reporter: Robert Metzger
>Assignee: Robert Metzger
> Fix For: pre-apache
>
>
> It would be nice to expose the end-to-end latency of a streaming job in the
> webinterface.
> To achieve this, my initial thought was to attach an ingestion-time timestamp
> at the sources to each record.
> However, this introduces overhead for a monitoring feature users might not
> even use (8 bytes for each element + System.currentTimeMilis() on each
> element).
> Therefore, I suggest to implement this feature by periodically sending
> special events, similar to watermarks through the topology.
> Those {{LatencyMarks}} are emitted at a configurable interval at the sources
> and forwarded by the tasks. The sinks will compare the timestamp of the
> latency marks with their current system time to determine the latency.
> The latency marks will not add to the latency of a job, but the marks will be
> delayed similarly than regular records, so their latency will approximate the
> record latency.
> Above suggestion expects the clocks on all taskmanagers to be in sync.
> Otherwise, the measured latencies would also include the offsets between the
> taskmanager's clocks.
> In a second step, we can try to mitigate the issue by using the JobManager as
> a central timing service. The TaskManagers will periodically query the JM for
> the current time in order to determine the offset with their clock.
> This offset would still include the network latency between TM and JM but it
> would still lead to reasonably good estimations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (FLINK-3951) Add Histogram Metric Type
Stephan Ewen created FLINK-3951: --- Summary: Add Histogram Metric Type Key: FLINK-3951 URL: https://issues.apache.org/jira/browse/FLINK-3951 Project: Flink Issue Type: Sub-task Components: Core Reporter: Stephan Ewen Fix For: 1.1.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-3950) Add Meter Metric Type
Stephan Ewen created FLINK-3950: --- Summary: Add Meter Metric Type Key: FLINK-3950 URL: https://issues.apache.org/jira/browse/FLINK-3950 Project: Flink Issue Type: Sub-task Components: Core Reporter: Stephan Ewen Fix For: 1.1.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-3949) Collect Metrics in Runtime Operators
Stephan Ewen created FLINK-3949: --- Summary: Collect Metrics in Runtime Operators Key: FLINK-3949 URL: https://issues.apache.org/jira/browse/FLINK-3949 Project: Flink Issue Type: Sub-task Components: Local Runtime Reporter: Stephan Ewen Fix For: 1.1.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (FLINK-1502) Expose metrics to graphite, ganglia and JMX.
[ https://issues.apache.org/jira/browse/FLINK-1502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-1502. --- > Expose metrics to graphite, ganglia and JMX. > > > Key: FLINK-1502 > URL: https://issues.apache.org/jira/browse/FLINK-1502 > Project: Flink > Issue Type: Sub-task > Components: JobManager, TaskManager >Affects Versions: 0.9 >Reporter: Robert Metzger >Assignee: Chesnay Schepler >Priority: Minor > Fix For: 1.1.0 > > > The metrics library allows to expose collected metrics easily to other > systems such as graphite, ganglia or Java's JVM (VisualVM). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLINK-1502) Expose metrics to graphite, ganglia and JMX.
[ https://issues.apache.org/jira/browse/FLINK-1502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-1502. - Resolution: Fixed Fix Version/s: (was: pre-apache) 1.1.0 Fixed via 003ce18efc0249fae874e56c3df6acf19f5f2429 and 707606ac40dbbbd497fcbbb5442870fec5468bf3 Apache Git - https://git-wip-us.apache.org/repos/asf?p=flink.git;a=commit;h=003ce18efc0249fae874e56c3df6acf19f5f2429 - https://git-wip-us.apache.org/repos/asf?p=flink.git;a=commit;h=707606ac40dbbbd497fcbbb5442870fec5468bf3 GitHub - https://github.com/apache/flink/commit/003ce18efc0249fae874e56c3df6acf19f5f2429 - https://github.com/apache/flink/commit/707606ac40dbbbd497fcbbb5442870fec5468bf3 > Expose metrics to graphite, ganglia and JMX. > > > Key: FLINK-1502 > URL: https://issues.apache.org/jira/browse/FLINK-1502 > Project: Flink > Issue Type: Sub-task > Components: JobManager, TaskManager >Affects Versions: 0.9 >Reporter: Robert Metzger >Assignee: Chesnay Schepler >Priority: Minor > Fix For: 1.1.0 > > > The metrics library allows to expose collected metrics easily to other > systems such as graphite, ganglia or Java's JVM (VisualVM). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1502) Expose metrics to graphite, ganglia and JMX.
[ https://issues.apache.org/jira/browse/FLINK-1502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295685#comment-15295685 ] ASF GitHub Bot commented on FLINK-1502: --- Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1947#issuecomment-220850557 I did a pass over the code and committed the result: Manually merged in 003ce18efc0249fae874e56c3df6acf19f5f2429 and 707606ac40dbbbd497fcbbb5442870fec5468bf3 > Expose metrics to graphite, ganglia and JMX. > > > Key: FLINK-1502 > URL: https://issues.apache.org/jira/browse/FLINK-1502 > Project: Flink > Issue Type: Sub-task > Components: JobManager, TaskManager >Affects Versions: 0.9 >Reporter: Robert Metzger >Assignee: Chesnay Schepler >Priority: Minor > Fix For: pre-apache > > > The metrics library allows to expose collected metrics easily to other > systems such as graphite, ganglia or Java's JVM (VisualVM). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1502] [WIP] Basic Metric System
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1947#issuecomment-220850557 I did a pass over the code and committed the result: Manually merged in 003ce18efc0249fae874e56c3df6acf19f5f2429 and 707606ac40dbbbd497fcbbb5442870fec5468bf3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (FLINK-3702) DataStream API PojoFieldAccessor doesn't support nested POJOs
[
https://issues.apache.org/jira/browse/FLINK-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Gevay reassigned FLINK-3702:
--
Assignee: Gabor Gevay
> DataStream API PojoFieldAccessor doesn't support nested POJOs
> -
>
> Key: FLINK-3702
> URL: https://issues.apache.org/jira/browse/FLINK-3702
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
>Affects Versions: 1.0.0
>Reporter: Robert Metzger
>Assignee: Gabor Gevay
>
> The {{PojoFieldAccessor}} (which is used by {{.sum(String)}} and similar
> methods) doesn't support nested POJOs right now.
> As part of FLINK-3697 I'll add a check for a nested POJO and fail with an
> exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-3702) DataStream API PojoFieldAccessor doesn't support nested POJOs
[
https://issues.apache.org/jira/browse/FLINK-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295621#comment-15295621
]
Gabor Gevay commented on FLINK-3702:
I'll try to quickly do it now.
> DataStream API PojoFieldAccessor doesn't support nested POJOs
> -
>
> Key: FLINK-3702
> URL: https://issues.apache.org/jira/browse/FLINK-3702
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
>Affects Versions: 1.0.0
>Reporter: Robert Metzger
>
> The {{PojoFieldAccessor}} (which is used by {{.sum(String)}} and similar
> methods) doesn't support nested POJOs right now.
> As part of FLINK-3697 I'll add a check for a nested POJO and fail with an
> exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request: [FLINK-3728] [tableAPI] Improve error message ...
GitHub user fhueske opened a pull request:
https://github.com/apache/flink/pull/2018
[FLINK-3728] [tableAPI] Improve error message and documentation for
unsupported SQL features.
This PR
1. extends the Table API / SQL documentation and lists known limitations.
2. improves the error message in case Calcite cannot fails to optimize the
program because it includes unsupported features.
- [X] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira
title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the
JIRA id)
- [X] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [X] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis
build has passed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fhueske/flink tableSupportedFeatures
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2018.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2018
commit bd8c4c94a220d57bfcfd3b4a6a52334f1e1940bd
Author: Fabian Hueske
Date: 2016-05-22T10:43:19Z
[FLINK-3728] [tableAPI] Improve error message and documentation for
unsupported SQL features.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (FLINK-3728) Throw meaningful exceptions for unsupported SQL features
[
https://issues.apache.org/jira/browse/FLINK-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295608#comment-15295608
]
ASF GitHub Bot commented on FLINK-3728:
---
GitHub user fhueske opened a pull request:
https://github.com/apache/flink/pull/2018
[FLINK-3728] [tableAPI] Improve error message and documentation for
unsupported SQL features.
This PR
1. extends the Table API / SQL documentation and lists known limitations.
2. improves the error message in case Calcite cannot fails to optimize the
program because it includes unsupported features.
- [X] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira
title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the
JIRA id)
- [X] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [X] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis
build has passed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fhueske/flink tableSupportedFeatures
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2018.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2018
commit bd8c4c94a220d57bfcfd3b4a6a52334f1e1940bd
Author: Fabian Hueske
Date: 2016-05-22T10:43:19Z
[FLINK-3728] [tableAPI] Improve error message and documentation for
unsupported SQL features.
> Throw meaningful exceptions for unsupported SQL features
>
>
> Key: FLINK-3728
> URL: https://issues.apache.org/jira/browse/FLINK-3728
> Project: Flink
> Issue Type: Bug
> Components: Table API
>Affects Versions: 1.1.0
>Reporter: Vasia Kalavri
>Assignee: Fabian Hueske
>
> We must explicitly exclude unsupported SQL features such as Grouping Sets
> from being translated to Flink programs.
> Otherwise, the resulting program will compute invalid results.
> For that we must restrict the Calcite rules that translate Logical
> {{RelNodes}} into {{DataSetRel}} or {{DataStreamRel}} nodes.
> We may only translate to {{DataSetRel}} or {{DataStreamRel}} nodes if these
> support the semantics of the {{RelNode}}.
> Not translating a {{RelNode}} will yield a Calcite {{CannotPlanException}}
> that we should catch and enrich with a meaningful error message.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (FLINK-3586) Risk of data overflow while use sum/count to calculate AVG value
[
https://issues.apache.org/jira/browse/FLINK-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabian Hueske reassigned FLINK-3586:
Assignee: Fabian Hueske (was: Chengxiang Li)
> Risk of data overflow while use sum/count to calculate AVG value
>
>
> Key: FLINK-3586
> URL: https://issues.apache.org/jira/browse/FLINK-3586
> Project: Flink
> Issue Type: Sub-task
> Components: Table API
>Reporter: Chengxiang Li
>Assignee: Fabian Hueske
>Priority: Minor
>
> Now, we use {{(sum: Long, count: Long}} to store AVG partial aggregate data,
> which may have data overflow risk, we should use unbounded data type(such as
> BigInteger) to store them for necessary data types.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-3586) Risk of data overflow while use sum/count to calculate AVG value
[
https://issues.apache.org/jira/browse/FLINK-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295591#comment-15295591
]
Fabian Hueske commented on FLINK-3586:
--
I have a fix for this issue and would like to take it over. Please let me know
if that's not OK with you, [~chengxiang li]. Thanks!
> Risk of data overflow while use sum/count to calculate AVG value
>
>
> Key: FLINK-3586
> URL: https://issues.apache.org/jira/browse/FLINK-3586
> Project: Flink
> Issue Type: Sub-task
> Components: Table API
>Reporter: Chengxiang Li
>Assignee: Chengxiang Li
>Priority: Minor
>
> Now, we use {{(sum: Long, count: Long}} to store AVG partial aggregate data,
> which may have data overflow risk, we should use unbounded data type(such as
> BigInteger) to store them for necessary data types.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2673) Scala API does not support Option type as key
[
https://issues.apache.org/jira/browse/FLINK-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295571#comment-15295571
]
ASF GitHub Bot commented on FLINK-2673:
---
Github user chiwanpark commented on the pull request:
https://github.com/apache/flink/pull/2017#issuecomment-220833561
Failing Travis is not related to this PR.
> Scala API does not support Option type as key
> -
>
> Key: FLINK-2673
> URL: https://issues.apache.org/jira/browse/FLINK-2673
> Project: Flink
> Issue Type: Wish
> Components: Scala API
>Reporter: Till Rohrmann
>Assignee: Chiwan Park
>Priority: Minor
>
> The Scala API does not support the {{Option}} type as a key. It could be
> useful to allow grouping on a field with this type.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request: [FLINK-2673] [type system] Add a comparator fo...
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/2017#issuecomment-220833561 Failing Travis is not related to this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (FLINK-3936) Add MIN / MAX aggregations function for BOOLEAN types
[
https://issues.apache.org/jira/browse/FLINK-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabian Hueske reassigned FLINK-3936:
Assignee: Fabian Hueske
> Add MIN / MAX aggregations function for BOOLEAN types
> -
>
> Key: FLINK-3936
> URL: https://issues.apache.org/jira/browse/FLINK-3936
> Project: Flink
> Issue Type: Improvement
> Components: Table API
>Affects Versions: 1.1.0
>Reporter: Fabian Hueske
>Assignee: Fabian Hueske
> Fix For: 1.1.0
>
>
> When executing TPC-H Q4, I observed that Calcite generates a MIN aggregate on
> Boolean literals to translate a decorrelate subquery in an {{EXIST}}
> predicate.
> MIN and MAX aggregates on Boolean data types are currently not supported and
> should be added.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (FLINK-3728) Throw meaningful exceptions for unsupported SQL features
[
https://issues.apache.org/jira/browse/FLINK-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabian Hueske reassigned FLINK-3728:
Assignee: Fabian Hueske
> Throw meaningful exceptions for unsupported SQL features
>
>
> Key: FLINK-3728
> URL: https://issues.apache.org/jira/browse/FLINK-3728
> Project: Flink
> Issue Type: Bug
> Components: Table API
>Affects Versions: 1.1.0
>Reporter: Vasia Kalavri
>Assignee: Fabian Hueske
>
> We must explicitly exclude unsupported SQL features such as Grouping Sets
> from being translated to Flink programs.
> Otherwise, the resulting program will compute invalid results.
> For that we must restrict the Calcite rules that translate Logical
> {{RelNodes}} into {{DataSetRel}} or {{DataStreamRel}} nodes.
> We may only translate to {{DataSetRel}} or {{DataStreamRel}} nodes if these
> support the semantics of the {{RelNode}}.
> Not translating a {{RelNode}} will yield a Calcite {{CannotPlanException}}
> that we should catch and enrich with a meaningful error message.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request: [FLINK-3632][TableAPI]Clean up Table API excep...
Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/2015#issuecomment-220829100 Thanks for the PR @yjshen. The changes look good. +1 to merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-3632) Clean up Table API exceptions
[
https://issues.apache.org/jira/browse/FLINK-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295539#comment-15295539
]
ASF GitHub Bot commented on FLINK-3632:
---
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/2015#issuecomment-220829100
Thanks for the PR @yjshen. The changes look good.
+1 to merge
> Clean up Table API exceptions
> -
>
> Key: FLINK-3632
> URL: https://issues.apache.org/jira/browse/FLINK-3632
> Project: Flink
> Issue Type: Improvement
> Components: Table API
>Reporter: Fabian Hueske
>Assignee: Yijie Shen
> Fix For: 1.1.0
>
>
> The Table API throws many different exception types including:
> - {{IllegalArgumentException}}
> - {{TableException}}
> - {{CodeGenException}}
> - {{PlanGenException}}
> - {{ExpressionParserException}}
> from various places of the query translation code.
> This needs to be cleaned up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2673) Scala API does not support Option type as key
[
https://issues.apache.org/jira/browse/FLINK-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295526#comment-15295526
]
ASF GitHub Bot commented on FLINK-2673:
---
GitHub user chiwanpark opened a pull request:
https://github.com/apache/flink/pull/2017
[FLINK-2673] [type system] Add a comparator for Scala Option type
Thanks for contributing to Apache Flink. Before you open your pull request,
please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your
pull request. For more information and/or questions please refer to the [How To
Contribute guide](http://flink.apache.org/how-to-contribute.html).
In addition to going through the list, please provide a meaningful
description of your changes.
- [x] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira
title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the
JIRA id)
- [ ] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [x] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis
build has passed
---
I've added `OptionTypeComparator` implementation, unit test cases based on
`ComparatorTestBase` and a simple integration test case in `JoinITCase`. I
followed [Till's opinion](http://markmail.org/message/vk3t3vpc6j6zbldk) about
order of `None` and `Some` values.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chiwanpark/flink FLINK-2673
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2017.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2017
commit 485b6904013c23316a9f61d651b44e245e47c61c
Author: Chiwan Park
Date: 2016-05-22T11:39:10Z
[FLINK-2673] [core] Add a comparator for Scala Option type
> Scala API does not support Option type as key
> -
>
> Key: FLINK-2673
> URL: https://issues.apache.org/jira/browse/FLINK-2673
> Project: Flink
> Issue Type: Wish
> Components: Scala API
>Reporter: Till Rohrmann
>Assignee: Chiwan Park
>Priority: Minor
>
> The Scala API does not support the {{Option}} type as a key. It could be
> useful to allow grouping on a field with this type.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request: [FLINK-2673] [type system] Add a comparator fo...
GitHub user chiwanpark opened a pull request:
https://github.com/apache/flink/pull/2017
[FLINK-2673] [type system] Add a comparator for Scala Option type
Thanks for contributing to Apache Flink. Before you open your pull request,
please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your
pull request. For more information and/or questions please refer to the [How To
Contribute guide](http://flink.apache.org/how-to-contribute.html).
In addition to going through the list, please provide a meaningful
description of your changes.
- [x] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira
title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the
JIRA id)
- [ ] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [x] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis
build has passed
---
I've added `OptionTypeComparator` implementation, unit test cases based on
`ComparatorTestBase` and a simple integration test case in `JoinITCase`. I
followed [Till's opinion](http://markmail.org/message/vk3t3vpc6j6zbldk) about
order of `None` and `Some` values.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chiwanpark/flink FLINK-2673
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2017.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2017
commit 485b6904013c23316a9f61d651b44e245e47c61c
Author: Chiwan Park
Date: 2016-05-22T11:39:10Z
[FLINK-2673] [core] Add a comparator for Scala Option type
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (FLINK-3702) DataStream API PojoFieldAccessor doesn't support nested POJOs
[
https://issues.apache.org/jira/browse/FLINK-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295502#comment-15295502
]
Rami commented on FLINK-3702:
-
This feature would really make a difference in our code. most of the events we
are having needs count, and many of those events share the nested rawevent
pojo, if we could have the count field only in the nested rawevent pojo , it
would be much cleaner code and easier to understand rather than having the
count field duplicated in the event and it's nested rawevent. Any ETA?
> DataStream API PojoFieldAccessor doesn't support nested POJOs
> -
>
> Key: FLINK-3702
> URL: https://issues.apache.org/jira/browse/FLINK-3702
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
>Affects Versions: 1.0.0
>Reporter: Robert Metzger
>
> The {{PojoFieldAccessor}} (which is used by {{.sum(String)}} and similar
> methods) doesn't support nested POJOs right now.
> As part of FLINK-3697 I'll add a check for a nested POJO and fail with an
> exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
