[jira] [Created] (FLINK-22278) Refactor sql client's DynamicResult
Kurt Young created FLINK-22278: -- Summary: Refactor sql client's DynamicResult Key: FLINK-22278 URL: https://issues.apache.org/jira/browse/FLINK-22278 Project: Flink Issue Type: Improvement Components: Table SQL / Client Reporter: Kurt Young We can simplify the design around sql client's Executor and DynamicResult, by reducing the responsibility of Executor when retrieving SELECT result. Page related logic should be handled by different CliResultViews instead of Executor and different DynamicResults. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22277) Refactor sql client's DynamicResult
Kurt Young created FLINK-22277: -- Summary: Refactor sql client's DynamicResult Key: FLINK-22277 URL: https://issues.apache.org/jira/browse/FLINK-22277 Project: Flink Issue Type: Improvement Components: Table SQL / Client Reporter: Kurt Young Right now, the sql client Executor has different result handling logic for different result display mode. Different result is handled by child classes of `DynamicResult`. This leads to introducing page related API to the Executor, such as `snapshotResult` and `retrieveResultPage`, which I think is inappropriate and will make things complicated. It will be benefit to simplify the responsibility of Executor about retrieving results to simply streaming back the SELECT result, and move the logic of dealing with different display mode into CliResultView. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22178) Support ignore-first-line option in new csv format
Kurt Young created FLINK-22178: -- Summary: Support ignore-first-line option in new csv format Key: FLINK-22178 URL: https://issues.apache.org/jira/browse/FLINK-22178 Project: Flink Issue Type: New Feature Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile) Affects Versions: 1.13.0 Reporter: Kurt Young -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-21236) Don't explicitly use HeapMemorySegment in row format serde
Kurt Young created FLINK-21236: -- Summary: Don't explicitly use HeapMemorySegment in row format serde Key: FLINK-21236 URL: https://issues.apache.org/jira/browse/FLINK-21236 Project: Flink Issue Type: Improvement Components: Table SQL / Planner Affects Versions: 1.12.0 Reporter: Kurt Young Fix For: 1.13.0 `RawFormatDeserializationSchema` and `RawFormatSerializationSchema` explicitly used `HeapMemorySegment`, and in a typical batch job, `HybridMemorySegment` will also be loaded and used as managed memory. This will prevent Class Hierarchy Analysis (CHA) to optimize the function call of MemorySegment. More details can be found here: [https://flink.apache.org/news/2015/09/16/off-heap-memory.html] We can use `ByteBuffer` instead of `HeapMemorySegment`. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-18224) Add document about sql client's tableau result mode
Kurt Young created FLINK-18224: -- Summary: Add document about sql client's tableau result mode Key: FLINK-18224 URL: https://issues.apache.org/jira/browse/FLINK-18224 Project: Flink Issue Type: Task Components: Documentation, Table SQL / Client Reporter: Kurt Young Assignee: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17756) Drop table/view shouldn't take affect on each other
Kurt Young created FLINK-17756: -- Summary: Drop table/view shouldn't take affect on each other Key: FLINK-17756 URL: https://issues.apache.org/jira/browse/FLINK-17756 Project: Flink Issue Type: Bug Components: Table SQL / API Reporter: Kurt Young Assignee: Kurt Young Fix For: 1.11.0 Currently "DROP VIEW" can successfully drop a table, and "DROP TABLE" can successfully a view. We should disable this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17749) Remove fromTableSource method from TableEnvironment
Kurt Young created FLINK-17749: -- Summary: Remove fromTableSource method from TableEnvironment Key: FLINK-17749 URL: https://issues.apache.org/jira/browse/FLINK-17749 Project: Flink Issue Type: Sub-task Components: Table SQL / API Reporter: Kurt Young Assignee: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17748) Remove registration of TableSource/TableSink in Table Env
Kurt Young created FLINK-17748: -- Summary: Remove registration of TableSource/TableSink in Table Env Key: FLINK-17748 URL: https://issues.apache.org/jira/browse/FLINK-17748 Project: Flink Issue Type: Sub-task Components: Table SQL / API Reporter: Kurt Young Assignee: Zhenghua Gao Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17635) Add documentation about view support
Kurt Young created FLINK-17635: -- Summary: Add documentation about view support Key: FLINK-17635 URL: https://issues.apache.org/jira/browse/FLINK-17635 Project: Flink Issue Type: Sub-task Components: Documentation Reporter: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17599) Update documents due to FLIP-84
Kurt Young created FLINK-17599: -- Summary: Update documents due to FLIP-84 Key: FLINK-17599 URL: https://issues.apache.org/jira/browse/FLINK-17599 Project: Flink Issue Type: Sub-task Components: Documentation Reporter: Kurt Young -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17406) add documentation about dynamic table options
Kurt Young created FLINK-17406: -- Summary: add documentation about dynamic table options Key: FLINK-17406 URL: https://issues.apache.org/jira/browse/FLINK-17406 Project: Flink Issue Type: Sub-task Components: Documentation Reporter: Kurt Young Assignee: Danny Chen Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17371) Failed to cast decimal from string or double
Kurt Young created FLINK-17371: -- Summary: Failed to cast decimal from string or double Key: FLINK-17371 URL: https://issues.apache.org/jira/browse/FLINK-17371 Project: Flink Issue Type: Improvement Components: Table SQL / Planner Reporter: Kurt Young see: [https://github.com/apache/flink/blob/master/flink-table/flink-table-planner-blink/src/test/scala/org/apache/flink/table/planner/expressions/DecimalTypeTest.scala#L134] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17362) Improve table examples to reflect latest status
Kurt Young created FLINK-17362: -- Summary: Improve table examples to reflect latest status Key: FLINK-17362 URL: https://issues.apache.org/jira/browse/FLINK-17362 Project: Flink Issue Type: Improvement Components: Examples Reporter: Kurt Young Fix For: 1.11.0 Currently the table examples seems outdated, especially after blink planner becomes the default choice. We might need to refactor the structure of all examples, and cover the following items: # streaming sql & table api examples # batch sql & table api examples # table/sql & datastream interoperation # table/sql & dataset interoperation # DDL & DML examples -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17340) Update docs which related to default planner changes
Kurt Young created FLINK-17340: -- Summary: Update docs which related to default planner changes Key: FLINK-17340 URL: https://issues.apache.org/jira/browse/FLINK-17340 Project: Flink Issue Type: Sub-task Components: Documentation Reporter: Kurt Young Assignee: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17339) Change default planner to blink and update test cases in both planners
Kurt Young created FLINK-17339: -- Summary: Change default planner to blink and update test cases in both planners Key: FLINK-17339 URL: https://issues.apache.org/jira/browse/FLINK-17339 Project: Flink Issue Type: Sub-task Components: Table SQL / Legacy Planner, Table SQL / Planner Reporter: Kurt Young Assignee: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17235) Dynamic table options will not work when there doesn't exist a space between + and OPTIONS
Kurt Young created FLINK-17235:
--
Summary: Dynamic table options will not work when there doesn't
exist a space between + and OPTIONS
Key: FLINK-17235
URL: https://issues.apache.org/jira/browse/FLINK-17235
Project: Flink
Issue Type: Sub-task
Components: Table SQL / API
Affects Versions: 1.11.0
Reporter: Kurt Young
I accidentally typed:
{code:java}
/*+OPTIONS('format.ignore-parse-errors'='true')*/{code}
an error is thrown:
{code:java}
java.lang.AssertionError: Hint: PTIONS should be registered in the
HintStrategyTable
{code}
Looks like it's required that there exists a space between + and OPTIONS
cc [~danny0405]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-17234) Show more error messages in taskmanager's log
Kurt Young created FLINK-17234:
--
Summary: Show more error messages in taskmanager's log
Key: FLINK-17234
URL: https://issues.apache.org/jira/browse/FLINK-17234
Project: Flink
Issue Type: Improvement
Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Runtime
/ Task
Reporter: Kurt Young
Fix For: 1.11.0
I created a csv table in sql client and trying to view the table content by
executing a simple SELECT * query.
First I got the error message printed in sql cli:
{code:java}
[ERROR] Could not execute SQL statement. Reason:
java.lang.IllegalStateException: can't switch state from terminal state READING
to CLOSED{code}
And then I open the TM's log to fin more information about what went wrong. The
only information I got from log file is similar with sql cli:
{code:java}
2020-04-19 11:50:28,630 WARN org.apache.flink.runtime.taskmanager.Task
[] - CsvTableSource(read fields: t, author) ->
SourceConversion(table=[default_catalog.default_database.contri, source:
[CsvTableSource(read fields: t, author)]], fields=[t, author]) ->
SinkConversionToRow -> Sink: Unnamed (1/1) (ed397f0f69e8f48b320c568f91a5976e)
switched from RUNNING to FAILED.2020-04-19 11:50:28,630 WARN
org.apache.flink.runtime.taskmanager.Task [] -
CsvTableSource(read fields: t, author) ->
SourceConversion(table=[default_catalog.default_database.contri, source:
[CsvTableSource(read fields: t, author)]], fields=[t, author]) ->
SinkConversionToRow -> Sink: Unnamed (1/1) (ed397f0f69e8f48b320c568f91a5976e)
switched from RUNNING to FAILED.java.lang.IllegalStateException: can't switch
state from terminal state READING to CLOSED at
org.apache.flink.util.Preconditions.checkState(Preconditions.java:217)
~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT] at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.switchState(ContinuousFileReaderOperator.java:366)
~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT] at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.lambda$new$0(ContinuousFileReaderOperator.java:213)
~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT] at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.run(StreamTaskActionExecutor.java:42)
~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT] at
org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:78)
~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT] at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:276)
~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT] at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxStep(MailboxProcessor.java:205)
~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT] at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:196)
~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT] at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:490)
~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT] at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:470)
~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT] at
org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:718)
[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT] at
org.apache.flink.runtime.taskmanager.Task.run(Task.java:542)
[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT] at
java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]{code}
Finally, it turns out I specified wrong column type of the csv table, and
something should have go wrong when reading the data file, but I can't get any
useful information from log file.
I'm not sure whether the root cause is the error message never thrown by csv
parser or got eat up by mailbox model yet.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-17233) Remove copied methods from Calcite introduced by fixing TIMESTAMPDIFF
Kurt Young created FLINK-17233: -- Summary: Remove copied methods from Calcite introduced by fixing TIMESTAMPDIFF Key: FLINK-17233 URL: https://issues.apache.org/jira/browse/FLINK-17233 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Kurt Young In order to FLINK-16823, we copied some methods from Calcite, which already fixed in Calcite 1.23. We should delete these copied methods after upgrading to Calcite 1.23 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16935) Open or delete ignored test cases in blink planner
Kurt Young created FLINK-16935: -- Summary: Open or delete ignored test cases in blink planner Key: FLINK-16935 URL: https://issues.apache.org/jira/browse/FLINK-16935 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Affects Versions: 1.10.0 Reporter: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16934) Change default planner to blink
Kurt Young created FLINK-16934: -- Summary: Change default planner to blink Key: FLINK-16934 URL: https://issues.apache.org/jira/browse/FLINK-16934 Project: Flink Issue Type: Improvement Components: Table SQL / API Affects Versions: 1.10.0 Reporter: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16642) CSV TableSource / TableSink shouldn't be in flink-table-api-java-bridge package
Kurt Young created FLINK-16642: -- Summary: CSV TableSource / TableSink shouldn't be in flink-table-api-java-bridge package Key: FLINK-16642 URL: https://issues.apache.org/jira/browse/FLINK-16642 Project: Flink Issue Type: Improvement Components: Table SQL / API Affects Versions: 1.10.0 Reporter: Kurt Young If user wants to use pure table api program to read/write csv data, he should be able to do this with only the following modules: # flink-table-common (for filesystem connector) # flink-table-api-java (for TableEnvironemtn) # flink-csv (for csv format) But right now it seems the user also has to depend on `flink-table-api-scala` because csv table source is in that module. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16534) Support specify savepoint path when submitting sql job through sql client
Kurt Young created FLINK-16534: -- Summary: Support specify savepoint path when submitting sql job through sql client Key: FLINK-16534 URL: https://issues.apache.org/jira/browse/FLINK-16534 Project: Flink Issue Type: New Feature Components: Table SQL / Client Reporter: Kurt Young When user submitted a sql job via sql client, they can stop/pause the job with savepoint. But after that, if user want to resume such job with the savepoint, there is no such feature yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16466) Group by on event time should produce insert only result
Kurt Young created FLINK-16466: -- Summary: Group by on event time should produce insert only result Key: FLINK-16466 URL: https://issues.apache.org/jira/browse/FLINK-16466 Project: Flink Issue Type: Improvement Components: Table SQL / Planner Affects Versions: 1.10.0 Reporter: Kurt Young Currently when doing aggregation queries, we can output insert only results only when grouping by windows. But when users defined event time and also watermark, we can also support emit insert only results when grouping on event time. To be more precise, it should only require event time is one of the grouping keys. One can think of grouping by event time is kind of a special window, with both window start and window end equals to event time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16343) Failed to read a table with watermark in batch mode
Kurt Young created FLINK-16343: -- Summary: Failed to read a table with watermark in batch mode Key: FLINK-16343 URL: https://issues.apache.org/jira/browse/FLINK-16343 Project: Flink Issue Type: Bug Components: Table SQL / Planner Affects Versions: 1.10.0 Reporter: Kurt Young We can just ignore watermark in batch mode. cc [~jark] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15950) Remove registration of TableSource/TableSink in Table Env and ConnectTableDescriptor
Kurt Young created FLINK-15950: -- Summary: Remove registration of TableSource/TableSink in Table Env and ConnectTableDescriptor Key: FLINK-15950 URL: https://issues.apache.org/jira/browse/FLINK-15950 Project: Flink Issue Type: Improvement Components: Table SQL / API Reporter: Kurt Young Fix For: 1.11.0 This ticket would track the removal of direct TableSource/TableSink registration to TableEnvironment. Since we have lots of tests rely on this, I will create some sub tasks to divide this big one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15896) Stop using TableSource::getTableSchema
Kurt Young created FLINK-15896: -- Summary: Stop using TableSource::getTableSchema Key: FLINK-15896 URL: https://issues.apache.org/jira/browse/FLINK-15896 Project: Flink Issue Type: Sub-task Components: Table SQL / API Reporter: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15895) Stop using TableSource::getReturnType except for compatibility purpose
Kurt Young created FLINK-15895: -- Summary: Stop using TableSource::getReturnType except for compatibility purpose Key: FLINK-15895 URL: https://issues.apache.org/jira/browse/FLINK-15895 Project: Flink Issue Type: Sub-task Components: Table SQL / API Reporter: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15894) Stop overriding TableSource::getTableSchema in flink walkthrough
Kurt Young created FLINK-15894: -- Summary: Stop overriding TableSource::getTableSchema in flink walkthrough Key: FLINK-15894 URL: https://issues.apache.org/jira/browse/FLINK-15894 Project: Flink Issue Type: Sub-task Components: Table SQL / API Reporter: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15893) Stop overriding TableSource::getTableSchema in python table source
Kurt Young created FLINK-15893: -- Summary: Stop overriding TableSource::getTableSchema in python table source Key: FLINK-15893 URL: https://issues.apache.org/jira/browse/FLINK-15893 Project: Flink Issue Type: Sub-task Components: API / Python, Table SQL / API Reporter: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15892) Stop overriding TableSource::getTableSchema in csv table source
Kurt Young created FLINK-15892: -- Summary: Stop overriding TableSource::getTableSchema in csv table source Key: FLINK-15892 URL: https://issues.apache.org/jira/browse/FLINK-15892 Project: Flink Issue Type: Sub-task Components: Table SQL / API Reporter: Kurt Young Assignee: Jingsong Lee Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15890) Stop overriding TableSource::getTableSchema in orc table source
Kurt Young created FLINK-15890: -- Summary: Stop overriding TableSource::getTableSchema in orc table source Key: FLINK-15890 URL: https://issues.apache.org/jira/browse/FLINK-15890 Project: Flink Issue Type: Sub-task Components: Connectors / ORC, Table SQL / API Reporter: Kurt Young Assignee: Jingsong Lee Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15891) Stop overriding TableSource::getTableSchema in parquet table source
Kurt Young created FLINK-15891: -- Summary: Stop overriding TableSource::getTableSchema in parquet table source Key: FLINK-15891 URL: https://issues.apache.org/jira/browse/FLINK-15891 Project: Flink Issue Type: Sub-task Components: Table SQL / API Reporter: Kurt Young Assignee: Jingsong Lee Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15889) Stop overriding TableSource::getTableSchema in jdbc connector
Kurt Young created FLINK-15889: -- Summary: Stop overriding TableSource::getTableSchema in jdbc connector Key: FLINK-15889 URL: https://issues.apache.org/jira/browse/FLINK-15889 Project: Flink Issue Type: Sub-task Components: Connectors / JDBC, Table SQL / API Reporter: Kurt Young Assignee: Zhenghua Gao Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15888) Stop overriding TableSource::getTableSchema in hbase
Kurt Young created FLINK-15888: -- Summary: Stop overriding TableSource::getTableSchema in hbase Key: FLINK-15888 URL: https://issues.apache.org/jira/browse/FLINK-15888 Project: Flink Issue Type: Sub-task Components: Connectors / HBase, Table SQL / API Reporter: Kurt Young Assignee: Leonard Xu Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15887) Stop overriding TableSource::getTableSchema in kafka
Kurt Young created FLINK-15887: -- Summary: Stop overriding TableSource::getTableSchema in kafka Key: FLINK-15887 URL: https://issues.apache.org/jira/browse/FLINK-15887 Project: Flink Issue Type: Sub-task Components: Connectors / Kafka, Table SQL / API Reporter: Kurt Young Assignee: Jark Wu Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15886) Stop overriding TableSource::getTableSchema in hive table source
Kurt Young created FLINK-15886: -- Summary: Stop overriding TableSource::getTableSchema in hive table source Key: FLINK-15886 URL: https://issues.apache.org/jira/browse/FLINK-15886 Project: Flink Issue Type: Sub-task Components: Connectors / Hive, Table SQL / API Reporter: Kurt Young Assignee: Rui Li Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15885) Stop overriding TableSource::getTableSchema in tests
Kurt Young created FLINK-15885: -- Summary: Stop overriding TableSource::getTableSchema in tests Key: FLINK-15885 URL: https://issues.apache.org/jira/browse/FLINK-15885 Project: Flink Issue Type: Sub-task Components: Table SQL / API, Tests Reporter: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15884) Stop overriding TableSource::getReturnType in python table source
Kurt Young created FLINK-15884: -- Summary: Stop overriding TableSource::getReturnType in python table source Key: FLINK-15884 URL: https://issues.apache.org/jira/browse/FLINK-15884 Project: Flink Issue Type: Sub-task Components: API / Python, Table SQL / API Reporter: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15882) Stop overriding TableSource::getReturnType in orc table source
Kurt Young created FLINK-15882: -- Summary: Stop overriding TableSource::getReturnType in orc table source Key: FLINK-15882 URL: https://issues.apache.org/jira/browse/FLINK-15882 Project: Flink Issue Type: Sub-task Components: Connectors / ORC, Table SQL / API Reporter: Kurt Young Assignee: Jingsong Lee Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15883) Stop overriding TableSource::getReturnType in parquet table source
Kurt Young created FLINK-15883: -- Summary: Stop overriding TableSource::getReturnType in parquet table source Key: FLINK-15883 URL: https://issues.apache.org/jira/browse/FLINK-15883 Project: Flink Issue Type: Sub-task Components: Table SQL / API Reporter: Kurt Young Assignee: Jingsong Lee Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15881) Stop overriding TableSource::getReturnType in jdbc connector
Kurt Young created FLINK-15881: -- Summary: Stop overriding TableSource::getReturnType in jdbc connector Key: FLINK-15881 URL: https://issues.apache.org/jira/browse/FLINK-15881 Project: Flink Issue Type: Sub-task Components: Connectors / JDBC, Table SQL / API Reporter: Kurt Young Assignee: Zhenghua Gao Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15879) Stop overriding TableSource::getReturnType in kafka connector
Kurt Young created FLINK-15879: -- Summary: Stop overriding TableSource::getReturnType in kafka connector Key: FLINK-15879 URL: https://issues.apache.org/jira/browse/FLINK-15879 Project: Flink Issue Type: Sub-task Components: Connectors / Kafka, Table SQL / API Reporter: Kurt Young Assignee: Jark Wu Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15880) Stop overriding TableSource::getReturnType in hbase connector
Kurt Young created FLINK-15880: -- Summary: Stop overriding TableSource::getReturnType in hbase connector Key: FLINK-15880 URL: https://issues.apache.org/jira/browse/FLINK-15880 Project: Flink Issue Type: Sub-task Components: Connectors / HBase, Table SQL / API Reporter: Kurt Young Assignee: Leonard Xu Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15878) Stop overriding TableSource::getReturnType in tests
Kurt Young created FLINK-15878: -- Summary: Stop overriding TableSource::getReturnType in tests Key: FLINK-15878 URL: https://issues.apache.org/jira/browse/FLINK-15878 Project: Flink Issue Type: Sub-task Components: Tests Reporter: Kurt Young Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15877) Stop using deprecated methods from TableSource interface
Kurt Young created FLINK-15877: -- Summary: Stop using deprecated methods from TableSource interface Key: FLINK-15877 URL: https://issues.apache.org/jira/browse/FLINK-15877 Project: Flink Issue Type: Improvement Components: Table SQL / API Reporter: Kurt Young Fix For: 1.11.0 This is an *umbrella* issue to track the cleaning work of current TableSource interface. Currently, methods like `getReturnType` and `getTableSchema` are already deprecated, but still used by lots of codes in various connectors and test codes. We should make sure no connector and testing codes would use these deprecated methods anymore, except for the backward compatibility callings. This is to prepare for the further interface improvement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15497) Topn
Kurt Young created FLINK-15497: -- Summary: Topn Key: FLINK-15497 URL: https://issues.apache.org/jira/browse/FLINK-15497 Project: Flink Issue Type: Bug Reporter: Kurt Young -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15440) Enable savepoint support for Table & SQL program
Kurt Young created FLINK-15440: -- Summary: Enable savepoint support for Table & SQL program Key: FLINK-15440 URL: https://issues.apache.org/jira/browse/FLINK-15440 Project: Flink Issue Type: New Feature Components: Table SQL / API Reporter: Kurt Young -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15124) types with precision can't be executed in sql client with blink planner
Kurt Young created FLINK-15124:
--
Summary: types with precision can't be executed in sql client with
blink planner
Key: FLINK-15124
URL: https://issues.apache.org/jira/browse/FLINK-15124
Project: Flink
Issue Type: Bug
Components: Table SQL / Client, Table SQL / Planner
Affects Versions: 1.10.0
Reporter: Kurt Young
I created a table in sql client with blink planner:
{noformat}
create table t (
a int,
b varchar,
c decimal(10, 5))
with (
'connector.type' = 'filesystem',
'format.type' = 'csv',
'format.derive-schema' = 'true',
'connector.path' = 'xxx'
);
{noformat}
The table description looks good:
{noformat}
Flink SQL> describe t;
root
|-- a: INT
|-- b: STRING
|-- c: DECIMAL(10, 5){noformat}
But the select query failed:
{noformat}
Flink SQL> select * from t;
[ERROR] Could not execute SQL statement. Reason:
org.apache.flink.table.planner.codegen.CodeGenException: Incompatible types of
expression and result type.
Expression[GeneratedExpression(field$3,isNull$3,,DECIMAL(38, 18),None)] type is
[DECIMAL(38, 18)], result type is [DECIMAL(10, 5)]
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-15091) JoinITCase.testFullJoinWithNonEquiJoinPred failed in travis
Kurt Young created FLINK-15091: -- Summary: JoinITCase.testFullJoinWithNonEquiJoinPred failed in travis Key: FLINK-15091 URL: https://issues.apache.org/jira/browse/FLINK-15091 Project: Flink Issue Type: Bug Components: Table SQL / Planner Affects Versions: 1.10.0 Reporter: Kurt Young Assignee: Jingsong Lee Fix For: 1.10.0 04:45:22.404 [ERROR] Tests run: 21, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 4.909 s <<< FAILURE! - in org.apache.flink.table.planner.runtime.batch.table.JoinITCase 04:45:22.406 [ERROR] testFullJoinWithNonEquiJoinPred(org.apache.flink.table.planner.runtime.batch.table.JoinITCase) Time elapsed: 0.168 s <<< ERROR! org.apache.flink.runtime.client.JobExecutionException: Job execution failed. at org.apache.flink.table.planner.runtime.batch.table.JoinITCase.testFullJoinWithNonEquiJoinPred(JoinITCase.scala:344) Caused by: org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy Caused by: org.apache.flink.runtime.memory.MemoryAllocationException: Could not allocate 32 pages. Only 0 pages are remaining. details: [https://api.travis-ci.org/v3/job/621407747/log.txt] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15073) sql client fails to run same query multiple times
Kurt Young created FLINK-15073: -- Summary: sql client fails to run same query multiple times Key: FLINK-15073 URL: https://issues.apache.org/jira/browse/FLINK-15073 Project: Flink Issue Type: Bug Components: Table SQL / Client Reporter: Kurt Young Assignee: Danny Chen Flink SQL> select abs(-1); [INFO] Result retrieval cancelled. Flink SQL> select abs(-1); [ERROR] Could not execute SQL statement. Reason: org.apache.flink.table.api.ValidationException: Table 'default: select abs(-1)' already exists. Please choose a different name. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-15066) Cannot run multiple `insert into csvTable values ()`
Kurt Young created FLINK-15066:
--
Summary: Cannot run multiple `insert into csvTable values ()`
Key: FLINK-15066
URL: https://issues.apache.org/jira/browse/FLINK-15066
Project: Flink
Issue Type: Improvement
Components: Table SQL / Client
Reporter: Kurt Young
Assignee: Danny Chen
Fix For: 1.10.0
I created a csv table in sql client, and tried to insert some data into this
table.
The first insert into success, but the second one failed with exception:
{code:java}
// Caused by: java.io.IOException: File or directory /.../xxx.csv already
exists. Existing files and directories are not overwritten in NO_OVERWRITE
mode. Use OVERWRITE mode to overwrite existing files and directories.at
org.apache.flink.core.fs.FileSystem.initOutPathLocalFS(FileSystem.java:817)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-15052) sql client doesn't clear previous job graph
Kurt Young created FLINK-15052: -- Summary: sql client doesn't clear previous job graph Key: FLINK-15052 URL: https://issues.apache.org/jira/browse/FLINK-15052 Project: Flink Issue Type: Bug Components: Table SQL / Client Reporter: Kurt Young Assignee: Danny Chen when executing multiple commands from sql client, the later job graph will include all job graphs which already executed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14931) ConfigOptionsDocsCompletenessITCase fails on traivs
Kurt Young created FLINK-14931: -- Summary: ConfigOptionsDocsCompletenessITCase fails on traivs Key: FLINK-14931 URL: https://issues.apache.org/jira/browse/FLINK-14931 Project: Flink Issue Type: Test Components: Tests Reporter: Kurt Young 23:49:18.967 [ERROR] ConfigOptionsDocsCompletenessITCase.testFullReferenceCompleteness:78->compareDocumentedAndExistingOptions:120 Documentation is outdated, please regenerate it according to the instructions in flink-docs/README.md. Problems: Documented default of classloader.parent-first-patterns.default in class org.apache.flink.configuration.CoreOptions is outdated. Expected: "java.;scala.;org.apache.flink.;com.esotericsoftware.kryo;org.apache.hadoop.;javax.annotation.;org.slf4j;org.apache.log4j;org.apache.logging;org.apache.commons.logging;ch.qos.logback;org.xml;javax.xml;org.apache.xerces;org.w3c" Actual: "java.;scala.;org.apache.flink.;com.esotericsoftware.kryo;org.apache.hadoop.;javax.annotation.;org.slf4j;org.apache.log4j;org.apache.logging;org.apache.commons.logging;ch.qos.logback" More details: [https://api.travis-ci.org/v3/job/615675692/log.txt] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14887) Provide a dedicated test class for SqlDateTimeUtils
Kurt Young created FLINK-14887: -- Summary: Provide a dedicated test class for SqlDateTimeUtils Key: FLINK-14887 URL: https://issues.apache.org/jira/browse/FLINK-14887 Project: Flink Issue Type: Improvement Reporter: Kurt Young Right now, SqlDateTimeUtils's logic in only covered by some code generation cases, we should create a dedicated test class for this utility and test it independently. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14694) Most tests from package o.a.f.table.planner.functions.aggfunctions are not executed during mvn test
Kurt Young created FLINK-14694: -- Summary: Most tests from package o.a.f.table.planner.functions.aggfunctions are not executed during mvn test Key: FLINK-14694 URL: https://issues.apache.org/jira/browse/FLINK-14694 Project: Flink Issue Type: Improvement Components: Table SQL / Planner Reporter: Kurt Young Only `ListAggWsWithRetractAggFunctionTest` and `ListAggWithRetractAggFunctionTest` are executed. And if we run the ignored tests from IDE, some of them will fail. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14693) python tox checks fails on travis
Kurt Young created FLINK-14693: -- Summary: python tox checks fails on travis Key: FLINK-14693 URL: https://issues.apache.org/jira/browse/FLINK-14693 Project: Flink Issue Type: Improvement Components: API / Python Reporter: Kurt Young ImportError: cannot import name 'ensure_is_path' from 'importlib_metadata._compat' (/home/travis/build/apache/flink/flink-python/dev/.conda/lib/python3.7/site-packages/importlib_metadata/_compat.py) tox checks... [FAILED] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14672) Make Executor stateful in sql client
Kurt Young created FLINK-14672: -- Summary: Make Executor stateful in sql client Key: FLINK-14672 URL: https://issues.apache.org/jira/browse/FLINK-14672 Project: Flink Issue Type: Improvement Components: Table SQL / Client Reporter: Kurt Young Currently sql client is stateless, it read config files into `Environment` and recorded all session changes into `SessionContext`. Each time sql client want to execute a new command, we have to first construct `ExecutionContext` by combining `Environment` and `SessionContext`, and then transfer all information into a new `EnvironmentInstance`. It works well before but kind of hard to extend now. One example is support creating table in sql client. We have to remember create statements and replay them every time we want to execute a new command. If we can make sql client stateful, and table environment be the state of sql client, things will be much more easier. This Jira should revisit all related classes, and define a clean relationship between old roles after making sql client stateful. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14671) Collaborate sql parser with sql cli
Kurt Young created FLINK-14671: -- Summary: Collaborate sql parser with sql cli Key: FLINK-14671 URL: https://issues.apache.org/jira/browse/FLINK-14671 Project: Flink Issue Type: Improvement Components: Table SQL / Client Reporter: Kurt Young Currently SQL CLI uses `SqlCommandParser` to parse the command, and we also have a dedicated sql parser to parse ddl, dml, and other stuff. We should find a way to collaborate these two components. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14663) Distinguish unknown column stats and zero
Kurt Young created FLINK-14663:
--
Summary: Distinguish unknown column stats and zero
Key: FLINK-14663
URL: https://issues.apache.org/jira/browse/FLINK-14663
Project: Flink
Issue Type: Improvement
Components: Connectors / Hive, Table SQL / API
Reporter: Kurt Young
When converting from hive stats to flink's column stats, we didn't check
whether some columns stats is really set or just an initial value. For example:
{code:java}
// code placeholder
LongColumnStatsData longColStats = stats.getLongStats();
return new CatalogColumnStatisticsDataLong(
longColStats.getLowValue(),
longColStats.getHighValue(),
longColStats.getNumDVs(),
longColStats.getNumNulls());
{code}
Hive `LongColumnStatsData` actually has information whether some stats is set
through APIs like `isSetNumDVs()`. And the initial values are all 0, it will
confuse us is it really 0 or just an initial value.
We can use -1 to represent UNKNOWN value for column stats.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-14662) Distinguish unknown table stats and zero
Kurt Young created FLINK-14662: -- Summary: Distinguish unknown table stats and zero Key: FLINK-14662 URL: https://issues.apache.org/jira/browse/FLINK-14662 Project: Flink Issue Type: Improvement Components: Connectors / Hive, Table SQL / API Reporter: Kurt Young Currently UNKNOWN table stats is represented with zeros, which might confuse with KNOWN table stats with exactly 0 row count. We can use -1 to represent UNKNOWN instead. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14608) avoid using Java Streams in JsonRowDeserializationSchema
Kurt Young created FLINK-14608: -- Summary: avoid using Java Streams in JsonRowDeserializationSchema Key: FLINK-14608 URL: https://issues.apache.org/jira/browse/FLINK-14608 Project: Flink Issue Type: Improvement Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile) Affects Versions: 1.10.0 Reporter: Kurt Young According to [Flink CodeStyle|[https://flink.apache.org/contributing/code-style-and-quality-java.html]], we should avoid using Java Streams in any performance critical code. Since this `DeserializationRuntimeConverter` will be called per field of each coming record, we should provide a non Java Streams implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14603) NetworkBufferPoolTest.testBlockingRequestFromMultiLocalBufferPool timeout in travis
Kurt Young created FLINK-14603: -- Summary: NetworkBufferPoolTest.testBlockingRequestFromMultiLocalBufferPool timeout in travis Key: FLINK-14603 URL: https://issues.apache.org/jira/browse/FLINK-14603 Project: Flink Issue Type: Test Components: Runtime / Network, Tests Reporter: Kurt Young 21:50:11.821 [ERROR] Tests run: 14, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 14.538 s <<< FAILURE! - in org.apache.flink.runtime.io.network.buffer.NetworkBufferPoolTest 21:50:11.828 [ERROR] testBlockingRequestFromMultiLocalBufferPool(org.apache.flink.runtime.io.network.buffer.NetworkBufferPoolTest) Time elapsed: 10.01 s <<< ERROR! org.junit.runners.model.TestTimedOutException: test timed out after 1 milliseconds at org.apache.flink.runtime.io.network.buffer.NetworkBufferPoolTest.testBlockingRequestFromMultiLocalBufferPool(NetworkBufferPoolTest.java:713) see: [https://api.travis-ci.org/v3/job/607303537/log.txt] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-13867) Write file only once when doing blocking broadcast shuffle
Kurt Young created FLINK-13867: -- Summary: Write file only once when doing blocking broadcast shuffle Key: FLINK-13867 URL: https://issues.apache.org/jira/browse/FLINK-13867 Project: Flink Issue Type: New Feature Components: Runtime / Network Reporter: Kurt Young When doing broadcast shuffle in BATCH/BLOCKING fashion, the producer can only write one copy of output data for all possible consumers, instead of writing one copy of data for each consumer. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (FLINK-13736) Support count window with blink planner in batch mode
Kurt Young created FLINK-13736: -- Summary: Support count window with blink planner in batch mode Key: FLINK-13736 URL: https://issues.apache.org/jira/browse/FLINK-13736 Project: Flink Issue Type: New Feature Components: Table SQL / Planner, Table SQL / Runtime Reporter: Kurt Young -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13735) Support session window with blink planner in batch mode
Kurt Young created FLINK-13735: -- Summary: Support session window with blink planner in batch mode Key: FLINK-13735 URL: https://issues.apache.org/jira/browse/FLINK-13735 Project: Flink Issue Type: New Feature Components: Table SQL / Planner Reporter: Kurt Young -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13688) HiveCatalogUseBlinkITCase.testBlinkUdf constantly failed with 1.9.0-rc2
Kurt Young created FLINK-13688:
--
Summary: HiveCatalogUseBlinkITCase.testBlinkUdf constantly failed
with 1.9.0-rc2
Key: FLINK-13688
URL: https://issues.apache.org/jira/browse/FLINK-13688
Project: Flink
Issue Type: Bug
Components: Connectors / Hive, Tests
Affects Versions: 1.9.0
Environment: Linux server, kernal 3.10.0
java version "1.8.0_102"
Reporter: Kurt Young
Assignee: Jingsong Lee
Fix For: 1.9.0
I tried to build flink 1.9.0-rc2 from source and ran all tests in a linux
server, HiveCatalogUseBlinkITCase.testBlinkUdf will be constantly fail.
Fail trace:
{code:java}
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 313.228
s <<< FAILURE! - in
org.apache.flink.table.catalog.hive.HiveCatalogUseBlinkITCase
[ERROR]
testBlinkUdf(org.apache.flink.table.catalog.hive.HiveCatalogUseBlinkITCase)
Time elapsed: 305.155 s <<< ERROR!
org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
at
org.apache.flink.table.catalog.hive.HiveCatalogUseBlinkITCase.testBlinkUdf(HiveCatalogUseBlinkITCase.java:180)
Caused by:
org.apache.flink.runtime.resourcemanager.exceptions.UnfulfillableSlotRequestException:
Could not fulfill slot request 35cf6fdc1b525de9b6eed13894e2e31d. Requested
resource profile (ResourceProfile{cpuCores=0.0, heapMemoryInMB=0,
directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0,
managedMemoryInMB=128}) is unfulfillable.
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (FLINK-13687) elasticsearch5.ElasticsearchSinkITCase constantly fail with 1.9.0-rc2
Kurt Young created FLINK-13687:
--
Summary: elasticsearch5.ElasticsearchSinkITCase constantly fail
with 1.9.0-rc2
Key: FLINK-13687
URL: https://issues.apache.org/jira/browse/FLINK-13687
Project: Flink
Issue Type: Bug
Components: Connectors / ElasticSearch, Tests
Affects Versions: 1.9.0
Environment: Linux server, kernal 3.10.0
java version "1.8.0_102"
Reporter: Kurt Young
Fix For: 1.9.0
I tried to build flink 1.9.0-rc2 from source and ran all tests in a linux
server, elasticsearch5.ElasticsearchSinkITCase will be constantly fail.
failed trace:
{code:java}
1528 [main] INFO
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase -
-
1528 [main] INFO
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase -
Starting embedded Elasticsearch node
1528 [main] INFO
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase -
-
ERROR StatusLogger Unable to create class
org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.apache.logging.slf4j.SLF4JLoggerContextFactory
specified in
file:/dump/1/kete.yangkt/flink-1.9.0-rc2/flink-1.9.0/flink-connectors/flink-connector-elasticsearch5/target/classes/META-INF/log4j-provider.properties
java.lang.ClassNotFoundException:
org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.apache.logging.slf4j.SLF4JLoggerContextFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:448)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:380)
at
org.apache.logging.log4j.spi.Provider.loadLoggerContextFactory(Provider.java:96)
at org.apache.logging.log4j.LogManager.(LogManager.java:91)
at
org.elasticsearch.common.logging.ESLoggerFactory.getLogger(ESLoggerFactory.java:49)
at org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:105)
at org.elasticsearch.node.Node.(Node.java:237)
at
org.apache.flink.streaming.connectors.elasticsearch.EmbeddedElasticsearchNodeEnvironmentImpl$PluginNode.(EmbeddedElasticsearchNodeEnvironmentImpl.java:78)
at
org.apache.flink.streaming.connectors.elasticsearch.EmbeddedElasticsearchNodeEnvironmentImpl.start(EmbeddedElasticsearchNodeEnvironmentImpl.java:54)
at
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase.prepare(ElasticsearchSinkTestBase.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
ERROR StatusLogger No log4j2 configuration file found. Using default
configuration: logging only errors to the console.
2546 [main] INFO
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase -
-
2546 [main] INFO
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkTestBase -
Shutting down embedded Elasticsearch node
2546 [main] INFO
[jira] [Created] (FLINK-13592) test_tpch.sh should not hardcode flink version
Kurt Young created FLINK-13592: -- Summary: test_tpch.sh should not hardcode flink version Key: FLINK-13592 URL: https://issues.apache.org/jira/browse/FLINK-13592 Project: Flink Issue Type: Bug Components: Tests Affects Versions: 1.9.0 Reporter: Kurt Young Fix For: 1.9.0 in flink-end-to-end-tests/test-scripts/test_tpch.sh, there are two hardcoded ` flink-tpch-test-1.10-SNAPSHOT.jar` -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13591) 'Completed Job List' in Flink web doesn't display right when job name is very long
Kurt Young created FLINK-13591: -- Summary: 'Completed Job List' in Flink web doesn't display right when job name is very long Key: FLINK-13591 URL: https://issues.apache.org/jira/browse/FLINK-13591 Project: Flink Issue Type: Bug Components: Runtime / Web Frontend Affects Versions: 1.9.0 Reporter: Kurt Young Attachments: 10_57_07__08_06_2019.jpg !10_57_07__08_06_2019.jpg! -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13238) Reduce blink planner's testing time
Kurt Young created FLINK-13238: -- Summary: Reduce blink planner's testing time Key: FLINK-13238 URL: https://issues.apache.org/jira/browse/FLINK-13238 Project: Flink Issue Type: Improvement Components: Table SQL / Planner Reporter: Kurt Young The blink planner has an independent CI profile, but still exceeded 50 min limit from time to time. We need to optimize the tests to reduce the testing time. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13234) TemporalTypesTest randomly failed on travis
Kurt Young created FLINK-13234: -- Summary: TemporalTypesTest randomly failed on travis Key: FLINK-13234 URL: https://issues.apache.org/jira/browse/FLINK-13234 Project: Flink Issue Type: Bug Components: Table SQL / Planner Affects Versions: 1.9.0 Reporter: Kurt Young Assignee: Jingsong Lee TemporalTypesTest>ExpressionTestBase.evaluateExprs:154 Wrong result for: [CURRENT_DATE] optimized to: [CURRENT_DATE] expected:<2019-07-1[2]> but was:<2019-07-1[1]> more details in: [https://api.travis-ci.org/v3/job/557582157/log.txt] -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13221) Blink planner should set ScheduleMode to LAZY_FROM_SOURCES_WITH_BATCH_SLOT_REQUEST for batch jobs
Kurt Young created FLINK-13221: -- Summary: Blink planner should set ScheduleMode to LAZY_FROM_SOURCES_WITH_BATCH_SLOT_REQUEST for batch jobs Key: FLINK-13221 URL: https://issues.apache.org/jira/browse/FLINK-13221 Project: Flink Issue Type: Improvement Components: Table SQL / Planner Reporter: Kurt Young -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13212) Unstable ChainLengthIncreaseTest
Kurt Young created FLINK-13212: -- Summary: Unstable ChainLengthIncreaseTest Key: FLINK-13212 URL: https://issues.apache.org/jira/browse/FLINK-13212 Project: Flink Issue Type: Test Components: Tests Reporter: Kurt Young 10:05:29.300 [ERROR] ChainLengthIncreaseTest>AbstractOperatorRestoreTestBase.testMigrationAndRestore:102->AbstractOperatorRestoreTestBase.migrateJob:138 » Execution 10:05:29.301 [ERROR] ChainLengthIncreaseTest>AbstractOperatorRestoreTestBase.testMigrationAndRestore:102->AbstractOperatorRestoreTestBase.migrateJob:138 » Execution More details in: [https://api.travis-ci.org/v3/job/557222905/log.txt] -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13208) Add Notice file for upgrading calcite to 1.20
Kurt Young created FLINK-13208: -- Summary: Add Notice file for upgrading calcite to 1.20 Key: FLINK-13208 URL: https://issues.apache.org/jira/browse/FLINK-13208 Project: Flink Issue Type: Task Components: Table SQL / Legacy Planner, Table SQL / Planner Reporter: Kurt Young Assignee: Kurt Young -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13202) Unstable StandaloneResourceManagerTest
Kurt Young created FLINK-13202: -- Summary: Unstable StandaloneResourceManagerTest Key: FLINK-13202 URL: https://issues.apache.org/jira/browse/FLINK-13202 Project: Flink Issue Type: Test Components: Runtime / Coordination Affects Versions: 1.9.0 Reporter: Kurt Young [https://api.travis-ci.org/v3/job/557150195/log.txt] 06:37:02.888 [ERROR] Failures: 06:37:02.889 [ERROR] StandaloneResourceManagerTest.testStartupPeriod:60->assertHappensUntil:114 condition was not fulfilled before the deadline -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13201) Unstable sql time udf test
Kurt Young created FLINK-13201:
--
Summary: Unstable sql time udf test
Key: FLINK-13201
URL: https://issues.apache.org/jira/browse/FLINK-13201
Project: Flink
Issue Type: Test
Components: Table SQL / Planner
Affects Versions: 1.9.0
Reporter: Kurt Young
Assignee: Jingsong Lee
org.apache.flink.table.runtime.batch.sql.CalcITCase:testTimeUDF will
occasionally fail when running all scala tests through IDE.
The output is:
{code:java}
java.lang.AssertionError:
Results do not match for query:
SELECT dateFunc(a), localDateFunc(a), dateFunc(b), localDateFunc(b),
timeFunc(c), localTimeFunc(c), timeFunc(d), localTimeFunc(d), timestampFunc(e),
datetimeFunc(e), timestampFunc(f), datetimeFunc(f) FROM MyTable
Results
== Correct Result - 1 ==
== Actual Result - 1 ==
!1984-07-12,1984-07-12,1984-07-12,1984-07-12,08:03:09,08:03:09,08:03:09,08:03:09,2019-09-19
08:03:09.0,2019-09-19T08:03:09,2019-09-19 08:03:09.0,2019-09-19T08:03:09
1984-07-11,1984-07-12,1984-07-11,1984-07-12,00:03:09,08:03:09,08:03:09,16:03:09,2019-09-19
00:03:09.0,2019-09-19T08:03:09,2019-09-19 08:03:09.0,2019-09-19T16:03:09
Plan:
== Abstract Syntax Tree ==
LogicalProject(EXPR$0=[dateFunc($0)], EXPR$1=[localDateFunc($0)],
EXPR$2=[dateFunc($1)], EXPR$3=[localDateFunc($1)], EXPR$4=[timeFunc($2)],
EXPR$5=[localTimeFunc($2)], EXPR$6=[timeFunc($3)], EXPR$7=[localTimeFunc($3)],
EXPR$8=[timestampFunc($4)], EXPR$9=[datetimeFunc($4)],
EXPR$10=[timestampFunc($5)], EXPR$11=[datetimeFunc($5)])
+- LogicalTableScan(table=[[default_catalog, default_database, MyTable]])
== Optimized Logical Plan ==
Calc(select=[dateFunc(a) AS EXPR$0, localDateFunc(a) AS EXPR$1, dateFunc(b) AS
EXPR$2, localDateFunc(b) AS EXPR$3, timeFunc(c) AS EXPR$4, localTimeFunc(c) AS
EXPR$5, timeFunc(d) AS EXPR$6, localTimeFunc(d) AS EXPR$7, timestampFunc(e) AS
EXPR$8, datetimeFunc(e) AS EXPR$9, timestampFunc(f) AS EXPR$10, datetimeFunc(f)
AS EXPR$11])
+- BoundedStreamScan(table=[[default_catalog, default_database, MyTable]],
fields=[a, b, c, d, e, f])
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (FLINK-12506) Add more over window unit tests
Kurt Young created FLINK-12506: -- Summary: Add more over window unit tests Key: FLINK-12506 URL: https://issues.apache.org/jira/browse/FLINK-12506 Project: Flink Issue Type: Improvement Components: Table SQL / Runtime, Tests Reporter: Kurt Young We only have ITCase for streaming over window, need to add more unit tests for various process functions -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-12088) Introduce unbounded streaming inner join operator
Kurt Young created FLINK-12088: -- Summary: Introduce unbounded streaming inner join operator Key: FLINK-12088 URL: https://issues.apache.org/jira/browse/FLINK-12088 Project: Flink Issue Type: Improvement Components: Table SQL / Runtime Reporter: Kurt Young Assignee: Kurt Young This operator is responsible for unbounded streaming inner join, and will be optimized in following cases: # If the join keys (with equality condition) are also primary key, we will have a more efficient state layout # If the inputs have primary keys, but join keys are not primary key, we can also come up with an efficient state layout # Inputs don't have primary keys, this will go to default implementation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-12062) Introduce bundle operator to streaming table runtime
Kurt Young created FLINK-12062: -- Summary: Introduce bundle operator to streaming table runtime Key: FLINK-12062 URL: https://issues.apache.org/jira/browse/FLINK-12062 Project: Flink Issue Type: Improvement Components: Table SQL / Runtime Reporter: Kurt Young Bundle operator will try to save incoming records in a key-value map. Once bundler triggers, the bundle function will be invoked. All buffered data will be passed in, and one can do some optimizations based on this. One useful scenario for bundle operator is "Group Aggregate". We can organize the bundle data with grouping key. Once bundle triggers, we can first pre aggregate all data belongs to same key in memory, then we only have to operate with state once for each key. This will save lots of cost and have better performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-12061) Add more window operator contract tests to table runtime
Kurt Young created FLINK-12061: -- Summary: Add more window operator contract tests to table runtime Key: FLINK-12061 URL: https://issues.apache.org/jira/browse/FLINK-12061 Project: Flink Issue Type: Improvement Components: Table SQL / Runtime Reporter: Kurt Young [FLINK-11959|https://issues.apache.org/jira/browse/FLINK-11959] introduced a window operator for table runtime. But the tests are not enough comparing to DataStream's window operator. We can borrow more tests from it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11959) Introduce window operator for blink streaming runtime
Kurt Young created FLINK-11959: -- Summary: Introduce window operator for blink streaming runtime Key: FLINK-11959 URL: https://issues.apache.org/jira/browse/FLINK-11959 Project: Flink Issue Type: New Feature Components: Runtime / Operators Reporter: Kurt Young Assignee: Kurt Young We introduced a new window operator in blink streaming runtime, the differences between blink's window operator and the one used in DataStream API are: # The blink's window operator is mainly used by window aggregate. It work closely with SQL's aggregate function, hence we didn't provide the flexibility to apply arbitrary `WindowFunction` like DataStream did. Instead, we only need to save the intermediate accumulate state for aggregate functions. There is no need for us to save original input rows into state, which will be much more efficient. # This new window operator can deal with retract messages. # We did some pane based optimization within sliding window operator, similar with [FLINK-7001|https://issues.apache.org/jira/browse/FLINK-7001]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11930) Split SegmentsUtil into some dedicated utilities
Kurt Young created FLINK-11930: -- Summary: Split SegmentsUtil into some dedicated utilities Key: FLINK-11930 URL: https://issues.apache.org/jira/browse/FLINK-11930 Project: Flink Issue Type: Improvement Reporter: Kurt Young SegmentsUtil contains at least 3 kinds of utilities now: # get/set primitives to segment array # copy data in and out with segment array # a reused thread-local byte[] We can separate these utilities into dedicated classes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11927) Code clean up and refactor BinaryHashTable and LongHybridHashTable
Kurt Young created FLINK-11927: -- Summary: Code clean up and refactor BinaryHashTable and LongHybridHashTable Key: FLINK-11927 URL: https://issues.apache.org/jira/browse/FLINK-11927 Project: Flink Issue Type: Improvement Components: Runtime / Operators Reporter: Kurt Young More detailed information should be added once we decide how to refactor these. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11872) update lz4 license file
Kurt Young created FLINK-11872: -- Summary: update lz4 license file Key: FLINK-11872 URL: https://issues.apache.org/jira/browse/FLINK-11872 Project: Flink Issue Type: Improvement Components: Build System Reporter: Kurt Young Assignee: Kurt Young -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11871) Introduce LongHashTable to improve performance when join key fits in long
Kurt Young created FLINK-11871: -- Summary: Introduce LongHashTable to improve performance when join key fits in long Key: FLINK-11871 URL: https://issues.apache.org/jira/browse/FLINK-11871 Project: Flink Issue Type: Improvement Components: Runtime / Operators Reporter: Kurt Young -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11864) Let compressed channel reader/writer reuse the logic of AsynchronousFileIOChannel
Kurt Young created FLINK-11864: -- Summary: Let compressed channel reader/writer reuse the logic of AsynchronousFileIOChannel Key: FLINK-11864 URL: https://issues.apache.org/jira/browse/FLINK-11864 Project: Flink Issue Type: Improvement Reporter: Kurt Young This is a follow up issue of [Flink-11863|https://issues.apache.org/jira/browse/FLINK-11863]. The introduced `CompressedBlockChannelReader` and `CompressedBlockChannelWriter` should reuse the logic of `AsynchronousFileIOChannel` by extending from it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11863) Introduce channel to read and write compressed data
Kurt Young created FLINK-11863: -- Summary: Introduce channel to read and write compressed data Key: FLINK-11863 URL: https://issues.apache.org/jira/browse/FLINK-11863 Project: Flink Issue Type: New Feature Reporter: Kurt Young Assignee: Kurt Young -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11858) Introduce block compressor/decompressor for batch table runtime
Kurt Young created FLINK-11858: -- Summary: Introduce block compressor/decompressor for batch table runtime Key: FLINK-11858 URL: https://issues.apache.org/jira/browse/FLINK-11858 Project: Flink Issue Type: Improvement Components: Runtime / Operators Reporter: Kurt Young Assignee: Kurt Young -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11856) Introduce BinaryHashTable and LongHashTable to batch table runtime
Kurt Young created FLINK-11856: -- Summary: Introduce BinaryHashTable and LongHashTable to batch table runtime Key: FLINK-11856 URL: https://issues.apache.org/jira/browse/FLINK-11856 Project: Flink Issue Type: Improvement Components: Runtime / Operators Reporter: Kurt Young Assignee: Kurt Young -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11832) Convert some InternalType property related functions to InternalType's method
Kurt Young created FLINK-11832: -- Summary: Convert some InternalType property related functions to InternalType's method Key: FLINK-11832 URL: https://issues.apache.org/jira/browse/FLINK-11832 Project: Flink Issue Type: Improvement Components: SQL / Planner Reporter: Kurt Young For example, in `CodeGenUtils` we have some utils methods like `needCopyForType` and `needCloneRefForType`. They are very error prone because there is no guarantee when someone add a new InternalType, he will check this functionality. There also exists some pattern-match style codes based on types, i think a feasible principle when to convert these to class's method and when to keep they as pattern-match is: "If the pattern-match codes stand for all further changes, we can keep it as it is. Otherwise, we should convert it to class's method as much as possible". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11831) Separate CodeGeneratorContext for different generation targets
Kurt Young created FLINK-11831: -- Summary: Separate CodeGeneratorContext for different generation targets Key: FLINK-11831 URL: https://issues.apache.org/jira/browse/FLINK-11831 Project: Flink Issue Type: Improvement Components: SQL / Planner Reporter: Kurt Young Currently, we have one big `CodeGeneratorContext` during code generation, it contains all possible reused statements even for different code generation targets. For example, when we want to codeden a `Function`, we may use the `reusableOpenStatements` and `reusableCloseStatements`. And when we codegen an operator, we will use something like `reusableEndInputStatements`. It is error prone and hard to know what should be placed into codegened classes and what shouldn't. I think we should have dedicated context for each codegen class. Like `FunctionCodegenContext` and `OperatorCodegenContext`, they can share a base class to reuse some common requirements, like reused class members. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11674) Add an initial Blink SQL code generator
Kurt Young created FLINK-11674: -- Summary: Add an initial Blink SQL code generator Key: FLINK-11674 URL: https://issues.apache.org/jira/browse/FLINK-11674 Project: Flink Issue Type: Sub-task Components: Table API SQL Reporter: Kurt Young A more detailed description can be found in [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions] This issue is an umbrella issue for tasks related to the code generator for Blink SQL planner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11675) Add an initial support for running batch jobs with streaming runtime
Kurt Young created FLINK-11675: -- Summary: Add an initial support for running batch jobs with streaming runtime Key: FLINK-11675 URL: https://issues.apache.org/jira/browse/FLINK-11675 Project: Flink Issue Type: Sub-task Reporter: Kurt Young cc [~pnowojski] This is an umbrella issue to add an initial support for running batch jobs with streaming runtime. It includes tasks like: * Add some necessary extension to StreamTransformation to meet the requirement of batch job * Make StreamTransformation, StreamGraph and StreamTask to support running batch jobs * other related necessary changes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-7987) Use RexBuilder when converting Expression to RexNode
Kurt Young created FLINK-7987:
-
Summary: Use RexBuilder when converting Expression to RexNode
Key: FLINK-7987
URL: https://issues.apache.org/jira/browse/FLINK-7987
Project: Flink
Issue Type: Improvement
Components: Table API & SQL
Reporter: Kurt Young
Currently, when converting {{Expression}} to {{RexNode}}, we passed in a
{{RelBuilder}}, which is more complex to construct than {{RexBuilder}}. E.g. we
need to push a fake table and a fake select node into RelBuilder, in order to
get input type information.
I think in this case, {{RexBuilder}} will be enough for generating {{RexNode}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (FLINK-7959) Split CodeGenerator into CodeGeneratorContext and ExprCodeGenerator
Kurt Young created FLINK-7959:
-
Summary: Split CodeGenerator into CodeGeneratorContext and
ExprCodeGenerator
Key: FLINK-7959
URL: https://issues.apache.org/jira/browse/FLINK-7959
Project: Flink
Issue Type: Improvement
Components: Table API & SQL
Reporter: Kurt Young
Assignee: Kurt Young
Priority: Major
Right now {{CodeGenerator}} actually acts two roles, one is responsible for
generating codes from RexNode, and the other one is keeping lots of reusable
statements. It makes more sense to split these logic into two dedicated class.
The new {{CodeGeneratorContext}} will keep all the reusable statements, while
the new {{ExprCodeGenerator}} will only do generating codes from RexNode.
And classes like {{AggregationCodeGenerator}} or {{FunctionCodeGenerator}}, I
think the should not be subclass of the {{CodeGenerator}}, but all as
standalone classes. They can create {{ExprCodeGenerator}} when the need to
generating codes from RexNode, and they can also generating codes by
themselves. The {{CodeGeneratorContext}} can be passed around to collect all
reusable statements, and list them in the final generated class.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (FLINK-7161) Fix misusage of Double.MIN_VALUE and Float.MIN_VALUE
Kurt Young created FLINK-7161: - Summary: Fix misusage of Double.MIN_VALUE and Float.MIN_VALUE Key: FLINK-7161 URL: https://issues.apache.org/jira/browse/FLINK-7161 Project: Flink Issue Type: Bug Affects Versions: 1.3.1 Reporter: Kurt Young Assignee: Kurt Young In FloatSummaryAggregator and DoubleSummaryAggregator, we used Float.MIN_VALUE and Double.MIN_VALUE to be the initial values when doing max aggregation which is wrong. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-6398) RowSerializer's duplicate should always return a new instance
Kurt Young created FLINK-6398:
-
Summary: RowSerializer's duplicate should always return a new
instance
Key: FLINK-6398
URL: https://issues.apache.org/jira/browse/FLINK-6398
Project: Flink
Issue Type: Bug
Components: Core
Affects Versions: 1.2.1, 1.2.0
Reporter: Kurt Young
Assignee: Kurt Young
RowSerializer is stateful because of {{nullMask}}, we should always return a
new instance when duplicating.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (FLINK-6309) Memory consumer weights should be calculated in job vertex level
Kurt Young created FLINK-6309:
-
Summary: Memory consumer weights should be calculated in job
vertex level
Key: FLINK-6309
URL: https://issues.apache.org/jira/browse/FLINK-6309
Project: Flink
Issue Type: Improvement
Components: Optimizer
Reporter: Kurt Young
Currently in {{PlanFinalizer}}, we travel the whole job vertexes to calculate
the memory consumer weights, and then assign the weights for each job vertex.
In a case of a large job graph, e.g. with multiple joins, group reduces, the
consumer weights will be high and the usable memory for each job vertex will be
very low.
I think it makes more sense to calculate the memory consumer weights in job
vertex level (after chaining) to maximize the memory utility.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (FLINK-6154) Refine code reusing between logical and physical operators
Kurt Young created FLINK-6154: - Summary: Refine code reusing between logical and physical operators Key: FLINK-6154 URL: https://issues.apache.org/jira/browse/FLINK-6154 Project: Flink Issue Type: Sub-task Components: Table API & SQL Reporter: Kurt Young This issue aim to be a following code clean up of https://issues.apache.org/jira/browse/FLINK-6149 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (FLINK-6150) change DataSetCost to FlinkRelOptCost
Kurt Young created FLINK-6150:
-
Summary: change DataSetCost to FlinkRelOptCost
Key: FLINK-6150
URL: https://issues.apache.org/jira/browse/FLINK-6150
Project: Flink
Issue Type: Sub-task
Reporter: Kurt Young
Currently the only {{RelOptCost}} we have is {{DataSetCost}}, but it's also
used in StreamTableEnvironment. And once we introduce an additional logical
operator layer, we also need a cost representation. Renaming {{DataSetCost}} to
{{FlinkRelOptCost}} and use it in both logical and physical environment seems
to be a possible solution.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
