[jira] [Created] (FLINK-11291) Reorganize the package of flink-cep module

2019-01-08 Thread Dian Fu (JIRA)
Dian Fu created FLINK-11291: --- Summary: Reorganize the package of flink-cep module Key: FLINK-11291 URL: https://issues.apache.org/jira/browse/FLINK-11291 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-11290) BatchTableEnvironment and StreamTableEnvironment should transparent to users

2019-01-08 Thread sunjincheng (JIRA)
sunjincheng created FLINK-11290: --- Summary: BatchTableEnvironment and StreamTableEnvironment should transparent to users Key: FLINK-11290 URL: https://issues.apache.org/jira/browse/FLINK-11290 Project:

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2019-01-08 Thread Bowen Li
Thank you, Xuefu and Timo, for putting together the FLIP! I like that both its scope and implementation plan are clear. Look forward to feedbacks from the group. I also added a few more complementary details in the doc. Thanks, Bowen On Mon, Jan 7, 2019 at 8:37 PM Zhang, Xuefu wrote: >

[DISCUSS] Checkpoint Failure process improvement

2019-01-08 Thread vino yang
Hi all, Currently, the checkpoint's failure handling logic is somewhat confusing (not focused), which makes some functions on existing code passive. So I provide a design document to improve the Checkpoint failure process logic. This design document primarily describes how to improve

[jira] [Created] (FLINK-11289) Rework example module structure to account for licensing

2019-01-08 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-11289: Summary: Rework example module structure to account for licensing Key: FLINK-11289 URL: https://issues.apache.org/jira/browse/FLINK-11289 Project: Flink

[jira] [Created] (FLINK-11288) Add separate flink-ml for building fat jar

2019-01-08 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-11288: Summary: Add separate flink-ml for building fat jar Key: FLINK-11288 URL: https://issues.apache.org/jira/browse/FLINK-11288 Project: Flink Issue

[jira] [Created] (FLINK-11287) RocksDBListState creation should use actual registered state serializer instead of serializer provided by descriptor

2019-01-08 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-11287: --- Summary: RocksDBListState creation should use actual registered state serializer instead of serializer provided by descriptor Key: FLINK-11287 URL:

[jira] [Created] (FLINK-11286) Support to send StreamStatus.IDLE for non-source operators

2019-01-08 Thread vinoyang (JIRA)
vinoyang created FLINK-11286: Summary: Support to send StreamStatus.IDLE for non-source operators Key: FLINK-11286 URL: https://issues.apache.org/jira/browse/FLINK-11286 Project: Flink Issue

Re: [DISCUSS] Support Interactive Programming in Flink Table API

2019-01-08 Thread Becket Qin
Just to clarify, when I say foo() like below, I assume that foo() must have a way to release its own cache, so it must have access to table env. void foo(Table t) { ... t.cache(); // create cache for t ... env.getCacheService().releaseCacheFor(t); // release cache for t } Thanks,

Re: [DISCUSS] Support Interactive Programming in Flink Table API

2019-01-08 Thread Becket Qin
Hi Piotr, I don't think it is feasible to ask every third party library to have method signature with CacheService as an argument. And even that signature does not really solve the problem. Imagine function foo() looks like following: void foo(Table t) { ... t.cache(); // create cache for t

[jira] [Created] (FLINK-11285) Change the modifier of CsvTableSourceFactoryBase.createTableSource and CsvTableSinkFactoryBase.createTableSink to private[flink]

2019-01-08 Thread Dian Fu (JIRA)
Dian Fu created FLINK-11285: --- Summary: Change the modifier of CsvTableSourceFactoryBase.createTableSource and CsvTableSinkFactoryBase.createTableSink to private[flink] Key: FLINK-11285 URL:

Re: [DISCUSS] Support Interactive Programming in Flink Table API

2019-01-08 Thread Piotr Nowojski
Hi, I think that introducing ref counting could be confusing and it will be error prone, since Flink-table’s users are not used to closing/releasing resources. I was more objecting placing the `uncache()`/`dropCache()`/`releaseCache()` (releaseCache sounds best to me) as a method in the

[jira] [Created] (FLINK-11284) RocksDBStateBackendMigrationTest.testStateBackendRestoreSucceedsIfNewKeySerializerRequiresReconfiguration fails on Travis

2019-01-08 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-11284: - Summary: RocksDBStateBackendMigrationTest.testStateBackendRestoreSucceedsIfNewKeySerializerRequiresReconfiguration fails on Travis Key: FLINK-11284 URL:

Re: [DISCUSS] Support Interactive Programming in Flink Table API

2019-01-08 Thread Becket Qin
Hi Piotr, You are right. There might be two intuitive meanings when users call 'a.uncache()', namely: 1. release the resource 2. Do not use cache for the next operation. Case (1) would likely be the dominant use case. So I would suggest we dedicate uncache() method to case (1), i.e. for resource

[jira] [Created] (FLINK-11283) Accessing the key when processing connected keyed stream

2019-01-08 Thread Truong Duc Kien (JIRA)
Truong Duc Kien created FLINK-11283: --- Summary: Accessing the key when processing connected keyed stream Key: FLINK-11283 URL: https://issues.apache.org/jira/browse/FLINK-11283 Project: Flink

Re: [DISCUSS] Detection Flink Backpressure

2019-01-08 Thread Piotr Nowojski
Hi, > I think the direct and accurate way to monitor backpressure is stating how > many times "availableMemorySegments.wait(2000)" is triggered during > "LocalBufferPool#requestMemorySegment", and reflecting the ratio as > backpressure if not affect theperformance. This could be an

Re: [DISCUSS] Support Interactive Programming in Flink Table API

2019-01-08 Thread Piotr Nowojski
Hi Becket, With `uncache` there are probably two features that we can think about: a) Physically dropping the cached table from the storage, freeing up the resources b) Hinting the optimizer to not cache the reads for the next query/table a) Has the issue as I wrote before, that it seemed to

[attention] now there is a test broken in CI environment

2019-01-08 Thread qianjin xu
[attention] now there is a test broken in CI environment, please do not submit the test for the time being, otherwise the test will fail!!!The error is shown below: [image: .png] Best, qianjin

[attention] now there is a test broken in CI environment

2019-01-08 Thread qianjin xu
[attention] now there is a test broken in CI environment, please do not submit the test for the time being, otherwise the test will fail!!!The error is shown below: [image: image.png] Best, qianjin

[jira] [Created] (FLINK-11282) Merge StreamRecordWriter into RecordWriter

2019-01-08 Thread zhijiang (JIRA)
zhijiang created FLINK-11282: Summary: Merge StreamRecordWriter into RecordWriter Key: FLINK-11282 URL: https://issues.apache.org/jira/browse/FLINK-11282 Project: Flink Issue Type: Sub-task