回复：Memory size for network buffers

Hi Thomas, The current calculation for network buffer size is "Math.min(taskmanager.network.memory.max, Math.max(taskmanager.network.memory.min, fraction * totalMem))". Based on your below configuration, the result is just 32768 bytes (8 buffers) from taskmanager.network.memory.min. If you

[jira] [Created] (FLINK-10857) Conflict between JMX and Prometheus Metrics reporter

Truong Duc Kien created FLINK-10857: --- Summary: Conflict between JMX and Prometheus Metrics reporter Key: FLINK-10857 URL: https://issues.apache.org/jira/browse/FLINK-10857 Project: Flink

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

Hi Piotr I have extracted the API portion of the design and the google doc is here. Please review and provide your feedback. Thanks, Xuefu -- Sender:Xuefu Sent at:2018 Nov 12 (Mon) 12:43 Recipient:Piotr Nowojski ; dev Cc:Bowen

Memory size for network buffers

Hi, I'm trying to understand the intention behind the size parameters for network buffers, specifically max, min and fraction. The reason we are looking at it is an attempt to make the memory allocation elastic, so that memory is allocated according to the actual number of buffers required

Re: Kinesis consumer e2e test

Hi, yes, that is correct. The failure mapper is there to cause a failover event for which we can then check i) that exactly-once or at-least-once is not violated, depending on the expected semantics and ii) that the restore works at all ;-). You might be able to reuse

Re: How to use RocksDBStateBackend predefined options

Sounds good. Perhaps it would also be good to allow the user to specify an options factory in flink-conf.yaml for more flexibility? Thomas On Mon, Nov 12, 2018 at 9:48 AM Stefan Richter wrote: > Hi, > > Ufuk is right, for historical reasons there is currently only the > programatic way but I

Re: How to use RocksDBStateBackend predefined options

Hi, Ufuk is right, for historical reasons there is currently only the programatic way but I think nothing speaks fundamentally against offering configuration via config in the future (maybe just a lot of config keys must be introduced to cover all options). Best, Stefan > On 9. Nov 2018, at

[jira] [Created] (FLINK-10855) CheckpointCoordinator does not delete checkpoint directory of late/failed checkpoints

Till Rohrmann created FLINK-10855: - Summary: CheckpointCoordinator does not delete checkpoint directory of late/failed checkpoints Key: FLINK-10855 URL: https://issues.apache.org/jira/browse/FLINK-10855

[jira] [Created] (FLINK-10856) Harden resume from externalized checkpoint E2E test

Till Rohrmann created FLINK-10856: - Summary: Harden resume from externalized checkpoint E2E test Key: FLINK-10856 URL: https://issues.apache.org/jira/browse/FLINK-10856 Project: Flink Issue

[jira] [Created] (FLINK-10853) Provide end-to-end test for Kafka 0.9 connector

vinoyang created FLINK-10853: Summary: Provide end-to-end test for Kafka 0.9 connector Key: FLINK-10853 URL: https://issues.apache.org/jira/browse/FLINK-10853 Project: Flink Issue Type: Test

[jira] [Created] (FLINK-10854) Provide end-to-end test for Kafka 0.8 connector

vinoyang created FLINK-10854: Summary: Provide end-to-end test for Kafka 0.8 connector Key: FLINK-10854 URL: https://issues.apache.org/jira/browse/FLINK-10854 Project: Flink Issue Type: Test

REST job submission

Hi to all, in our ETL we need to call an external (REST) service once a job ends: we extract informations about accumulators and we update the job status. However this is only possible if using the CLI client: if we call the job via the REST API o Web UI (that is very useful to decouple our UI

[jira] [Created] (FLINK-10852) Decremented number of unfinished producers below 0. This is most likely a bug in the execution state/intermediate result partition management

ouyangzhe created FLINK-10852: - Summary: Decremented number of unfinished producers below 0. This is most likely a bug in the execution state/intermediate result partition management Key: FLINK-10852 URL:

[jira] [Created] (FLINK-10851) sqlUpdate support complex insert grammar

frank wang created FLINK-10851: -- Summary: sqlUpdate support complex insert grammar Key: FLINK-10851 URL: https://issues.apache.org/jira/browse/FLINK-10851 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-10850) Job may hang on FAILING state if taskmanager updateTaskExecutionState failed

ouyangzhe created FLINK-10850: - Summary: Job may hang on FAILING state if taskmanager updateTaskExecutionState failed Key: FLINK-10850 URL: https://issues.apache.org/jira/browse/FLINK-10850 Project: