回复：[DISCUSS] Proposal of external shuffle service

Glad to receive your positive feedbacks Till! Actually our motivation is to support batch job well as you mentioned. For output level, flink already has the Subpartition abstraction(writer), and currently there are PipelinedSubpartition(memory output) and SpillableSubpartition(one-sp-one-file

[ANNOUNCE] Weekly community update #35

Dear community, this is the weekly community update thread #35. Please post any news and updates you want to share with the community to this thread. # Flink 1.7 community roadmap The Flink community started discussing which features will be included in the next upcoming major release. Join the

Re: [DISCUSS][TABLE] How to handle empty delete for UpsertSource

Hi Hequn, That's great! Yes, let's go with option 2 (from the source's point of view) and later extend Join and Aggregation to discard empty deletes. I agree that the filtering at the sink should be optional and configurable via the query configuration. Again, thanks for starting this

[jira] [Created] (FLINK-10256) Rework JobManagerFailsITCase and JobManagerTest into JobMasterITCase and JobMasterHAITCase

陈梓立 created FLINK-10256: --- Summary: Rework JobManagerFailsITCase and JobManagerTest into JobMasterITCase and JobMasterHAITCase Key: FLINK-10256 URL: https://issues.apache.org/jira/browse/FLINK-10256 Project:

Re: [DISCUSS][TABLE] How to handle empty delete for UpsertSource

Hi Fabian Thanks for your update. The opinions on upsert streams are highly enlightening. I think now I am agree with you that we can choose option 2 to solve the problem: Throw away empty deletes when source generate retractions, otherwise pass empty deletes down. As for the UpsertSink, I think

[jira] [Created] (FLINK-10255) Standby Dispatcher locks submitted JobGraphs

Till Rohrmann created FLINK-10255: - Summary: Standby Dispatcher locks submitted JobGraphs Key: FLINK-10255 URL: https://issues.apache.org/jira/browse/FLINK-10255 Project: Flink Issue Type:

[jira] [Created] (FLINK-10254) Fix check in stateBackend

aitozi created FLINK-10254: -- Summary: Fix check in stateBackend Key: FLINK-10254 URL: https://issues.apache.org/jira/browse/FLINK-10254 Project: Flink Issue Type: Improvement Components:

Re: [DISCUSS] Proposal of external shuffle service

Thanks for starting this design discussion Zhijiang! I really like the idea to introduce a ShuffleService abstraction which allows to have different implementations depending on the actual use case. Especially for batch jobs I can clearly see the benefits of persisting the results somewhere else.

[jira] [Created] (FLINK-10253) Run MetricQueryService with lower priority

Till Rohrmann created FLINK-10253: - Summary: Run MetricQueryService with lower priority Key: FLINK-10253 URL: https://issues.apache.org/jira/browse/FLINK-10253 Project: Flink Issue Type:

[jira] [Created] (FLINK-10252) Handle oversized metric messges

Till Rohrmann created FLINK-10252: - Summary: Handle oversized metric messges Key: FLINK-10252 URL: https://issues.apache.org/jira/browse/FLINK-10252 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10251) Handle oversized response messages in AkkaRpcActor

Till Rohrmann created FLINK-10251: - Summary: Handle oversized response messages in AkkaRpcActor Key: FLINK-10251 URL: https://issues.apache.org/jira/browse/FLINK-10251 Project: Flink Issue

[jira] [Created] (FLINK-10250) Flink doesn't support functions that extend RichAggregateFunction in a group window

PengYang created FLINK-10250: Summary: Flink doesn't support functions that extend RichAggregateFunction in a group window Key: FLINK-10250 URL: https://issues.apache.org/jira/browse/FLINK-10250 Project:

[jira] [Created] (FLINK-10249) Document hadoop/presto s3 file system configuration forwarding

Andrey Zagrebin created FLINK-10249: --- Summary: Document hadoop/presto s3 file system configuration forwarding Key: FLINK-10249 URL: https://issues.apache.org/jira/browse/FLINK-10249 Project: Flink

[jira] [Created] (FLINK-10248) Add HBase Table Sink

Shimin Yang created FLINK-10248: --- Summary: Add HBase Table Sink Key: FLINK-10248 URL: https://issues.apache.org/jira/browse/FLINK-10248 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10247) Run MetricQueryService in separate thread pool

Till Rohrmann created FLINK-10247: - Summary: Run MetricQueryService in separate thread pool Key: FLINK-10247 URL: https://issues.apache.org/jira/browse/FLINK-10247 Project: Flink Issue Type:

[jira] [Created] (FLINK-10246) Harden and separate MetricQueryService

Till Rohrmann created FLINK-10246: - Summary: Harden and separate MetricQueryService Key: FLINK-10246 URL: https://issues.apache.org/jira/browse/FLINK-10246 Project: Flink Issue Type:

[jira] [Created] (FLINK-10245) Add DataStream HBase Sink

Shimin Yang created FLINK-10245: --- Summary: Add DataStream HBase Sink Key: FLINK-10245 URL: https://issues.apache.org/jira/browse/FLINK-10245 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10244) Add option to measure latency as absolute value

Chesnay Schepler created FLINK-10244: Summary: Add option to measure latency as absolute value Key: FLINK-10244 URL: https://issues.apache.org/jira/browse/FLINK-10244 Project: Flink

[jira] [Created] (FLINK-10243) Add option to reduce latency metrics granularity

Chesnay Schepler created FLINK-10243: Summary: Add option to reduce latency metrics granularity Key: FLINK-10243 URL: https://issues.apache.org/jira/browse/FLINK-10243 Project: Flink

[jira] [Created] (FLINK-10242) Disable latency metrics by default

Chesnay Schepler created FLINK-10242: Summary: Disable latency metrics by default Key: FLINK-10242 URL: https://issues.apache.org/jira/browse/FLINK-10242 Project: Flink Issue Type: