Re: [BULK]Re: [SURVEY] Remove Mesos support

2021-03-28 Thread Yangze Guo
+1

Best,
Yangze Guo

On Mon, Mar 29, 2021 at 11:31 AM Xintong Song  wrote:
>
> +1
> It's already a matter of fact for a while that we no longer port new features 
> to the Mesos deployment.
>
> Thank you~
>
> Xintong Song
>
>
>
> On Fri, Mar 26, 2021 at 10:37 PM Till Rohrmann  wrote:
>>
>> +1 for officially deprecating this component for the 1.13 release.
>>
>> Cheers,
>> Till
>>
>> On Thu, Mar 25, 2021 at 1:49 PM Konstantin Knauf  wrote:
>>>
>>> Hi Matthias,
>>>
>>> Thank you for following up on this. +1 to officially deprecate Mesos in the 
>>> code and documentation, too. It will be confusing for users if this 
>>> diverges from the roadmap.
>>>
>>> Cheers,
>>>
>>> Konstantin
>>>
>>> On Thu, Mar 25, 2021 at 12:23 PM Matthias Pohl  
>>> wrote:

 Hi everyone,
 considering the upcoming release of Flink 1.13, I wanted to revive the
 discussion about the Mesos support ones more. Mesos is also already listed
 as deprecated in Flink's overall roadmap [1]. Maybe, it's time to align the
 documentation accordingly to make it more explicit?

 What do you think?

 Best,
 Matthias

 [1] https://flink.apache.org/roadmap.html#feature-radar

 On Wed, Oct 28, 2020 at 9:40 AM Till Rohrmann  wrote:

 > Hi Oleksandr,
 >
 > yes you are right. The biggest problem is at the moment the lack of test
 > coverage and thereby confidence to make changes. We have some e2e tests
 > which you can find here [1]. These tests are, however, quite coarse 
 > grained
 > and are missing a lot of cases. One idea would be to add a Mesos e2e test
 > based on Flink's end-to-end test framework [2]. I think what needs to be
 > done there is to add a Mesos resource and a way to submit jobs to a Mesos
 > cluster to write e2e tests.
 >
 > [1] https://github.com/apache/flink/tree/master/flink-jepsen
 > [2]
 > https://github.com/apache/flink/tree/master/flink-end-to-end-tests/flink-end-to-end-tests-common
 >
 > Cheers,
 > Till
 >
 > On Tue, Oct 27, 2020 at 12:29 PM Oleksandr Nitavskyi <
 > o.nitavs...@criteo.com> wrote:
 >
 >> Hello Xintong,
 >>
 >> Thanks for the insights and support.
 >>
 >> Browsing the Mesos backlog and didn't identify anything critical, which
 >> is left there.
 >>
 >> I see that there are were quite a lot of contributions to the Flink 
 >> Mesos
 >> in the recent version:
 >> https://github.com/apache/flink/commits/master/flink-mesos.
 >> We plan to validate the current Flink master (or release 1.12 branch) 
 >> our
 >> Mesos setup. In case of any issues, we will try to propose changes.
 >> My feeling is that our test results shouldn't affect the Flink 1.12
 >> release cycle. And if any potential commits will land into the 1.12.1 it
 >> should be totally fine.
 >>
 >> In the future, we would be glad to help you guys with any
 >> maintenance-related questions. One of the highest priorities around this
 >> component seems to be the development of the full e2e test.
 >>
 >> Kind Regards
 >> Oleksandr Nitavskyi
 >> 
 >> From: Xintong Song 
 >> Sent: Tuesday, October 27, 2020 7:14 AM
 >> To: dev ; user 
 >> Cc: Piyush Narang 
 >> Subject: [BULK]Re: [SURVEY] Remove Mesos support
 >>
 >> Hi Piyush,
 >>
 >> Thanks a lot for sharing the information. It would be a great relief 
 >> that
 >> you are good with Flink on Mesos as is.
 >>
 >> As for the jira issues, I believe the most essential ones should have
 >> already been resolved. You may find some remaining open issues here [1],
 >> but not all of them are necessary if we decide to keep Flink on Mesos 
 >> as is.
 >>
 >> At the moment and in the short future, I think helps are mostly needed 
 >> on
 >> testing the upcoming release 1.12 with Mesos use cases. The community is
 >> currently actively preparing the new release, and hopefully we could 
 >> come
 >> up with a release candidate early next month. It would be greatly
 >> appreciated if you fork as experienced Flink on Mesos users can help 
 >> with
 >> verifying the release candidates.
 >>
 >>
 >> Thank you~
 >>
 >> Xintong Song
 >>
 >> [1]
 >> https://issues.apache.org/jira/browse/FLINK-17402?jql=project%20%3D%20FLINK%20AND%20component%20%3D%20%22Deployment%20%2F%20Mesos%22%20AND%20status%20%3D%20Open
 >> <
 >> 

[jira] [Created] (FLINK-22005) SQL Client end-to-end test (Old planner) Elasticsearch (v7.5.1)

2021-03-28 Thread Guowei Ma (Jira)
Guowei Ma created FLINK-22005:
-

 Summary: SQL Client end-to-end test (Old planner) Elasticsearch 
(v7.5.1) 
 Key: FLINK-22005
 URL: https://issues.apache.org/jira/browse/FLINK-22005
 Project: Flink
  Issue Type: Bug
  Components: Table SQL / Client
Affects Versions: 1.13.0
Reporter: Guowei Ma


The test fail because of Waiting for Elasticsearch records indefinitely.
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=15583=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=ff888d9b-cd34-53cc-d90f-3e446d355529=19826




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [BULK]Re: [SURVEY] Remove Mesos support

2021-03-28 Thread Xintong Song
+1
It's already a matter of fact for a while that we no longer port new
features to the Mesos deployment.

Thank you~

Xintong Song



On Fri, Mar 26, 2021 at 10:37 PM Till Rohrmann  wrote:

> +1 for officially deprecating this component for the 1.13 release.
>
> Cheers,
> Till
>
> On Thu, Mar 25, 2021 at 1:49 PM Konstantin Knauf 
> wrote:
>
>> Hi Matthias,
>>
>> Thank you for following up on this. +1 to officially deprecate Mesos in
>> the code and documentation, too. It will be confusing for users if this
>> diverges from the roadmap.
>>
>> Cheers,
>>
>> Konstantin
>>
>> On Thu, Mar 25, 2021 at 12:23 PM Matthias Pohl 
>> wrote:
>>
>>> Hi everyone,
>>> considering the upcoming release of Flink 1.13, I wanted to revive the
>>> discussion about the Mesos support ones more. Mesos is also already
>>> listed
>>> as deprecated in Flink's overall roadmap [1]. Maybe, it's time to align
>>> the
>>> documentation accordingly to make it more explicit?
>>>
>>> What do you think?
>>>
>>> Best,
>>> Matthias
>>>
>>> [1] https://flink.apache.org/roadmap.html#feature-radar
>>>
>>> On Wed, Oct 28, 2020 at 9:40 AM Till Rohrmann 
>>> wrote:
>>>
>>> > Hi Oleksandr,
>>> >
>>> > yes you are right. The biggest problem is at the moment the lack of
>>> test
>>> > coverage and thereby confidence to make changes. We have some e2e tests
>>> > which you can find here [1]. These tests are, however, quite coarse
>>> grained
>>> > and are missing a lot of cases. One idea would be to add a Mesos e2e
>>> test
>>> > based on Flink's end-to-end test framework [2]. I think what needs to
>>> be
>>> > done there is to add a Mesos resource and a way to submit jobs to a
>>> Mesos
>>> > cluster to write e2e tests.
>>> >
>>> > [1] https://github.com/apache/flink/tree/master/flink-jepsen
>>> > [2]
>>> >
>>> https://github.com/apache/flink/tree/master/flink-end-to-end-tests/flink-end-to-end-tests-common
>>> >
>>> > Cheers,
>>> > Till
>>> >
>>> > On Tue, Oct 27, 2020 at 12:29 PM Oleksandr Nitavskyi <
>>> > o.nitavs...@criteo.com> wrote:
>>> >
>>> >> Hello Xintong,
>>> >>
>>> >> Thanks for the insights and support.
>>> >>
>>> >> Browsing the Mesos backlog and didn't identify anything critical,
>>> which
>>> >> is left there.
>>> >>
>>> >> I see that there are were quite a lot of contributions to the Flink
>>> Mesos
>>> >> in the recent version:
>>> >> https://github.com/apache/flink/commits/master/flink-mesos.
>>> >> We plan to validate the current Flink master (or release 1.12 branch)
>>> our
>>> >> Mesos setup. In case of any issues, we will try to propose changes.
>>> >> My feeling is that our test results shouldn't affect the Flink 1.12
>>> >> release cycle. And if any potential commits will land into the 1.12.1
>>> it
>>> >> should be totally fine.
>>> >>
>>> >> In the future, we would be glad to help you guys with any
>>> >> maintenance-related questions. One of the highest priorities around
>>> this
>>> >> component seems to be the development of the full e2e test.
>>> >>
>>> >> Kind Regards
>>> >> Oleksandr Nitavskyi
>>> >> 
>>> >> From: Xintong Song 
>>> >> Sent: Tuesday, October 27, 2020 7:14 AM
>>> >> To: dev ; user 
>>> >> Cc: Piyush Narang 
>>> >> Subject: [BULK]Re: [SURVEY] Remove Mesos support
>>> >>
>>> >> Hi Piyush,
>>> >>
>>> >> Thanks a lot for sharing the information. It would be a great relief
>>> that
>>> >> you are good with Flink on Mesos as is.
>>> >>
>>> >> As for the jira issues, I believe the most essential ones should have
>>> >> already been resolved. You may find some remaining open issues here
>>> [1],
>>> >> but not all of them are necessary if we decide to keep Flink on Mesos
>>> as is.
>>> >>
>>> >> At the moment and in the short future, I think helps are mostly
>>> needed on
>>> >> testing the upcoming release 1.12 with Mesos use cases. The community
>>> is
>>> >> currently actively preparing the new release, and hopefully we could
>>> come
>>> >> up with a release candidate early next month. It would be greatly
>>> >> appreciated if you fork as experienced Flink on Mesos users can help
>>> with
>>> >> verifying the release candidates.
>>> >>
>>> >>
>>> >> Thank you~
>>> >>
>>> >> Xintong Song
>>> >>
>>> >> [1]
>>> >>
>>> https://issues.apache.org/jira/browse/FLINK-17402?jql=project%20%3D%20FLINK%20AND%20component%20%3D%20%22Deployment%20%2F%20Mesos%22%20AND%20status%20%3D%20Open
>>> >> <
>>> >>
>>> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FFLINK-17402%3Fjql%3Dproject%2520%253D%2520FLINK%2520AND%2520component%2520%253D%2520%2522Deployment%2520%252F%2520Mesos%2522%2520AND%2520status%2520%253D%2520Open=04%7C01%7Co.nitavskyi%40criteo.com%7C3585e1f25bdf4e091af808d87a3f92db%7C2a35d8fd574d48e3927c8c398e225a01%7C1%7C0%7C637393760750820881%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000=hytJFQE0MCPzMLiQTQTdbg3GVckX5M3r1NPRGrRV8j4%3D=0
>>> >> >
>>> >>
>>> >> On Tue, Oct 27, 2020 at 

[jira] [Created] (FLINK-22004) Translate Flink Roadmap to Chinese.

2021-03-28 Thread Yuan Mei (Jira)
Yuan Mei created FLINK-22004:


 Summary: Translate Flink Roadmap to Chinese.
 Key: FLINK-22004
 URL: https://issues.apache.org/jira/browse/FLINK-22004
 Project: Flink
  Issue Type: Task
  Components: Documentation
Reporter: Yuan Mei



https://flink.apache.org/roadmap.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (FLINK-22002) AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg fail because of submitting task time-out.

2021-03-28 Thread Guowei Ma (Jira)
Guowei Ma created FLINK-22002:
-

 Summary: 
AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg fail 
because of submitting task time-out.
 Key: FLINK-22002
 URL: https://issues.apache.org/jira/browse/FLINK-22002
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Coordination
Affects Versions: 1.12.2
Reporter: Guowei Ma


https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=15634=logs=955770d3-1fed-5a0a-3db6-0c7554c910cb=14447d61-56b4-5000-80c1-daa459247f6a=6424


{code:java}
org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase
2021-03-29T00:27:25.3406344Z [ERROR] 
testSingleAggOnTable_HashAgg_WithLocalAgg(org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase)
  Time elapsed: 21.908 s  <<< ERROR!
2021-03-29T00:27:25.3407190Z java.lang.RuntimeException: Failed to fetch next 
result
2021-03-29T00:27:25.3407792Zat 
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109)
2021-03-29T00:27:25.3408502Zat 
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80)
2021-03-29T00:27:25.3409188Zat 
org.apache.flink.table.planner.sinks.SelectTableSinkBase$RowIteratorWrapper.hasNext(SelectTableSinkBase.java:117)
2021-03-29T00:27:25.3416724Zat 
org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:350)
2021-03-29T00:27:25.3417510Zat 
java.util.Iterator.forEachRemaining(Iterator.java:115)
2021-03-29T00:27:25.3418416Zat 
org.apache.flink.util.CollectionUtil.iteratorToList(CollectionUtil.java:108)
2021-03-29T00:27:25.3419031Zat 
org.apache.flink.table.planner.runtime.utils.BatchTestBase.executeQuery(BatchTestBase.scala:298)
2021-03-29T00:27:25.3419657Zat 
org.apache.flink.table.planner.runtime.utils.BatchTestBase.check(BatchTestBase.scala:138)
2021-03-29T00:27:25.3420638Zat 
org.apache.flink.table.planner.runtime.utils.BatchTestBase.checkResult(BatchTestBase.scala:104)
2021-03-29T00:27:25.3421384Zat 
org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase.testSingleAggOnTable(AggregateReduceGroupingITCase.scala:182)
2021-03-29T00:27:25.3422284Zat 
org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg(AggregateReduceGroupingITCase.scala:135)
2021-03-29T00:27:25.3422975Zat 
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2021-03-29T00:27:25.3423504Zat 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2021-03-29T00:27:25.3424298Zat 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2021-03-29T00:27:25.3425229Zat 
java.lang.reflect.Method.invoke(Method.java:498)
2021-03-29T00:27:25.3426107Zat 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
2021-03-29T00:27:25.3426756Zat 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
2021-03-29T00:27:25.3427743Zat 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
2021-03-29T00:27:25.3428520Zat 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
2021-03-29T00:27:25.3429128Zat 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
2021-03-29T00:27:25.3429715Zat 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
2021-03-29T00:27:25.3433435Zat 
org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
2021-03-29T00:27:25.3433977Zat 
org.junit.rules.RunRules.evaluate(RunRules.java:20)
2021-03-29T00:27:25.3434476Zat 
org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
2021-03-29T00:27:25.3435607Zat 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
2021-03-29T00:27:25.3436460Zat 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
2021-03-29T00:27:25.3437054Zat 
org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
2021-03-29T00:27:25.3437673Zat 
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
2021-03-29T00:27:25.3438765Zat 
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
2021-03-29T00:27:25.3439362Zat 
org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
2021-03-29T00:27:25.3440504Zat 
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
2021-03-29T00:27:25.3441100Zat 
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
2021-03-29T00:27:25.3441673Zat 
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
2021-03-29T00:27:25.3442205Zat 

[jira] [Created] (FLINK-22001) Exceptions from JobMaster initialization are not forwarded to the user

2021-03-28 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-22001:
--

 Summary: Exceptions from JobMaster initialization are not 
forwarded to the user
 Key: FLINK-22001
 URL: https://issues.apache.org/jira/browse/FLINK-22001
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Coordination
Affects Versions: 1.13.0
Reporter: Robert Metzger


Steps to reproduce:
Set up a streaming job with an invalid parallelism configuration, for example:
{code}
.setParallelism(15).setMaxParallelism(1);
{code}

This should report the following exception to the user:
{code}
Caused by: org.apache.flink.runtime.JobException: Vertex 
Window(GlobalWindows(), DeltaTrigger, TimeEvictor, ComparableAggregator, 
PassThroughWindowFunction)'s parallelism (15) is higher than the max 
parallelism (1). Please lower the parallelism or increase the max parallelism.
at 
org.apache.flink.runtime.executiongraph.ExecutionJobVertex.(ExecutionJobVertex.java:160)
at 
org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.attachJobGraph(DefaultExecutionGraph.java:781)
at 
org.apache.flink.runtime.executiongraph.DefaultExecutionGraphBuilder.buildGraph(DefaultExecutionGraphBuilder.java:193)
at 
org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory.createAndRestoreExecutionGraph(DefaultExecutionGraphFactory.java:106)
at 
org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:252)
at 
org.apache.flink.runtime.scheduler.SchedulerBase.(SchedulerBase.java:185)
at 
org.apache.flink.runtime.scheduler.DefaultScheduler.(DefaultScheduler.java:119)
at 
org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:132)
at 
org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:110)
at 
org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:340)
at 
org.apache.flink.runtime.jobmaster.JobMaster.(JobMaster.java:317)
at 
org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:94)
at 
org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:39)
at 
org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.startJobMasterServiceSafely(JobManagerRunnerImpl.java:363)
... 13 more
{code}

However, what the user sees is 
{code}
2021-03-28 20:32:33,935 INFO  
org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Job 
419f60eac551619fc1081c670ced3649 reached globally terminal state FAILED.

...

2021-03-28 20:32:33,974 INFO  
org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Stopped 
dispatcher akka://flink/user/rpc/dispatcher_2.
2021-03-28 20:32:33,977 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService  
   [] - Stopping Akka RPC service.
Exception in thread "main" org.apache.flink.util.FlinkException: Failed to 
execute job 'CarTopSpeedWindowingExample'.
at 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1975)
at 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1853)
at 
org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:69)
at 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1839)
at 
org.apache.flink.streaming.examples.windowing.TopSpeedWindowing.main(TopSpeedWindowing.java:101)
Caused by: java.lang.RuntimeException: Error while waiting for job to be 
initialized
at 
org.apache.flink.client.ClientUtils.waitUntilJobInitializationFinished(ClientUtils.java:160)
at 
org.apache.flink.client.program.PerJobMiniClusterFactory.lambda$submitJob$2(PerJobMiniClusterFactory.java:83)
at 
org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:73)
at 
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
at 
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
at 
java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:457)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at 
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at 

[jira] [Created] (FLINK-22000) Set a default character set in InputStreamReader to solve potential garbled problems

2021-03-28 Thread dbgp2021 (Jira)
dbgp2021 created FLINK-22000:


 Summary: Set a default character set in InputStreamReader to solve 
potential garbled problems
 Key: FLINK-22000
 URL: https://issues.apache.org/jira/browse/FLINK-22000
 Project: Flink
  Issue Type: Improvement
Reporter: dbgp2021


When a InputStreamReader is used, the parameter setting of a default character 
set is recommended to solve potential garbled problem.
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (FLINK-21999) The logic about whether Checkpoint is enabled.

2021-03-28 Thread ZhangWei (Jira)
ZhangWei created FLINK-21999:


 Summary: The logic about whether Checkpoint is enabled.
 Key: FLINK-21999
 URL: https://issues.apache.org/jira/browse/FLINK-21999
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Checkpointing
Reporter: ZhangWei


org.apache.flink.runtime.executiongraph.DefaultExecutionGraphBuilder#isCheckpointingEnabled
 assumes checkpoint enabled when JobCheckpointingSettings is not null. While 
this is not enough, we must also guarantee the checkpoint interval is between 
[MINIMAL_CHECKPOINT_TIME, Long.MaxValue). That is like the 
JobGraph#isCheckpointingEnabled does.
   In current implement, when we do not set checkpoint interval, leaving it the 
default value -1, the interval  will be changed to Long.MaxValue. Thus 
DefaultExecutionGraphBuilder#isCheckpointingEnabled will return true. That is 
not correct.

in addition, there are different classes assume checkpoint enabled with 
different interval range.
1. CheckpointConfig -> (0,Long.MaxValue*]*.
2. JobGraph -> (0,Long.MaxValue)
This is not consistent. And the correct range is [MINIMAL_CHECKPOINT_TIME, 
Long.MaxValue).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (FLINK-21998) Copy more code from hive and move them to a dedicated package

2021-03-28 Thread Rui Li (Jira)
Rui Li created FLINK-21998:
--

 Summary: Copy more code from hive and move them to a dedicated 
package
 Key: FLINK-21998
 URL: https://issues.apache.org/jira/browse/FLINK-21998
 Project: Flink
  Issue Type: Sub-task
  Components: Connectors / Hive
Reporter: Rui Li
 Fix For: 1.13.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)