[jira] [Comment Edited] (FLINK-15333) Using FlinkCostFactory rather than RelOptCostImpl in blink planner
[
https://issues.apache.org/jira/browse/FLINK-15333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003131#comment-17003131
]
Leonard Xu edited comment on FLINK-15333 at 12/25/19 7:52 AM:
--
[~lzljs3620320] It's hard to reproduce because planner code path is really
complex.
Both [~godfreyhe] and I encountered this issue by accident, we convinced it's
a bug after checked related code. I suggest we can fix it later,
[~godfreyhe] how do you think?
was (Author: leonard xu):
[~lzljs3620320] It's hard to reproduce because planner code path is really
complex.
Both [~godfreyhe] and I encountered this issue by accident, we convinced it's
a bug after checked related code. I we can fix it later, [~godfreyhe] how do
you think?
> Using FlinkCostFactory rather than RelOptCostImpl in blink planner
> ---
>
> Key: FLINK-15333
> URL: https://issues.apache.org/jira/browse/FLINK-15333
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.10.0
>Reporter: Leonard Xu
>Assignee: godfrey he
>Priority: Major
> Fix For: 1.10.0
>
>
> When I test FLINK SQL in flink 1.10, I found an exception which is a bug and
> need to fix.
> {code:java}
> // Some comments here
> Exception in thread "main" java.lang.ClassCastException:
> org.apache.calcite.plan.RelOptCostImpl$Factory cannot be cast to
> org.apache.flink.table.planner.plan.cost.FlinkCostFactory
> at
> org.apache.flink.table.planner.plan.nodes.common.CommonPhysicalExchange.computeSelfCost(CommonPhysicalExchange.scala:50)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdNonCumulativeCost.getNonCumulativeCost(FlinkRelMdNonCumulativeCost.scala:41)
> at
> GeneratedMetadataHandler_NonCumulativeCost.getNonCumulativeCost_$(Unknown
> Source)
> at
> GeneratedMetadataHandler_NonCumulativeCost.getNonCumulativeCost(Unknown
> Source)
> at
> org.apache.calcite.rel.metadata.RelMetadataQuery.getNonCumulativeCost(RelMetadataQuery.java:301)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdCumulativeCost.getCumulativeCost(FlinkRelMdCumulativeCost.scala:38)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost_$(Unknown
> Source)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost(Unknown
> Source)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost_$(Unknown
> Source)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost(Unknown
> Source)
> at
> org.apache.calcite.rel.metadata.RelMetadataQuery.getCumulativeCost(RelMetadataQuery.java:282)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdCumulativeCost$$anonfun$getCumulativeCost$1.apply(FlinkRelMdCumulativeCost.scala:41)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdCumulativeCost$$anonfun$getCumulativeCost$1.apply(FlinkRelMdCumulativeCost.scala:40)
> at
> scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
> at
> scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at
> scala.collection.TraversableOnce$class.foldLeft(TraversableOnce.scala:157)
> at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdCumulativeCost.getCumulativeCost(FlinkRelMdCumulativeCost.scala:39)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost_$(Unknown
> Source)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost(Unknown
> Source)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost_$(Unknown
> Source)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost(Unknown
> Source)
> at
> org.apache.calcite.rel.metadata.RelMetadataQuery.getCumulativeCost(RelMetadataQuery.java:282)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdCumulativeCost$$anonfun$getCumulativeCost$1.apply(FlinkRelMdCumulativeCost.scala:41)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdCumulativeCost$$anonfun$getCumulativeCost$1.apply(FlinkRelMdCumulativeCost.scala:40)
> at
> scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
> at
> scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
> at
[jira] [Commented] (FLINK-15333) Using FlinkCostFactory rather than RelOptCostImpl in blink planner
[
https://issues.apache.org/jira/browse/FLINK-15333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003131#comment-17003131
]
Leonard Xu commented on FLINK-15333:
[~lzljs3620320] It's hard to reproduce because planner code path is really
complex.
Both [~godfreyhe] and I encountered this issue by accident, we convinced it's
a bug after checked related code. I we can fix it later, [~godfreyhe] how do
you think?
> Using FlinkCostFactory rather than RelOptCostImpl in blink planner
> ---
>
> Key: FLINK-15333
> URL: https://issues.apache.org/jira/browse/FLINK-15333
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.10.0
>Reporter: Leonard Xu
>Assignee: godfrey he
>Priority: Major
> Fix For: 1.10.0
>
>
> When I test FLINK SQL in flink 1.10, I found an exception which is a bug and
> need to fix.
> {code:java}
> // Some comments here
> Exception in thread "main" java.lang.ClassCastException:
> org.apache.calcite.plan.RelOptCostImpl$Factory cannot be cast to
> org.apache.flink.table.planner.plan.cost.FlinkCostFactory
> at
> org.apache.flink.table.planner.plan.nodes.common.CommonPhysicalExchange.computeSelfCost(CommonPhysicalExchange.scala:50)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdNonCumulativeCost.getNonCumulativeCost(FlinkRelMdNonCumulativeCost.scala:41)
> at
> GeneratedMetadataHandler_NonCumulativeCost.getNonCumulativeCost_$(Unknown
> Source)
> at
> GeneratedMetadataHandler_NonCumulativeCost.getNonCumulativeCost(Unknown
> Source)
> at
> org.apache.calcite.rel.metadata.RelMetadataQuery.getNonCumulativeCost(RelMetadataQuery.java:301)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdCumulativeCost.getCumulativeCost(FlinkRelMdCumulativeCost.scala:38)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost_$(Unknown
> Source)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost(Unknown
> Source)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost_$(Unknown
> Source)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost(Unknown
> Source)
> at
> org.apache.calcite.rel.metadata.RelMetadataQuery.getCumulativeCost(RelMetadataQuery.java:282)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdCumulativeCost$$anonfun$getCumulativeCost$1.apply(FlinkRelMdCumulativeCost.scala:41)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdCumulativeCost$$anonfun$getCumulativeCost$1.apply(FlinkRelMdCumulativeCost.scala:40)
> at
> scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
> at
> scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at
> scala.collection.TraversableOnce$class.foldLeft(TraversableOnce.scala:157)
> at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdCumulativeCost.getCumulativeCost(FlinkRelMdCumulativeCost.scala:39)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost_$(Unknown
> Source)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost(Unknown
> Source)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost_$(Unknown
> Source)
> at GeneratedMetadataHandler_CumulativeCost.getCumulativeCost(Unknown
> Source)
> at
> org.apache.calcite.rel.metadata.RelMetadataQuery.getCumulativeCost(RelMetadataQuery.java:282)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdCumulativeCost$$anonfun$getCumulativeCost$1.apply(FlinkRelMdCumulativeCost.scala:41)
> at
> org.apache.flink.table.planner.plan.metadata.FlinkRelMdCumulativeCost$$anonfun$getCumulativeCost$1.apply(FlinkRelMdCumulativeCost.scala:40)
> at
> scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
> at
> scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at
> scala.collection.TraversableOnce$class.foldLeft(TraversableOnce.scala:157)
> at
[GitHub] [flink] flinkbot edited a comment on issue #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
flinkbot edited a comment on issue #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping URL: https://github.com/apache/flink/pull/10682#issuecomment-568850864 ## CI report: * a7411a5d1a86381078670898c1f20be4fe3a88f4 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142305023) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3903) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10672: [FLINK-15373][core][config] Update descriptions for framework / task off-heap memory config options
flinkbot edited a comment on issue #10672: [FLINK-15373][core][config] Update descriptions for framework / task off-heap memory config options URL: https://github.com/apache/flink/pull/10672#issuecomment-568663425 ## CI report: * 31a2ccbabb6675673d445b6a9d258e6622d295d8 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142190064) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3874) * e9328aa4fdbf2978171da9214bd44a87b7d8e016 Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/142305015) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3902) * ed7853792489b3a5e492d8529400d779ece938f9 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/142305997) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3904) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Issue Comment Deleted] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[
https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hiliuxg updated FLINK-15388:
Comment: was deleted
(was: Hi Xintong Song:
the complete log file can be download by link
https://pan.baidu.com/s/1JQJQ-U4Vif6EXR12Agql6w passwd p7ci
i jstat the recently full gc , it takes 0.921 seconds to complete the full gc
S0CS1CS0US1U EC EUOC OU MC MU
CCSC CCSU YGC YGCTFGCFGCT GCT
0.0 753664.0 0.0 753664.0 93716480.0 32931840.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 38731776.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 44728320.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 50954240.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 57245696.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 62750720.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 67207168.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 71630848.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 75792384.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 80281600.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 84672512.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
)
> The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
> --
>
> Key: FLINK-15388
> URL: https://issues.apache.org/jira/browse/FLINK-15388
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.8.0
> Environment: model : standalone,not yarn
> version : flink 1.8.0
> configration :
> jobmanager.heap.size: 4096m
> taskmanager.heap.size: 144gb
> taskmanager.numberOfTaskSlots: 48
> taskmanager.memory.fraction: 0.7
> taskmanager.memory.off-heap: false
> parallelism.default: 1
>
>Reporter: hiliuxg
>Priority: Major
>
> the taskmanager's slot was removed , there was not full gc or oom , what's
> the problem ? the error bellow
> {code:java}
> org.apache.flink.util.FlinkException: The assigned slot
> bae00218c818157649eb9e3c533b86af_32 was removed.
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)
> at
> akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
> at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
> at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
>
[jira] [Commented] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[
https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003128#comment-17003128
]
hiliuxg commented on FLINK-15388:
-
Hi Xintong Song:
the complete log file can be download by link
https://pan.baidu.com/s/1JQJQ-U4Vif6EXR12Agql6w passwd p7ci
i jstat the recently full gc , it takes 0.921 seconds to complete the full gc
S0CS1CS0US1U EC EUOC OU MC MU
CCSC CCSU YGC YGCTFGCFGCT GCT
0.0 753664.0 0.0 753664.0 93716480.0 32931840.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 38731776.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 44728320.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 50954240.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 57245696.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 62750720.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 67207168.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 71630848.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 75792384.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 80281600.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 84672512.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
> The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
> --
>
> Key: FLINK-15388
> URL: https://issues.apache.org/jira/browse/FLINK-15388
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.8.0
> Environment: model : standalone,not yarn
> version : flink 1.8.0
> configration :
> jobmanager.heap.size: 4096m
> taskmanager.heap.size: 144gb
> taskmanager.numberOfTaskSlots: 48
> taskmanager.memory.fraction: 0.7
> taskmanager.memory.off-heap: false
> parallelism.default: 1
>
>Reporter: hiliuxg
>Priority: Major
>
> the taskmanager's slot was removed , there was not full gc or oom , what's
> the problem ? the error bellow
> {code:java}
> org.apache.flink.util.FlinkException: The assigned slot
> bae00218c818157649eb9e3c533b86af_32 was removed.
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)
> at
> akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
> at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
> at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
>
[jira] [Commented] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[
https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003129#comment-17003129
]
hiliuxg commented on FLINK-15388:
-
Hi Xintong Song:
the complete log file can be download by link
https://pan.baidu.com/s/1JQJQ-U4Vif6EXR12Agql6w passwd p7ci
i jstat the recently full gc , it takes 0.921 seconds to complete the full gc
S0CS1CS0US1U EC EUOC OU MC MU
CCSC CCSU YGC YGCTFGCFGCT GCT
0.0 753664.0 0.0 753664.0 93716480.0 32931840.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 38731776.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 44728320.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 50954240.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 57245696.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 62750720.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 67207168.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 71630848.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 75792384.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 80281600.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
0.0 753664.0 0.0 753664.0 93716480.0 84672512.0 55476224.0 34114585.2
805120.0 704223.6 0.00.025905 3870.407 1 0.921 3871.328
> The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
> --
>
> Key: FLINK-15388
> URL: https://issues.apache.org/jira/browse/FLINK-15388
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.8.0
> Environment: model : standalone,not yarn
> version : flink 1.8.0
> configration :
> jobmanager.heap.size: 4096m
> taskmanager.heap.size: 144gb
> taskmanager.numberOfTaskSlots: 48
> taskmanager.memory.fraction: 0.7
> taskmanager.memory.off-heap: false
> parallelism.default: 1
>
>Reporter: hiliuxg
>Priority: Major
>
> the taskmanager's slot was removed , there was not full gc or oom , what's
> the problem ? the error bellow
> {code:java}
> org.apache.flink.util.FlinkException: The assigned slot
> bae00218c818157649eb9e3c533b86af_32 was removed.
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)
> at
> akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
> at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
> at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
>
[GitHub] [flink] zjuwangg commented on a change in pull request #10669: [FLINK-15192][docs][table] Split 'SQL' page into multiple sub pages for better readability
zjuwangg commented on a change in pull request #10669:
[FLINK-15192][docs][table] Split 'SQL' page into multiple sub pages for better
readability
URL: https://github.com/apache/flink/pull/10669#discussion_r361272710
##
File path: docs/dev/table/catalogs.md
##
@@ -144,7 +144,7 @@ List tables = catalog.listTables("mydb); // tables
should contain "mytab
## Catalog API
Note: only catalog program APIs are listed here. Users can achieve many of the
same funtionalities with SQL DDL.
-For detailed DDL information, please refer to [SQL
DDL](https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/table/sql.html#ddl).
+For detailed DDL information, please refer to [SQL DDL](sql/ddl.html).
Review comment:
Should the link here be `/dev/table/sql/ddl.html`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10683: [FLINK-15277][doc] Update SQL DDL document to add syntax and example …
flinkbot commented on issue #10683: [FLINK-15277][doc] Update SQL DDL document to add syntax and example … URL: https://github.com/apache/flink/pull/10683#issuecomment-568857595 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit ad443e9d7364bbd309d5a68b4c8fa5f218ca0756 (Wed Dec 25 07:35:11 UTC 2019) **Warnings:** * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-15277).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15277) Update SQL DDL document to add syntax and example of computed column
[ https://issues.apache.org/jira/browse/FLINK-15277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-15277: --- Labels: pull-request-available (was: ) > Update SQL DDL document to add syntax and example of computed column > > > Key: FLINK-15277 > URL: https://issues.apache.org/jira/browse/FLINK-15277 > Project: Flink > Issue Type: Sub-task > Components: Documentation, Table SQL / API >Affects Versions: 1.10.0 >Reporter: Danny Chen >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] danny0405 opened a new pull request #10683: [FLINK-15277][doc] Update SQL DDL document to add syntax and example …
danny0405 opened a new pull request #10683: [FLINK-15277][doc] Update SQL DDL document to add syntax and example … URL: https://github.com/apache/flink/pull/10683 …of computed column ## What is the purpose of the change Add document for create table computed columns. ## Brief change log - Add computed column syntax for computed column - Add notes for table source sink table schema This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15346) OldCsvValidator has no property of "emptyColumnAsNull".
[ https://issues.apache.org/jira/browse/FLINK-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003124#comment-17003124 ] Leonard Xu commented on FLINK-15346: [~jark] we had supported this feature in 1.10, user can set `emptyColumnAsNull` by table API，what we need to do is just support it in DDL, so I suggest we can merge it in 1.10. > OldCsvValidator has no property of "emptyColumnAsNull". > --- > > Key: FLINK-15346 > URL: https://issues.apache.org/jira/browse/FLINK-15346 > Project: Flink > Issue Type: Bug > Components: Connectors / FileSystem >Affects Versions: 1.10.0 >Reporter: xiaojin.wy >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > > The OldCsvValidator class and CsvTableSourceFactoryBase class should add a > "format.empty-column-as-null" property, so that the users can use it . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] JingsongLi commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] CompileUtils::COMPILED_CACHE leaks class loaders
JingsongLi commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] CompileUtils::COMPILED_CACHE leaks class loaders URL: https://github.com/apache/flink/pull/10620#discussion_r361271907 ## File path: flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/generated/CompileUtils.java ## @@ -45,7 +48,7 @@ * number of Meta zone GC (class unloading), resulting in performance bottlenecks. So we add * a cache to avoid this problem. */ - protected static final Cache, Class> COMPILED_CACHE = CacheBuilder + protected static final Cache> COMPILED_CACHE = CacheBuilder .newBuilder() .maximumSize(100) // estimated cache size Review comment: 1. For the execution mode of Flink, the `UserCodeClassLoader` used by each task may be different. 2. The class from CodeGenerated may contain the dependence on user code class, which is in `UserCodeClassLoader` of a task. So, if we ignore class loader, that will lead to errors. Maybe class not found or class not match. I don't know the details of Spark and Calcite, but maybe Spark breaks # 2 and Calcite breaks # 1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
flinkbot edited a comment on issue #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping URL: https://github.com/apache/flink/pull/10682#issuecomment-568850864 ## CI report: * a7411a5d1a86381078670898c1f20be4fe3a88f4 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/142305023) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3903) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15382) Flink failed generating python config docs
[
https://issues.apache.org/jira/browse/FLINK-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003120#comment-17003120
]
Xintong Song commented on FLINK-15382:
--
[~dian.fu]
I do not have flink-python of 1.11-SNAPSHOT in my local repository, but I do
find flink-python_2.11 of 1.11-SNAPSHOT in my local repository.
Also, if the dependency jar file is missing, the document building should fail
instead of success with empty generated table.
> Flink failed generating python config docs
> ---
>
> Key: FLINK-15382
> URL: https://issues.apache.org/jira/browse/FLINK-15382
> Project: Flink
> Issue Type: Bug
> Components: API / Python, Runtime / Configuration
>Reporter: Xintong Song
>Priority: Major
>
> When generating config option docs with the command suggested by
> {{flink-docs/README.md}}, the generated
> {{docs/_includes/generated/python_configuration.html}} does not contain any
> config options despite that there are 4 options in {{PythonOptions}}.
> I encountered this problem at the commit
> {{545534e43ed37f518fe59b6ddd8ed56ae82a234b}} on master branch.
> Command used to generate doc:
> {code:bash}mvn package -Dgenerate-config-docs -pl flink-docs -am -nsu
> -DskipTests{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10672: [FLINK-15373][core][config] Update descriptions for framework / task off-heap memory config options
flinkbot edited a comment on issue #10672: [FLINK-15373][core][config] Update descriptions for framework / task off-heap memory config options URL: https://github.com/apache/flink/pull/10672#issuecomment-568663425 ## CI report: * 31a2ccbabb6675673d445b6a9d258e6622d295d8 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142190064) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3874) * e9328aa4fdbf2978171da9214bd44a87b7d8e016 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/142305015) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3902) * ed7853792489b3a5e492d8529400d779ece938f9 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] streaming-olap commented on a change in pull request #8751: [FLINK-11937][StateBackend]Resolve small file problem in RocksDB incremental checkpoint
streaming-olap commented on a change in pull request #8751:
[FLINK-11937][StateBackend]Resolve small file problem in RocksDB incremental
checkpoint
URL: https://github.com/apache/flink/pull/8751#discussion_r361260570
##
File path:
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/snapshot/RocksIncrementalSnapshotStrategy.java
##
@@ -154,8 +160,12 @@ public RocksIncrementalSnapshotStrategy(
final SnapshotDirectory snapshotDirectory =
prepareLocalSnapshotDirectory(checkpointId);
LOG.trace("Local RocksDB checkpoint goes to backup path {}.",
snapshotDirectory);
+ if (this.checkpointStreamFactory == null) {
+ this.checkpointStreamFactory = checkpointStreamFactory;
Review comment:
The line 175 may use this.checkpointStreamFactory replace
checkpointStreamFactory. now It causes the connection leak.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[
https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003113#comment-17003113
]
Xintong Song commented on FLINK-15388:
--
BTW, you have configured very large TM heap size ("taskmanager.heap.size:
144gb"). It's probably the TM GC takes too long. But again, further information
is needed to confirm the problem.
> The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
> --
>
> Key: FLINK-15388
> URL: https://issues.apache.org/jira/browse/FLINK-15388
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.8.0
> Environment: model : standalone,not yarn
> version : flink 1.8.0
> configration :
> jobmanager.heap.size: 4096m
> taskmanager.heap.size: 144gb
> taskmanager.numberOfTaskSlots: 48
> taskmanager.memory.fraction: 0.7
> taskmanager.memory.off-heap: false
> parallelism.default: 1
>
>Reporter: hiliuxg
>Priority: Major
>
> the taskmanager's slot was removed , there was not full gc or oom , what's
> the problem ? the error bellow
> {code:java}
> org.apache.flink.util.FlinkException: The assigned slot
> bae00218c818157649eb9e3c533b86af_32 was removed.
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)
> at
> akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
> at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
> at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-15239) CompileUtils::COMPILED_CACHE leaks class loaders
[
https://issues.apache.org/jira/browse/FLINK-15239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated FLINK-15239:
---
Summary: CompileUtils::COMPILED_CACHE leaks class loaders (was: TM
Metaspace memory leak)
> CompileUtils::COMPILED_CACHE leaks class loaders
>
>
> Key: FLINK-15239
> URL: https://issues.apache.org/jira/browse/FLINK-15239
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
>Affects Versions: 1.10.0
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Start a standalone cluster and then submit multiple queries for Hive tables
> via SQL CLI. Hive connector dependencies are specified via the {{library}}
> option. TM will fail eventually with:
> {noformat}
> 2019-12-13 15:11:03,698 INFO org.apache.flink.runtime.taskmanager.Task
> - Source: Values(tuples=[[{ 4.3 }]], values=[EXPR$0]) ->
> SinkConversionToRow -> Sink: Unnamed (1/1) (b9f9667f686fd97c1c5af65b8b163c44)
> switched from RUNNING to FAILED.
> java.lang.OutOfMemoryError: Metaspace
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at
> org.apache.flink.util.ChildFirstClassLoader.loadClass(ChildFirstClassLoader.java:60)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createNNProxyWithClientProtocol(NameNodeProxies.java:439)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:324)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
> at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:687)
> at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:628)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
> at
> org.apache.flink.connectors.hive.HadoopFileSystemFactory.create(HadoopFileSystemFactory.java:46)
> at
> org.apache.flink.table.filesystem.PartitionTempFileManager.(PartitionTempFileManager.java:73)
> at
> org.apache.flink.table.filesystem.FileSystemOutputFormat.open(FileSystemOutputFormat.java:104)
> at
> org.apache.flink.streaming.api.functions.sink.OutputFormatSinkFunction.open(OutputFormatSinkFunction.java:65)
> at
> org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
> at
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
> at
> org.apache.flink.streaming.api.operators.StreamSink.open(StreamSink.java:48)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.initializeStateAndOpen(StreamTask.java:1018)
> {noformat}
> Even for the succeeded queries, TM prints the following errors:
> {noformat}
> Exception in thread "LeaseRenewer:lirui@localhost:8020"
> java.lang.NoClassDefFoundError: org/apache/hadoop/hdfs/LeaseRenewer$2
> at org.apache.hadoop.hdfs.LeaseRenewer.renew(LeaseRenewer.java:412)
> at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:448)
> at
> org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71)
> at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:304)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hdfs.LeaseRenewer$2
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at
>
[GitHub] [flink] flinkbot edited a comment on issue #8751: [FLINK-11937][StateBackend]Resolve small file problem in RocksDB incremental checkpoint
flinkbot edited a comment on issue #8751: [FLINK-11937][StateBackend]Resolve small file problem in RocksDB incremental checkpoint URL: https://github.com/apache/flink/pull/8751#issuecomment-515749731 ## CI report: * 2dda201498d913b3fbf2e7d078dd5440652e6a19 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/142305001) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[
https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003111#comment-17003111
]
Xintong Song commented on FLINK-15388:
--
Could you attach the complete JM and the problematic TM
(bae00218c818157649eb9e3c533b86af) log files?
>From the error messages you posted, the slot is removed because of task
>manager heartbeat timeout.
There are usually several reasons that may cause the heartbeat timeout.
* TM failure.
* Network problem between RM/TM.
* Either RM or TM suffers severe GC and could not response to the heartbeats.
It's hard to tell which is the cause in your case without further information.
> The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
> --
>
> Key: FLINK-15388
> URL: https://issues.apache.org/jira/browse/FLINK-15388
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.8.0
> Environment: model : standalone,not yarn
> version : flink 1.8.0
> configration :
> jobmanager.heap.size: 4096m
> taskmanager.heap.size: 144gb
> taskmanager.numberOfTaskSlots: 48
> taskmanager.memory.fraction: 0.7
> taskmanager.memory.off-heap: false
> parallelism.default: 1
>
>Reporter: hiliuxg
>Priority: Major
>
> the taskmanager's slot was removed , there was not full gc or oom , what's
> the problem ? the error bellow
> {code:java}
> org.apache.flink.util.FlinkException: The assigned slot
> bae00218c818157649eb9e3c533b86af_32 was removed.
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)
> at
> akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
> at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
> at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] danny0405 commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] TM Metaspace memory leak
danny0405 commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] TM Metaspace memory leak URL: https://github.com/apache/flink/pull/10620#discussion_r361266737 ## File path: flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/generated/CompileUtils.java ## @@ -45,7 +48,7 @@ * number of Meta zone GC (class unloading), resulting in performance bottlenecks. So we add * a cache to avoid this problem. */ - protected static final Cache, Class> COMPILED_CACHE = CacheBuilder + protected static final Cache> COMPILED_CACHE = CacheBuilder .newBuilder() .maximumSize(100) // estimated cache size Review comment: Sorry, i didn't have much knowledge, i'm just asking my questions. I also noticed that Apache Spark[1] and Apache Calcite [2] both only cache the classes. [1] https://github.com/apache/spark/blob/8f07839e743889bbffe91302f31388ab892bf08a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala#L1463 [2] https://github.com/apache/calcite/blob/a69d24951794f6d3ff67f40b0d117fd0df9a919b/core/src/main/java/org/apache/calcite/adapter/enumerable/EnumerableInterpretable.java#L100 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] danny0405 commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] TM Metaspace memory leak
danny0405 commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] TM Metaspace memory leak URL: https://github.com/apache/flink/pull/10620#discussion_r361266737 ## File path: flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/generated/CompileUtils.java ## @@ -45,7 +48,7 @@ * number of Meta zone GC (class unloading), resulting in performance bottlenecks. So we add * a cache to avoid this problem. */ - protected static final Cache, Class> COMPILED_CACHE = CacheBuilder + protected static final Cache> COMPILED_CACHE = CacheBuilder .newBuilder() .maximumSize(100) // estimated cache size Review comment: Sorry, i didn't have much knowledge about the classloader thing, i'm just asking my questions. I also noticed that Apache Spark[1] and Apache Calcite [2] both only cache the classes. [1] https://github.com/apache/spark/blob/8f07839e743889bbffe91302f31388ab892bf08a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala#L1463 [2] https://github.com/apache/calcite/blob/a69d24951794f6d3ff67f40b0d117fd0df9a919b/core/src/main/java/org/apache/calcite/adapter/enumerable/EnumerableInterpretable.java#L100 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15382) Flink failed generating python config docs
[
https://issues.apache.org/jira/browse/FLINK-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003108#comment-17003108
]
Dian Fu commented on FLINK-15382:
-
The jar of flink-python should be available for the doc generation. This also
holds for the config options of other components. You could try to remove the
option "-nsu" to allow it to download the dependencies which are not available
in the local repository. The flink-python module should be built by default
when building flink and there is no special arguments required.
> Flink failed generating python config docs
> ---
>
> Key: FLINK-15382
> URL: https://issues.apache.org/jira/browse/FLINK-15382
> Project: Flink
> Issue Type: Bug
> Components: API / Python, Runtime / Configuration
>Reporter: Xintong Song
>Priority: Major
>
> When generating config option docs with the command suggested by
> {{flink-docs/README.md}}, the generated
> {{docs/_includes/generated/python_configuration.html}} does not contain any
> config options despite that there are 4 options in {{PythonOptions}}.
> I encountered this problem at the commit
> {{545534e43ed37f518fe59b6ddd8ed56ae82a234b}} on master branch.
> Command used to generate doc:
> {code:bash}mvn package -Dgenerate-config-docs -pl flink-docs -am -nsu
> -DskipTests{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-15390) List/Map/Struct types support for vectorized orc reader
[ https://issues.apache.org/jira/browse/FLINK-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated FLINK-15390: - Summary: List/Map/Struct types support for vectorized orc reader (was: List/Map/Struct support for vectorized orc reader) > List/Map/Struct types support for vectorized orc reader > --- > > Key: FLINK-15390 > URL: https://issues.apache.org/jira/browse/FLINK-15390 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Hive, Connectors / ORC >Reporter: Jingsong Lee >Priority: Major > Fix For: 1.11.0 > > > Now we only support non-complex types for vectorized orc reader. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on issue #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
flinkbot commented on issue #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping URL: https://github.com/apache/flink/pull/10682#issuecomment-568850864 ## CI report: * a7411a5d1a86381078670898c1f20be4fe3a88f4 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-15390) List/Map/Struct support for vectorized orc reader
Jingsong Lee created FLINK-15390: Summary: List/Map/Struct support for vectorized orc reader Key: FLINK-15390 URL: https://issues.apache.org/jira/browse/FLINK-15390 Project: Flink Issue Type: Sub-task Components: Connectors / Hive, Connectors / ORC Reporter: Jingsong Lee Fix For: 1.11.0 Now we only support non-complex types for vectorized orc reader. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10672: [FLINK-15373][core][config] Update descriptions for framework / task off-heap memory config options
flinkbot edited a comment on issue #10672: [FLINK-15373][core][config] Update descriptions for framework / task off-heap memory config options URL: https://github.com/apache/flink/pull/10672#issuecomment-568663425 ## CI report: * 31a2ccbabb6675673d445b6a9d258e6622d295d8 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142190064) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3874) * e9328aa4fdbf2978171da9214bd44a87b7d8e016 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #10672: [FLINK-15373][core][config] Update descriptions for framework / task off-heap memory config options
xintongsong commented on a change in pull request #10672:
[FLINK-15373][core][config] Update descriptions for framework / task off-heap
memory config options
URL: https://github.com/apache/flink/pull/10672#discussion_r361265641
##
File path:
flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##
@@ -275,8 +276,9 @@
public static final ConfigOption TASK_OFF_HEAP_MEMORY =
key("taskmanager.memory.task.off-heap.size")
.defaultValue("0b")
- .withDescription("Task Heap Memory size for
TaskExecutors. This is the size of off heap memory (JVM direct"
- + " memory or native memory) reserved for user
code.");
+ .withDescription("Task Off-Heap Memory size for
TaskExecutors. This is the size of off heap memory (JVM"
+ + " direct memory and native memory) reserved
for user code. The configured value will be fully counted"
Review comment:
Nice catch, thanks. Comment addressed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] JingsongLi commented on issue #10681: [FLINK-14849][hive][doc] Fix documentation about Hive dependencies
JingsongLi commented on issue #10681: [FLINK-14849][hive][doc] Fix documentation about Hive dependencies URL: https://github.com/apache/flink/pull/10681#issuecomment-568849252 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhuzhurk commented on a change in pull request #10672: [FLINK-15373][core][config] Update descriptions for framework / task off-heap memory config options
zhuzhurk commented on a change in pull request #10672:
[FLINK-15373][core][config] Update descriptions for framework / task off-heap
memory config options
URL: https://github.com/apache/flink/pull/10672#discussion_r361264512
##
File path:
flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##
@@ -275,8 +276,9 @@
public static final ConfigOption TASK_OFF_HEAP_MEMORY =
key("taskmanager.memory.task.off-heap.size")
.defaultValue("0b")
- .withDescription("Task Heap Memory size for
TaskExecutors. This is the size of off heap memory (JVM direct"
- + " memory or native memory) reserved for user
code.");
+ .withDescription("Task Off-Heap Memory size for
TaskExecutors. This is the size of off heap memory (JVM"
+ + " direct memory and native memory) reserved
for user code. The configured value will be fully counted"
Review comment:
I think it should be `for tasks` rather than `for user code`, e.g. it can be
used by Flink sql operators or state backends.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] lirui-apache commented on issue #10681: [FLINK-14849][hive][doc] Fix documentation about Hive dependencies
lirui-apache commented on issue #10681: [FLINK-14849][hive][doc] Fix documentation about Hive dependencies URL: https://github.com/apache/flink/pull/10681#issuecomment-568848774   @JingsongLi Attached are screenshots of my locally built web-site. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15382) Flink failed generating python config docs
[
https://issues.apache.org/jira/browse/FLINK-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003099#comment-17003099
]
Xintong Song commented on FLINK-15382:
--
[~dian.fu]
There's only flink-python jar of version 1.9-SNAPSHOT in my local repository.
Do we need any special argument to build the flink-python module when building
flink?
> Flink failed generating python config docs
> ---
>
> Key: FLINK-15382
> URL: https://issues.apache.org/jira/browse/FLINK-15382
> Project: Flink
> Issue Type: Bug
> Components: API / Python, Runtime / Configuration
>Reporter: Xintong Song
>Priority: Major
>
> When generating config option docs with the command suggested by
> {{flink-docs/README.md}}, the generated
> {{docs/_includes/generated/python_configuration.html}} does not contain any
> config options despite that there are 4 options in {{PythonOptions}}.
> I encountered this problem at the commit
> {{545534e43ed37f518fe59b6ddd8ed56ae82a234b}} on master branch.
> Command used to generate doc:
> {code:bash}mvn package -Dgenerate-config-docs -pl flink-docs -am -nsu
> -DskipTests{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #8751: [FLINK-11937][StateBackend]Resolve small file problem in RocksDB incremental checkpoint
flinkbot edited a comment on issue #8751: [FLINK-11937][StateBackend]Resolve small file problem in RocksDB incremental checkpoint URL: https://github.com/apache/flink/pull/8751#issuecomment-515749731 ## CI report: * 2dda201498d913b3fbf2e7d078dd5440652e6a19 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zoudaokoulife commented on a change in pull request #10674: [FLINK-15220][Connector/Kafka][Table] Add startFromTimestamp in KafkaTableSource
zoudaokoulife commented on a change in pull request #10674:
[FLINK-15220][Connector/Kafka][Table] Add startFromTimestamp in KafkaTableSource
URL: https://github.com/apache/flink/pull/10674#discussion_r361264136
##
File path:
flink-connectors/flink-connector-kafka-0.10/src/main/java/org/apache/flink/streaming/connectors/kafka/Kafka010TableSource.java
##
@@ -62,7 +64,8 @@ public Kafka010TableSource(
Properties properties,
DeserializationSchema deserializationSchema,
StartupMode startupMode,
- Map specificStartupOffsets) {
+ Map specificStartupOffsets,
+ long startupTimestampMillis) {
Review comment:
can reuse the param of properties , not add new param startupTimestampMillis?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on issue #10672: [FLINK-15373][core][config] Update descriptions for framework / task off-heap memory config options
xintongsong commented on issue #10672: [FLINK-15373][core][config] Update descriptions for framework / task off-heap memory config options URL: https://github.com/apache/flink/pull/10672#issuecomment-568847383 @zhuzhurk Done rebasing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10677: [FLINK-15371] Change FLIP-49 memory configurations to use the new memory type config options
flinkbot edited a comment on issue #10677: [FLINK-15371] Change FLIP-49 memory configurations to use the new memory type config options URL: https://github.com/apache/flink/pull/10677#issuecomment-568707829 ## CI report: * b7dc5e6e30b140815530d04cb8898efd1b9cf19c Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142210122) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3885) * f02488f343aabe6a27a41674c1b99225ed7fc16b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142212407) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3887) * 6f5e783884428632d0e3e28867ec05ad726037f4 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142298614) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3895) * de9eb49fc5b0f9c2a54f976d63e2800629e44062 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142300973) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3901) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
flinkbot commented on issue #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping URL: https://github.com/apache/flink/pull/10682#issuecomment-568846548 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit a7411a5d1a86381078670898c1f20be4fe3a88f4 (Wed Dec 25 06:13:16 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[ https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003093#comment-17003093 ] hiliuxg commented on FLINK-15388: - Hi Xintong Song : There is also a error log ,Does not is related to this ? 2019-12-25 07:53:08.937 [Kafka Fetcher for Source: Custom Source -> Filter -> from: (defaultTime, defaultOffset, defaultTopic, defaultPartition, actionTime, actionId, platformId, channelId, kugouId, fanxId, actorId, roomId, isFollower, p1, p2, p3, gatewayTime) -> where: (AND(=(actionId, _UTF-16LE'fx_listpg_rm_enter_room'), IsValUserId(kugouId), IsValUserId(actorId))), select: (gatewayTime, kugouId, actorId, CAST(gatewayTime) AS gatewayTime0) -> time attribute: (gatewayTime) (1/1)] ERROR org.apache.kafka.common.metrics.Metrics - Error when removing metric from org.apache.kafka.common.metrics.JmxReporter org.apache.kafka.common.KafkaException: Error unregistering mbean at org.apache.kafka.common.metrics.JmxReporter.unregister(JmxReporter.java:160) ~[blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.kafka.common.metrics.JmxReporter.reregister(JmxReporter.java:165) ~[blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.kafka.common.metrics.JmxReporter.metricRemoval(JmxReporter.java:100) ~[blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.kafka.common.metrics.Metrics.removeMetric(Metrics.java:529) [blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.kafka.common.metrics.Metrics.removeSensor(Metrics.java:446) [blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.kafka.common.network.Selector$SelectorMetrics.close(Selector.java:1065) [blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.kafka.common.network.Selector.close(Selector.java:323) [blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.kafka.clients.NetworkClient.close(NetworkClient.java:583) [blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.close(ConsumerNetworkClient.java:516) [blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.kafka.clients.ClientUtils.closeQuietly(ClientUtils.java:73) [blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:2148) [blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:2108) [blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:2059) [blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] at org.apache.flink.streaming.connectors.kafka.internal.KafkaConsumerThread.run(KafkaConsumerThread.java:282) [blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] Caused by: javax.management.InstanceNotFoundException: kafka.consumer:type=consumer-node-metrics,client-id=consumer-2,node-id=node-2 at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095) ~[na:1.8.0_211] at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427) ~[na:1.8.0_211] at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415) ~[na:1.8.0_211] at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546) ~[na:1.8.0_211] at org.apache.kafka.common.metrics.JmxReporter.unregister(JmxReporter.java:158) ~[blob_p-7389ec91865981ca89b8c4bbafd884bfcebf707f-e70e451f92f2e8796590f94642ba6f4b:na] ... 13 common frames omitted > The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed. > -- > > Key: FLINK-15388 > URL: https://issues.apache.org/jira/browse/FLINK-15388 > Project: Flink > Issue Type: Bug > Components: Runtime / Task >Affects Versions: 1.8.0 > Environment: model : standalone,not yarn > version : flink 1.8.0 > configration : > jobmanager.heap.size: 4096m > taskmanager.heap.size: 144gb > taskmanager.numberOfTaskSlots: 48 > taskmanager.memory.fraction: 0.7 >
[jira] [Updated] (FLINK-15247) Closing (Testing)MiniCluster may cause ConcurrentModificationException
[
https://issues.apache.org/jira/browse/FLINK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-15247:
---
Labels: pull-request-available test-stability (was: test-stability)
> Closing (Testing)MiniCluster may cause ConcurrentModificationException
> --
>
> Key: FLINK-15247
> URL: https://issues.apache.org/jira/browse/FLINK-15247
> Project: Flink
> Issue Type: Bug
> Components: Tests
>Affects Versions: 1.10.0
>Reporter: Gary Yao
>Assignee: Andrey Zagrebin
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 1.10.0
>
>
> {noformat}
> Test
> operatorsBecomeBackPressured(org.apache.flink.test.streaming.runtime.BackPressureITCase)
> failed with:
> org.apache.flink.util.FlinkException: Could not close resource.
> at
> org.apache.flink.util.AutoCloseableAsync.close(AutoCloseableAsync.java:42)org.apache.flink.test.streaming.runtime.BackPressureITCase.tearDown(BackPressureITCase.java:165)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> Caused by: org.apache.flink.util.FlinkException: Error while shutting the
> TaskExecutor down.
> at
> org.apache.flink.runtime.taskexecutor.TaskExecutor.handleOnStopException(TaskExecutor.java:397)
> at
> org.apache.flink.runtime.taskexecutor.TaskExecutor.lambda$onStop$0(TaskExecutor.java:382)
> at
> java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822)
> at
> java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797)
> at
>
[GitHub] [flink] zhuzhurk commented on issue #10672: [FLINK-15373][core][config] Update descriptions for framework / task off-heap memory config options
zhuzhurk commented on issue #10672: [FLINK-15373][core][config] Update descriptions for framework / task off-heap memory config options URL: https://github.com/apache/flink/pull/10672#issuecomment-568846147 @xintongsong #10673 is merged. Would you rebase this PR onto master? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] azagrebin opened a new pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
azagrebin opened a new pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping URL: https://github.com/apache/flink/pull/10682 ## What is the purpose of the change #10161 introduced canceling and waiting of all tasks in TM to release resources and exit before shutting down TM services on stop. #10034 assigned a separate memory manager per slot and #10330 introduced dynamic slots and state to `TaskSlotTable`. To achieve clean lifecycle of slot, table and TM, this PR introduces freeing of all slots before shutting down TM services and stopping slot table on TM stop. Freeing of all slots cancels the running tasks and releases slot resources, including memory manager. ## Brief change log - Introduce closing future to `TaskSlot` - Introduce `TaskSlotTable#freeAllSlots` - Adjust task executor tests - Add more test coverage for `TaskSlot` and `TaskSlotTable` ## Verifying this change CI unit tests ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] azagrebin commented on issue #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
azagrebin commented on issue #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping URL: https://github.com/apache/flink/pull/10682#issuecomment-568846213 cc @xintongsong This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[
https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003090#comment-17003090
]
hiliuxg commented on FLINK-15388:
-
Hi,Xintong Song:
Thanks for your patience，the related logs has shared，but it seems that nothing
can be find
2019-12-25 07:53:08.887 [flink-akka.actor.default-dispatcher-71] INFO
org.apache.flink.runtime.taskmanager.Task - Source: Custom Source -> Filter ->
from: (defaultTime, defaultOffset, defaultTopic, defaultPartition, actionTime,
actionId, platformId, channelId, kugouId, fanxId, actorId, roomId, isFollower,
p1, p2, p3, gatewayTime) -> where: (AND(=(actionId,
_UTF-16LE'fx_listpg_rm_exit_room'), IsValUserId(kugouId), IsValUserId(actorId),
>=(CAST(JsonValue(p3, _UTF-16LE'dur')), 10))), select:
(CONCAT(_UTF-16LE'kg.live.online.feature.userstar.quittmies10sec.past1day.flink:',
kugouId, _UTF-16LE':', actorId) AS k, gatewayTime AS t) -> to: Tuple2 (1/1)
(9c8d658fff9b759985e3402128feb720) switched from RUNNING to FAILED.
org.apache.flink.util.FlinkException: The assigned slot
bae00218c818157649eb9e3c533b86af_35 was removed.
at
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at
org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at
org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at
akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
~[flink-dist_2.11-1.8.0.jar:1.8.0]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
[flink-dist_2.11-1.8.0.jar:1.8.0]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
[flink-dist_2.11-1.8.0.jar:1.8.0]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
[flink-dist_2.11-1.8.0.jar:1.8.0]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
[flink-dist_2.11-1.8.0.jar:1.8.0]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
[flink-dist_2.11-1.8.0.jar:1.8.0]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
[flink-dist_2.11-1.8.0.jar:1.8.0]
at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
[flink-dist_2.11-1.8.0.jar:1.8.0]
at
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
[flink-dist_2.11-1.8.0.jar:1.8.0]
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[flink-dist_2.11-1.8.0.jar:1.8.0]
> The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
> --
>
> Key: FLINK-15388
> URL: https://issues.apache.org/jira/browse/FLINK-15388
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.8.0
> Environment: model : standalone,not yarn
> version : flink 1.8.0
> configration :
> jobmanager.heap.size: 4096m
> taskmanager.heap.size: 144gb
> taskmanager.numberOfTaskSlots: 48
> taskmanager.memory.fraction: 0.7
> taskmanager.memory.off-heap: false
> parallelism.default: 1
>
>Reporter: hiliuxg
>Priority: Major
>
> the taskmanager's slot was removed , there was not full gc or oom , what's
> the problem ? the error bellow
> {code:java}
>
[jira] [Closed] (FLINK-15374) Update descriptions for jvm overhead config options
[ https://issues.apache.org/jira/browse/FLINK-15374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-15374. --- Resolution: Fixed Fixed via master: e9c8282becb947ec230893b3e212153118ffc7b1 release-1.10: d9d75d8b0f561338e71774fb9c51ca96536d7834 > Update descriptions for jvm overhead config options > --- > > Key: FLINK-15374 > URL: https://issues.apache.org/jira/browse/FLINK-15374 > Project: Flink > Issue Type: Improvement > Components: Runtime / Configuration >Reporter: Xintong Song >Assignee: Xintong Song >Priority: Blocker > Labels: pull-request-available > Fix For: 1.10.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update descriptions for "taskmanager.memory.jvm-overhead.[min|max|fraction]" > to remove "I/O direct memory" and explicitly state that it's not counted into > MaxDirectMemorySize. > Detailed discussion can be found in this [ML > thread|http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Some-feedback-after-trying-out-the-new-FLIP-49-memory-configurations-td36129.html]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] zhuzhurk merged pull request #10673: [FLINK-15374][core][config] Update descriptions for jvm overhead config options
zhuzhurk merged pull request #10673: [FLINK-15374][core][config] Update descriptions for jvm overhead config options URL: https://github.com/apache/flink/pull/10673 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhuzhurk commented on issue #10673: [FLINK-15374][core][config] Update descriptions for jvm overhead config options
zhuzhurk commented on issue #10673: [FLINK-15374][core][config] Update descriptions for jvm overhead config options URL: https://github.com/apache/flink/pull/10673#issuecomment-568845069 Thanks for opening this PR @xintongsong . LGTM. Merging. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] streaming-olap commented on a change in pull request #8751: [FLINK-11937][StateBackend]Resolve small file problem in RocksDB incremental checkpoint
streaming-olap commented on a change in pull request #8751:
[FLINK-11937][StateBackend]Resolve small file problem in RocksDB incremental
checkpoint
URL: https://github.com/apache/flink/pull/8751#discussion_r361260570
##
File path:
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/snapshot/RocksIncrementalSnapshotStrategy.java
##
@@ -154,8 +160,12 @@ public RocksIncrementalSnapshotStrategy(
final SnapshotDirectory snapshotDirectory =
prepareLocalSnapshotDirectory(checkpointId);
LOG.trace("Local RocksDB checkpoint goes to backup path {}.",
snapshotDirectory);
+ if (this.checkpointStreamFactory == null) {
+ this.checkpointStreamFactory = checkpointStreamFactory;
Review comment:
The line 175 may use this.checkpointStreamFactory replace
checkpointStreamFactory.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-15382) Flink failed generating python config docs
[
https://issues.apache.org/jira/browse/FLINK-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003088#comment-17003088
]
Dian Fu commented on FLINK-15382:
-
Hi [~xintongsong] [~karmagyz], thanks for reporting this issue. Could you help
to check if the jar of flink-python is available(and correct) in your local
maven repository? I have executed the above command and it seems everything
works well in my local machine.
> Flink failed generating python config docs
> ---
>
> Key: FLINK-15382
> URL: https://issues.apache.org/jira/browse/FLINK-15382
> Project: Flink
> Issue Type: Bug
> Components: API / Python, Runtime / Configuration
>Reporter: Xintong Song
>Priority: Major
>
> When generating config option docs with the command suggested by
> {{flink-docs/README.md}}, the generated
> {{docs/_includes/generated/python_configuration.html}} does not contain any
> config options despite that there are 4 options in {{PythonOptions}}.
> I encountered this problem at the commit
> {{545534e43ed37f518fe59b6ddd8ed56ae82a234b}} on master branch.
> Command used to generate doc:
> {code:bash}mvn package -Dgenerate-config-docs -pl flink-docs -am -nsu
> -DskipTests{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10681: [FLINK-14849][hive][doc] Fix documentation about Hive dependencies
flinkbot edited a comment on issue #10681: [FLINK-14849][hive][doc] Fix documentation about Hive dependencies URL: https://github.com/apache/flink/pull/10681#issuecomment-568831538 ## CI report: * 4347bedaf5a20884347af39baf197932238c6d24 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142300202) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3900) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10679: [hotfix][runtime] Remove unused IOManager parameter while constructing BufferStorage
flinkbot edited a comment on issue #10679: [hotfix][runtime] Remove unused IOManager parameter while constructing BufferStorage URL: https://github.com/apache/flink/pull/10679#issuecomment-568831518 ## CI report: * d14d86ba9ca471a1ee135342d52c38cb43b33136 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142300181) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3898) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10680: [FLINK-15125][table-planner-blink] PROCTIME() computed column defined…
flinkbot edited a comment on issue #10680: [FLINK-15125][table-planner-blink] PROCTIME() computed column defined… URL: https://github.com/apache/flink/pull/10680#issuecomment-568831525 ## CI report: * a94add4b3b5dca61b1083575dfa5f2e34d50295c Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142300188) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3899) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis
flinkbot edited a comment on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis URL: https://github.com/apache/flink/pull/10675#issuecomment-568700654 ## CI report: * 9840f455797dc9fba5d17aa9a9e1375cb8a41768 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142206005) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3882) * 51b3ef890acc3b206a329f2a678d127c10d25e56 Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/142299409) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3896) * 049a265d3a4346dc5ec05f787e0751bfdf695cfa Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142300171) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3897) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10677: [FLINK-15371] Change FLIP-49 memory configurations to use the new memory type config options
flinkbot edited a comment on issue #10677: [FLINK-15371] Change FLIP-49 memory configurations to use the new memory type config options URL: https://github.com/apache/flink/pull/10677#issuecomment-568707829 ## CI report: * b7dc5e6e30b140815530d04cb8898efd1b9cf19c Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142210122) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3885) * f02488f343aabe6a27a41674c1b99225ed7fc16b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142212407) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3887) * 6f5e783884428632d0e3e28867ec05ad726037f4 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142298614) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3895) * de9eb49fc5b0f9c2a54f976d63e2800629e44062 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142300973) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3901) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (FLINK-15377) Mesos WordCount test fails on travis
[
https://issues.apache.org/jira/browse/FLINK-15377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu closed FLINK-15377.
---
Resolution: Fixed
> Mesos WordCount test fails on travis
>
>
> Key: FLINK-15377
> URL: https://issues.apache.org/jira/browse/FLINK-15377
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Mesos
>Affects Versions: 1.10.0
>Reporter: Yu Li
>Assignee: Yangze Guo
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The "Run Mesos WordCount test" fails nightly run on travis with below error:
> {code}
> rm: cannot remove
> '/home/travis/build/apache/flink/flink-end-to-end-tests/test-scripts/test-data/log/mesos-sl/mesos-slave.INFO':
> Permission denied
> rm: cannot remove
> '/home/travis/build/apache/flink/flink-end-to-end-tests/test-scripts/test-data/log/mesos-sl/mesos-fetcher.INFO':
> Permission denied
> rm: cannot remove
> '/home/travis/build/apache/flink/flink-end-to-end-tests/test-scripts/test-data/log/mesos-sl/mesos-slave.4a4fda410c57.invalid-user.log.INFO.20191224-031307.1':
> Permission denied
> ...
> [FAIL] 'Run Mesos WordCount test' failed after 5 minutes and 26 seconds! Test
> exited with exit code 0 but the logs contained errors, exceptions or
> non-empty .out files
> {code}
> https://api.travis-ci.org/v3/job/628795106/log.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15377) Mesos WordCount test fails on travis
[
https://issues.apache.org/jira/browse/FLINK-15377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003085#comment-17003085
]
Zhu Zhu commented on FLINK-15377:
-
Fixed via
master:
3036dd97eef4d1451baf6984885ad3784333a617
1.10.0:
d3eb451c2551ffa928bb04c8445c0967af69da60
> Mesos WordCount test fails on travis
>
>
> Key: FLINK-15377
> URL: https://issues.apache.org/jira/browse/FLINK-15377
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Mesos
>Affects Versions: 1.10.0
>Reporter: Yu Li
>Assignee: Yangze Guo
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The "Run Mesos WordCount test" fails nightly run on travis with below error:
> {code}
> rm: cannot remove
> '/home/travis/build/apache/flink/flink-end-to-end-tests/test-scripts/test-data/log/mesos-sl/mesos-slave.INFO':
> Permission denied
> rm: cannot remove
> '/home/travis/build/apache/flink/flink-end-to-end-tests/test-scripts/test-data/log/mesos-sl/mesos-fetcher.INFO':
> Permission denied
> rm: cannot remove
> '/home/travis/build/apache/flink/flink-end-to-end-tests/test-scripts/test-data/log/mesos-sl/mesos-slave.4a4fda410c57.invalid-user.log.INFO.20191224-031307.1':
> Permission denied
> ...
> [FAIL] 'Run Mesos WordCount test' failed after 5 minutes and 26 seconds! Test
> exited with exit code 0 but the logs contained errors, exceptions or
> non-empty .out files
> {code}
> https://api.travis-ci.org/v3/job/628795106/log.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] zhuzhurk merged pull request #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis
zhuzhurk merged pull request #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis URL: https://github.com/apache/flink/pull/10675 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-15384) Allow to pass ExecutionEnvironment/StreamExecutionEnvironment to TableEnvironment
[
https://issues.apache.org/jira/browse/FLINK-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003080#comment-17003080
]
Jeff Zhang edited comment on FLINK-15384 at 12/25/19 5:31 AM:
--
[~ykt836] I can do it for StreamTableEnvironment, but there's no such api for
BatchTableEnvironment.
And I guess using TableEnvironment.create is the preferred way.
BTW, the document of creating TableEnvironment is really very confusing.
There're so many different approaches for creating different TableEnvironment
(flink/blink planner, stream/batch)
https://ci.apache.org/projects/flink/flink-docs-master/dev/table/common.html#create-a-tableenvironment
was (Author: zjffdu):
[~ykt836] I can do for StreamTableEnvironment, but there's no such api for
BatchTableEnvironment.
And I guess using TableEnvironment.create is the preferred way.
BTW, the document of creating TableEnvironment is really very confusing.
There're so many different approaches for creating different TableEnvironment
(flink/blink planner, stream/batch)
https://ci.apache.org/projects/flink/flink-docs-master/dev/table/common.html#create-a-tableenvironment
> Allow to pass ExecutionEnvironment/StreamExecutionEnvironment to
> TableEnvironment
> --
>
> Key: FLINK-15384
> URL: https://issues.apache.org/jira/browse/FLINK-15384
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
>Affects Versions: 1.10.0
>Reporter: Jeff Zhang
>Priority: Major
>
> In the new approach of creating TableEnvironement, it is not possible to pass
> ExecutionEnvironment/StreamExecutionEnvironment to TableEnvironement. This
> cause the all the features in ExecutionEnvironment/StreamExecutionEnvironment
> is not available in table api. Such as add JobListener, execute Job async. So
> I suggest to allow to pass ExecutionEnvironment/StreamExecutionEnvironment to
> TableEnvironment
> {code}
> EnvironmentSettings bbSettings =
> EnvironmentSettings.newInstance().useBlinkPlanner().inBatchMode().build();
> TableEnvironment bbTableEnv = TableEnvironment.create(bbSettings);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15384) Allow to pass ExecutionEnvironment/StreamExecutionEnvironment to TableEnvironment
[
https://issues.apache.org/jira/browse/FLINK-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003080#comment-17003080
]
Jeff Zhang commented on FLINK-15384:
[~ykt836] I can do for StreamTableEnvironment, but there's no such api for
BatchTableEnvironment.
And I guess using TableEnvironment.create is the preferred way.
BTW, the document of creating TableEnvironment is really very confusing.
There're so many different approaches for creating different TableEnvironment
(flink/blink planner, stream/batch)
https://ci.apache.org/projects/flink/flink-docs-master/dev/table/common.html#create-a-tableenvironment
> Allow to pass ExecutionEnvironment/StreamExecutionEnvironment to
> TableEnvironment
> --
>
> Key: FLINK-15384
> URL: https://issues.apache.org/jira/browse/FLINK-15384
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
>Affects Versions: 1.10.0
>Reporter: Jeff Zhang
>Priority: Major
>
> In the new approach of creating TableEnvironement, it is not possible to pass
> ExecutionEnvironment/StreamExecutionEnvironment to TableEnvironement. This
> cause the all the features in ExecutionEnvironment/StreamExecutionEnvironment
> is not available in table api. Such as add JobListener, execute Job async. So
> I suggest to allow to pass ExecutionEnvironment/StreamExecutionEnvironment to
> TableEnvironment
> {code}
> EnvironmentSettings bbSettings =
> EnvironmentSettings.newInstance().useBlinkPlanner().inBatchMode().build();
> TableEnvironment bbTableEnv = TableEnvironment.create(bbSettings);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] JingsongLi commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] TM Metaspace memory leak
JingsongLi commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] TM Metaspace memory leak URL: https://github.com/apache/flink/pull/10620#discussion_r361257891 ## File path: flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/generated/CompileUtils.java ## @@ -45,7 +48,7 @@ * number of Meta zone GC (class unloading), resulting in performance bottlenecks. So we add * a cache to avoid this problem. */ - protected static final Cache, Class> COMPILED_CACHE = CacheBuilder + protected static final Cache> COMPILED_CACHE = CacheBuilder .newBuilder() .maximumSize(100) // estimated cache size Review comment: Why cache code? For what? Can you see above comments in `CompileUtils.COMPILED_CACHE`, we cache classes for performance and GC. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis
flinkbot edited a comment on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis URL: https://github.com/apache/flink/pull/10675#issuecomment-568700654 ## CI report: * 9840f455797dc9fba5d17aa9a9e1375cb8a41768 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142206005) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3882) * 51b3ef890acc3b206a329f2a678d127c10d25e56 Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/142299409) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3896) * 049a265d3a4346dc5ec05f787e0751bfdf695cfa Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142300171) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3897) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10680: [FLINK-15125][table-planner-blink] PROCTIME() computed column defined…
flinkbot edited a comment on issue #10680: [FLINK-15125][table-planner-blink] PROCTIME() computed column defined… URL: https://github.com/apache/flink/pull/10680#issuecomment-568831525 ## CI report: * a94add4b3b5dca61b1083575dfa5f2e34d50295c Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142300188) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3899) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10679: [hotfix][runtime] Remove unused IOManager parameter while constructing BufferStorage
flinkbot edited a comment on issue #10679: [hotfix][runtime] Remove unused IOManager parameter while constructing BufferStorage URL: https://github.com/apache/flink/pull/10679#issuecomment-568831518 ## CI report: * d14d86ba9ca471a1ee135342d52c38cb43b33136 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142300181) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3898) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10677: [FLINK-15371] Change FLIP-49 memory configurations to use the new memory type config options
flinkbot edited a comment on issue #10677: [FLINK-15371] Change FLIP-49 memory configurations to use the new memory type config options URL: https://github.com/apache/flink/pull/10677#issuecomment-568707829 ## CI report: * b7dc5e6e30b140815530d04cb8898efd1b9cf19c Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142210122) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3885) * f02488f343aabe6a27a41674c1b99225ed7fc16b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142212407) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3887) * 6f5e783884428632d0e3e28867ec05ad726037f4 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142298614) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3895) * de9eb49fc5b0f9c2a54f976d63e2800629e44062 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/142300973) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3901) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] KarmaGYZ commented on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis
KarmaGYZ commented on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis URL: https://github.com/apache/flink/pull/10675#issuecomment-568835108 Relevant e2e tests pass [here](https://travis-ci.org/KarmaGYZ/flink/builds/629276303) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10679: [hotfix][runtime] Remove unused IOManager parameter while constructing BufferStorage
flinkbot edited a comment on issue #10679: [hotfix][runtime] Remove unused IOManager parameter while constructing BufferStorage URL: https://github.com/apache/flink/pull/10679#issuecomment-568831518 ## CI report: * d14d86ba9ca471a1ee135342d52c38cb43b33136 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/142300181) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3898) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10680: [FLINK-15125][table-planner-blink] PROCTIME() computed column defined…
flinkbot edited a comment on issue #10680: [FLINK-15125][table-planner-blink] PROCTIME() computed column defined… URL: https://github.com/apache/flink/pull/10680#issuecomment-568831525 ## CI report: * a94add4b3b5dca61b1083575dfa5f2e34d50295c Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/142300188) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3899) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10681: [FLINK-14849][hive][doc] Fix documentation about Hive dependencies
flinkbot edited a comment on issue #10681: [FLINK-14849][hive][doc] Fix documentation about Hive dependencies URL: https://github.com/apache/flink/pull/10681#issuecomment-568831538 ## CI report: * 4347bedaf5a20884347af39baf197932238c6d24 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/142300202) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3900) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis
flinkbot edited a comment on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis URL: https://github.com/apache/flink/pull/10675#issuecomment-568700654 ## CI report: * 9840f455797dc9fba5d17aa9a9e1375cb8a41768 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142206005) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3882) * 51b3ef890acc3b206a329f2a678d127c10d25e56 Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/142299409) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3896) * 049a265d3a4346dc5ec05f787e0751bfdf695cfa Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/142300171) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3897) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10677: [FLINK-15371] Change FLIP-49 memory configurations to use the new memory type config options
flinkbot edited a comment on issue #10677: [FLINK-15371] Change FLIP-49 memory configurations to use the new memory type config options URL: https://github.com/apache/flink/pull/10677#issuecomment-568707829 ## CI report: * b7dc5e6e30b140815530d04cb8898efd1b9cf19c Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142210122) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3885) * f02488f343aabe6a27a41674c1b99225ed7fc16b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142212407) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3887) * 6f5e783884428632d0e3e28867ec05ad726037f4 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142298614) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3895) * de9eb49fc5b0f9c2a54f976d63e2800629e44062 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] danny0405 commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] TM Metaspace memory leak
danny0405 commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] TM Metaspace memory leak URL: https://github.com/apache/flink/pull/10620#discussion_r361251365 ## File path: flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/generated/CompileUtils.java ## @@ -45,7 +48,7 @@ * number of Meta zone GC (class unloading), resulting in performance bottlenecks. So we add * a cache to avoid this problem. */ - protected static final Cache, Class> COMPILED_CACHE = CacheBuilder + protected static final Cache> COMPILED_CACHE = CacheBuilder .newBuilder() .maximumSize(100) // estimated cache size Review comment: Why couldn't we just cache the code not the class ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[
https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003067#comment-17003067
]
Xintong Song commented on FLINK-15388:
--
Could you share the complete log file?
> The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
> --
>
> Key: FLINK-15388
> URL: https://issues.apache.org/jira/browse/FLINK-15388
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.8.0
> Environment: model : standalone,not yarn
> version : flink 1.8.0
> configration :
> jobmanager.heap.size: 4096m
> taskmanager.heap.size: 144gb
> taskmanager.numberOfTaskSlots: 48
> taskmanager.memory.fraction: 0.7
> taskmanager.memory.off-heap: false
> parallelism.default: 1
>
>Reporter: hiliuxg
>Priority: Major
>
> the taskmanager's slot was removed , there was not full gc or oom , what's
> the problem ? the error bellow
> {code:java}
> org.apache.flink.util.FlinkException: The assigned slot
> bae00218c818157649eb9e3c533b86af_32 was removed.
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)
> at
> akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
> at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
> at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-15389) JDBCUpsertOutputFormat no need to create schedule flush when flushMaxSize = 1
xiaodao created FLINK-15389: --- Summary: JDBCUpsertOutputFormat no need to create schedule flush when flushMaxSize = 1 Key: FLINK-15389 URL: https://issues.apache.org/jira/browse/FLINK-15389 Project: Flink Issue Type: Improvement Components: Connectors / JDBC Affects Versions: 1.9.1 Reporter: xiaodao when flushMaxSize set 1; then it will call fush func every time; it's no need to create schedule to flush data. if (flushIntervalMills != 0) it's better to modify with if (flushIntervalMills != 0 and flushMaxSize != 1) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-15382) Flink failed generating python config docs
[
https://issues.apache.org/jira/browse/FLINK-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003066#comment-17003066
]
Yangze Guo commented on FLINK-15382:
Another instance in my environment.
> Flink failed generating python config docs
> ---
>
> Key: FLINK-15382
> URL: https://issues.apache.org/jira/browse/FLINK-15382
> Project: Flink
> Issue Type: Bug
> Components: API / Python, Runtime / Configuration
>Reporter: Xintong Song
>Priority: Major
>
> When generating config option docs with the command suggested by
> {{flink-docs/README.md}}, the generated
> {{docs/_includes/generated/python_configuration.html}} does not contain any
> config options despite that there are 4 options in {{PythonOptions}}.
> I encountered this problem at the commit
> {{545534e43ed37f518fe59b6ddd8ed56ae82a234b}} on master branch.
> Command used to generate doc:
> {code:bash}mvn package -Dgenerate-config-docs -pl flink-docs -am -nsu
> -DskipTests{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[
https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003064#comment-17003064
]
hiliuxg commented on FLINK-15388:
-
Hi xintong Song,
The TM is not failed also the heap is not full .
Looking closely at the stack, it seems that JM cannot successfully close TM, or
that JM and TM communication timed out
> The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
> --
>
> Key: FLINK-15388
> URL: https://issues.apache.org/jira/browse/FLINK-15388
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.8.0
> Environment: model : standalone,not yarn
> version : flink 1.8.0
> configration :
> jobmanager.heap.size: 4096m
> taskmanager.heap.size: 144gb
> taskmanager.numberOfTaskSlots: 48
> taskmanager.memory.fraction: 0.7
> taskmanager.memory.off-heap: false
> parallelism.default: 1
>
>Reporter: hiliuxg
>Priority: Major
>
> the taskmanager's slot was removed , there was not full gc or oom , what's
> the problem ? the error bellow
> {code:java}
> org.apache.flink.util.FlinkException: The assigned slot
> bae00218c818157649eb9e3c533b86af_32 was removed.
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)
> at
> akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
> at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
> at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot commented on issue #10680: [FLINK-15125][table-planner-blink] PROCTIME() computed column defined…
flinkbot commented on issue #10680: [FLINK-15125][table-planner-blink] PROCTIME() computed column defined… URL: https://github.com/apache/flink/pull/10680#issuecomment-568831525 ## CI report: * a94add4b3b5dca61b1083575dfa5f2e34d50295c UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10681: [FLINK-14849][hive][doc] Fix documentation about Hive dependencies
flinkbot commented on issue #10681: [FLINK-14849][hive][doc] Fix documentation about Hive dependencies URL: https://github.com/apache/flink/pull/10681#issuecomment-568831538 ## CI report: * 4347bedaf5a20884347af39baf197932238c6d24 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10679: [hotfix][runtime] Remove unused IOManager parameter while constructing BufferStorage
flinkbot commented on issue #10679: [hotfix][runtime] Remove unused IOManager parameter while constructing BufferStorage URL: https://github.com/apache/flink/pull/10679#issuecomment-568831518 ## CI report: * d14d86ba9ca471a1ee135342d52c38cb43b33136 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis
flinkbot edited a comment on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis URL: https://github.com/apache/flink/pull/10675#issuecomment-568700654 ## CI report: * 9840f455797dc9fba5d17aa9a9e1375cb8a41768 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142206005) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3882) * 51b3ef890acc3b206a329f2a678d127c10d25e56 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/142299409) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3896) * 049a265d3a4346dc5ec05f787e0751bfdf695cfa UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10677: [FLINK-15371] Change FLIP-49 memory configurations to use the new memory type config options
flinkbot edited a comment on issue #10677: [FLINK-15371] Change FLIP-49 memory configurations to use the new memory type config options URL: https://github.com/apache/flink/pull/10677#issuecomment-568707829 ## CI report: * b7dc5e6e30b140815530d04cb8898efd1b9cf19c Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142210122) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3885) * f02488f343aabe6a27a41674c1b99225ed7fc16b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142212407) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3887) * 6f5e783884428632d0e3e28867ec05ad726037f4 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142298614) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3895) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] KarmaGYZ commented on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis
KarmaGYZ commented on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis URL: https://github.com/apache/flink/pull/10675#issuecomment-568830796 > Thanks for opening this PR @KarmaGYZ . > This change generally looks good to me. I just think we should enrich the commit message for the 2 hotfix commits to describe why we need those changes. Thanks for the review. I have updated the PR description. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15385) Translate "SQL" page of Table API into Chinese
[ https://issues.apache.org/jira/browse/FLINK-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003061#comment-17003061 ] Steve OU commented on FLINK-15385: -- The ticket title was wrong. Already corrected it. :) > Translate "SQL" page of Table API into Chinese > -- > > Key: FLINK-15385 > URL: https://issues.apache.org/jira/browse/FLINK-15385 > Project: Flink > Issue Type: Task > Components: chinese-translation >Reporter: Steve OU >Assignee: Steve OU >Priority: Minor > > The page url is > https://ci.apache.org/projects/flink/flink-docs-master/zh/dev/table/sql.html > The markdown file is located in flink/docs/dev/table/sql.zh.md -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] JingsongLi commented on issue #10681: [FLINK-14849][hive][doc] Fix documentation about Hive dependencies
JingsongLi commented on issue #10681: [FLINK-14849][hive][doc] Fix documentation about Hive dependencies URL: https://github.com/apache/flink/pull/10681#issuecomment-568830579 I think this is pretty good, can you build web-site and show the picture? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15385) Translate "SQL" page of Table API into Chinese
[ https://issues.apache.org/jira/browse/FLINK-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve OU updated FLINK-15385: - Summary: Translate "SQL" page of Table API into Chinese (was: Translate "Concepts & Common API" page of Table API into Chinese) > Translate "SQL" page of Table API into Chinese > -- > > Key: FLINK-15385 > URL: https://issues.apache.org/jira/browse/FLINK-15385 > Project: Flink > Issue Type: Task > Components: chinese-translation >Reporter: Steve OU >Assignee: Steve OU >Priority: Minor > > The page url is > https://ci.apache.org/projects/flink/flink-docs-master/zh/dev/table/sql.html > The markdown file is located in flink/docs/dev/table/sql.zh.md -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] zhuzhurk commented on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis
zhuzhurk commented on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis URL: https://github.com/apache/flink/pull/10675#issuecomment-568830424 Thanks for opening this PR @KarmaGYZ . This change generally looks good to me. I just think we should enrich the commit message for the 2 hotfix commits to describe why we need those changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] JingsongLi commented on issue #10667: [FLINK-15313][table] Fix can't insert decimal data into sink using TypeInformation
JingsongLi commented on issue #10667: [FLINK-15313][table] Fix can't insert decimal data into sink using TypeInformation URL: https://github.com/apache/flink/pull/10667#issuecomment-568830177 You can fix failure cases too. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (FLINK-15385) Translate "Concepts & Common API" page of Table API into Chinese
[ https://issues.apache.org/jira/browse/FLINK-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu reassigned FLINK-15385: --- Assignee: Steve OU > Translate "Concepts & Common API" page of Table API into Chinese > > > Key: FLINK-15385 > URL: https://issues.apache.org/jira/browse/FLINK-15385 > Project: Flink > Issue Type: Task > Components: chinese-translation >Reporter: Steve OU >Assignee: Steve OU >Priority: Minor > > The page url is > https://ci.apache.org/projects/flink/flink-docs-master/zh/dev/table/sql.html > The markdown file is located in flink/docs/dev/table/sql.zh.md -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] JingsongLi commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] TM Metaspace memory leak
JingsongLi commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] TM Metaspace memory leak URL: https://github.com/apache/flink/pull/10620#discussion_r361248432 ## File path: flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/generated/CompileUtils.java ## @@ -45,7 +48,7 @@ * number of Meta zone GC (class unloading), resulting in performance bottlenecks. So we add * a cache to avoid this problem. */ - protected static final Cache, Class> COMPILED_CACHE = CacheBuilder + protected static final Cache> COMPILED_CACHE = CacheBuilder .newBuilder() .maximumSize(100) // estimated cache size Review comment: First, different tasks of one Job maybe have different class loader, so yes, same code maybe have multiple Java Classes. Second, same sql run in different client maybe generate same codes too. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on issue #10667: [FLINK-15313][table] Fix can't insert decimal data into sink using TypeInformation
wuchong commented on issue #10667: [FLINK-15313][table] Fix can't insert decimal data into sink using TypeInformation URL: https://github.com/apache/flink/pull/10667#issuecomment-568830017 FLINK-15381 is not a blocker to this PR, that is an independent issue but should be fixed before 1.10 release. I will verify the relative issues in my local machine. Thanks for the reminder. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15381) INSERT INTO VALUES statement fails if a cast project is applied
[
https://issues.apache.org/jira/browse/FLINK-15381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-15381:
Priority: Blocker (was: Critical)
> INSERT INTO VALUES statement fails if a cast project is applied
> ---
>
> Key: FLINK-15381
> URL: https://issues.apache.org/jira/browse/FLINK-15381
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Reporter: Jark Wu
>Assignee: godfrey he
>Priority: Blocker
> Fix For: 1.10.0
>
>
> The following query will fail:
> {code:scala}
> @Test
> def test(): Unit = {
> val sinkDDL =
> """
> |create table t2(
> | a int,
> | b string
> |) with (
> | 'connector' = 'COLLECTION'
> |)
> """.stripMargin
> val query =
> """
> |insert into t2 select cast(a as int), cast(b as varchar) from
> (values (3, 'c')) T(a,b)
> """.stripMargin
> tableEnv.sqlUpdate(sinkDDL)
> tableEnv.sqlUpdate(query)
> execJob("testJob")
> }
> {code}
> exception:
> {code}
> org.apache.flink.table.api.TableException: Cannot generate a valid execution
> plan for the given query:
> LogicalSink(name=[`default_catalog`.`default_database`.`t2`], fields=[a, b])
> +- LogicalProject(EXPR$0=[$0], EXPR$1=[CAST($1):VARCHAR(2147483647) CHARACTER
> SET "UTF-16LE" NOT NULL])
>+- LogicalValues(type=[RecordType(INTEGER a, CHAR(1) b)], tuples=[[{ 3,
> _UTF-16LE'c' }]])
> This exception indicates that the query uses an unsupported SQL feature.
> Please check the documentation for the set of currently supported SQL
> features.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[
https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003057#comment-17003057
]
Xintong Song commented on FLINK-15388:
--
Hi [~hiliuxg],
The exception "slot was removed" usually indicates that the corresponding task
manager failed. We need to look into the task manager log to find out why it
failed.
> The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
> --
>
> Key: FLINK-15388
> URL: https://issues.apache.org/jira/browse/FLINK-15388
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.8.0
> Environment: model : standalone,not yarn
> version : flink 1.8.0
> configration :
> jobmanager.heap.size: 4096m
> taskmanager.heap.size: 144gb
> taskmanager.numberOfTaskSlots: 48
> taskmanager.memory.fraction: 0.7
> taskmanager.memory.off-heap: false
> parallelism.default: 1
>
>Reporter: hiliuxg
>Priority: Major
>
> the taskmanager's slot was removed , there was not full gc or oom , what's
> the problem ? the error bellow
> {code:java}
> org.apache.flink.util.FlinkException: The assigned slot
> bae00218c818157649eb9e3c533b86af_32 was removed.
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058)
> at
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
> at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147)
> at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)
> at
> akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
> at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
> at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] JingsongLi commented on issue #10667: [FLINK-15313][table] Fix can't insert decimal data into sink using TypeInformation
JingsongLi commented on issue #10667: [FLINK-15313][table] Fix can't insert decimal data into sink using TypeInformation URL: https://github.com/apache/flink/pull/10667#issuecomment-568829609 Can you verify: https://issues.apache.org/jira/browse/FLINK-15309 https://issues.apache.org/jira/browse/FLINK-15310 https://issues.apache.org/jira/browse/FLINK-15326 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lirui-apache commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] TM Metaspace memory leak
lirui-apache commented on a change in pull request #10620: [FLINK-15239][table-planner-blink] TM Metaspace memory leak URL: https://github.com/apache/flink/pull/10620#discussion_r361248052 ## File path: flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/generated/CompileUtils.java ## @@ -45,7 +48,7 @@ * number of Meta zone GC (class unloading), resulting in performance bottlenecks. So we add * a cache to avoid this problem. */ - protected static final Cache, Class> COMPILED_CACHE = CacheBuilder + protected static final Cache> COMPILED_CACHE = CacheBuilder .newBuilder() .maximumSize(100) // estimated cache size Review comment: The origin cache uses tuple of class loader and class name as key. So yes, I guess theoretically there can be multiple class instances for the same name, otherwise we don't need to involve class loaders at all. Perhaps it can happen if we run multiple queries concurrently in one TM. @JingsongLi any insights about this? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] JingsongLi commented on issue #10667: [FLINK-15313][table] Fix can't insert decimal data into sink using TypeInformation
JingsongLi commented on issue #10667: [FLINK-15313][table] Fix can't insert decimal data into sink using TypeInformation URL: https://github.com/apache/flink/pull/10667#issuecomment-568829538 Shall https://issues.apache.org/jira/browse/FLINK-15381 is blocker to this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[
https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hiliuxg updated FLINK-15388:
Description:
the taskmanager's slot was removed , there was not full gc or oom , what's the
problem ? the error bellow
{code:java}
org.apache.flink.util.FlinkException: The assigned slot
bae00218c818157649eb9e3c533b86af_32 was removed.
at
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893)
at
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863)
at
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058)
at
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385)
at
org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847)
at
org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185)
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147)
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)
at
akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
{code}
was:
the taskmanager's slot was removed , there was not full gc or oom , what's the
problem ? the error bellow
org.apache.flink.util.FlinkException: The assigned slot
bae00218c818157649eb9e3c533b86af_32 was removed.
at
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893)
at
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863)
at
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058)
at
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385)
at
org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847)
at
org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185)
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147)
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)
at
akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
> --
>
> Key: FLINK-15388
> URL: https://issues.apache.org/jira/browse/FLINK-15388
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.8.0
> Environment: model : standalone,not yarn
>
[GitHub] [flink] wuchong commented on a change in pull request #10667: [FLINK-15313][table] Fix can't insert decimal data into sink using TypeInformation
wuchong commented on a change in pull request #10667: [FLINK-15313][table] Fix can't insert decimal data into sink using TypeInformation URL: https://github.com/apache/flink/pull/10667#discussion_r361247666 ## File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/metadata/FlinkRelMdUniqueKeys.scala ## @@ -135,8 +138,8 @@ class FlinkRelMdUniqueKeys private extends MetadataHandler[BuiltInMetadata.Uniqu } case _ => // ignore } - //rename - case a: RexCall if a.getKind.equals(SqlKind.AS) && + //rename or cast + case a: RexCall if (a.getKind.equals(SqlKind.AS) || isFidelityCast(a)) && Review comment: That's true. That's why we only pick the fidelity casts that just change the precision or nullabilities (not-null -> nullable). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[ https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hiliuxg updated FLINK-15388: Environment: model : standalone,not yarn version : flink 1.8.0 configration : jobmanager.heap.size: 4096m taskmanager.heap.size: 144gb taskmanager.numberOfTaskSlots: 48 taskmanager.memory.fraction: 0.7 taskmanager.memory.off-heap: false parallelism.default: 1 was: model : standalone,not yarn version : flink 1.8.0 configration : jobmanager.heap.size: 4096m taskmanager.heap.size: 144gb taskmanager.numberOfTaskSlots: 48 taskmanager.memory.fraction: 0.7 taskmanager.memory.off-heap: false parallelism.default: 1 > The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed. > -- > > Key: FLINK-15388 > URL: https://issues.apache.org/jira/browse/FLINK-15388 > Project: Flink > Issue Type: Bug > Components: Runtime / Task >Affects Versions: 1.8.0 > Environment: model : standalone,not yarn > version : flink 1.8.0 > configration : > jobmanager.heap.size: 4096m > taskmanager.heap.size: 144gb > taskmanager.numberOfTaskSlots: 48 > taskmanager.memory.fraction: 0.7 > taskmanager.memory.off-heap: false > parallelism.default: 1 > >Reporter: hiliuxg >Priority: Major > > the taskmanager's slot was removed , there was not full gc or oom , what's > the problem ? the error bellow > org.apache.flink.util.FlinkException: The assigned slot > bae00218c818157649eb9e3c533b86af_32 was removed. > at > org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893) > at > org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863) > at > org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058) > at > org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385) > at > org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847) > at > org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161) > at > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392) > at > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185) > at > org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74) > at > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147) > at > org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40) > at > akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165) > at akka.actor.Actor$class.aroundReceive(Actor.scala:502) > at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95) > at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526) > at akka.actor.ActorCell.invoke(ActorCell.scala:495) > at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) > at akka.dispatch.Mailbox.run(Mailbox.scala:224) > at akka.dispatch.Mailbox.exec(Mailbox.scala:234) > at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > at > scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > at > scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-15388) The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed.
[ https://issues.apache.org/jira/browse/FLINK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hiliuxg updated FLINK-15388: Environment: model : standalone,not yarn version : flink 1.8.0 configration : jobmanager.heap.size: 4096m taskmanager.heap.size: 144gb taskmanager.numberOfTaskSlots: 48 taskmanager.memory.fraction: 0.7 taskmanager.memory.off-heap: false parallelism.default: 1 was: version : flink 1.8.0 configration : jobmanager.heap.size: 4096m taskmanager.heap.size: 144gb taskmanager.numberOfTaskSlots: 48 taskmanager.memory.fraction: 0.7 taskmanager.memory.off-heap: false parallelism.default: 1 > The assigned slot bae00218c818157649eb9e3c533b86af_32 was removed. > -- > > Key: FLINK-15388 > URL: https://issues.apache.org/jira/browse/FLINK-15388 > Project: Flink > Issue Type: Bug > Components: Runtime / Task >Affects Versions: 1.8.0 > Environment: model : standalone,not yarn > version : flink 1.8.0 > configration : > jobmanager.heap.size: 4096m > taskmanager.heap.size: 144gb > taskmanager.numberOfTaskSlots: 48 > taskmanager.memory.fraction: 0.7 > taskmanager.memory.off-heap: false > parallelism.default: 1 > >Reporter: hiliuxg >Priority: Major > > the taskmanager's slot was removed , there was not full gc or oom , what's > the problem ? the error bellow > org.apache.flink.util.FlinkException: The assigned slot > bae00218c818157649eb9e3c533b86af_32 was removed. > at > org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:893) > at > org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:863) > at > org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:1058) > at > org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:385) > at > org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:847) > at > org.apache.flink.runtime.resourcemanager.ResourceManager$TaskManagerHeartbeatListener$1.run(ResourceManager.java:1161) > at > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:392) > at > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:185) > at > org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74) > at > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147) > at > org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40) > at > akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165) > at akka.actor.Actor$class.aroundReceive(Actor.scala:502) > at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95) > at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526) > at akka.actor.ActorCell.invoke(ActorCell.scala:495) > at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) > at akka.dispatch.Mailbox.run(Mailbox.scala:224) > at akka.dispatch.Mailbox.exec(Mailbox.scala:234) > at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > at > scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > at > scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10677: [FLINK-15371] Change FLIP-49 memory configurations to use the new memory type config options
flinkbot edited a comment on issue #10677: [FLINK-15371] Change FLIP-49 memory configurations to use the new memory type config options URL: https://github.com/apache/flink/pull/10677#issuecomment-568707829 ## CI report: * b7dc5e6e30b140815530d04cb8898efd1b9cf19c Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142210122) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3885) * f02488f343aabe6a27a41674c1b99225ed7fc16b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142212407) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3887) * 6f5e783884428632d0e3e28867ec05ad726037f4 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/142298614) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3895) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis
flinkbot edited a comment on issue #10675: [FLINK-15377][e2e] Mesos WordCount test fails on travis URL: https://github.com/apache/flink/pull/10675#issuecomment-568700654 ## CI report: * 9840f455797dc9fba5d17aa9a9e1375cb8a41768 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142206005) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3882) * 51b3ef890acc3b206a329f2a678d127c10d25e56 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
