[jira] [Commented] (FLINK-8230) NPE in OrcRowInputFormat on nested structs

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348140#comment-16348140 ] ASF GitHub Bot commented on FLINK-8230: --- Github user packet23 commented on the issue:

[jira] [Commented] (FLINK-8230) NPE in OrcRowInputFormat on nested structs

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348141#comment-16348141 ] ASF GitHub Bot commented on FLINK-8230: --- Github user packet23 closed the pull request at:

[GitHub] flink issue #5145: [FLINK-8230] [ORC] Improved mapping of Orc records to Fli...

2018-01-31 Thread packet23
Github user packet23 commented on the issue: https://github.com/apache/flink/pull/5145 @twalthr Sure. And thanks to @fhueske for fixing the existing IF. ---

[GitHub] flink pull request #5145: [FLINK-8230] [ORC] Improved mapping of Orc records...

2018-01-31 Thread packet23
Github user packet23 closed the pull request at: https://github.com/apache/flink/pull/5145 ---

[jira] [Commented] (FLINK-8360) Implement task-local state recovery

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348132#comment-16348132 ] ASF GitHub Bot commented on FLINK-8360: --- Github user tillrohrmann commented on a diff in the pull

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165275719 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/StateSnapshotContextSynchronousImpl.java --- @@ -109,14 +109,20 @@ public

[jira] [Commented] (FLINK-8533) Support MasterTriggerRestoreHook state reinitialization

2018-01-31 Thread Eron Wright (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348017#comment-16348017 ] Eron Wright commented on FLINK-8533: - Incidentally, a variation on this problem would (I believe)

[jira] [Comment Edited] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

2018-01-31 Thread yanxiaobin (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347951#comment-16347951 ] yanxiaobin edited comment on FLINK-8500 at 2/1/18 3:28 AM: --- Greate!I think it's

[jira] [Commented] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347953#comment-16347953 ] zhu.qing commented on FLINK-8534: - Just set Xms to 1g will reproduce the bug > if insert too much

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

2018-01-31 Thread yanxiaobin (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347951#comment-16347951 ] yanxiaobin commented on FLINK-8500: --- Greate!I think it's reasonable.But I think it's best to optimize

[jira] [Commented] (FLINK-6160) Retry JobManager/ResourceManager connection in case of timeout

2018-01-31 Thread mingleizhang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347928#comment-16347928 ] mingleizhang commented on FLINK-6160: - Thanks [~till.rohrmann] I will take a look on what you said. >

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu.qing updated FLINK-8534: Description: When insert too much entry into bucket (MutableHashTable insertBucketEntry() line 1054 more

[jira] [Commented] (FLINK-8357) enable rolling in default log settings

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347921#comment-16347921 ] ASF GitHub Bot commented on FLINK-8357: --- Github user zhangminglei commented on the issue:

[GitHub] flink issue #5371: [FLINK-8357] [conf] Enable rolling in default log setting...

2018-01-31 Thread zhangminglei
Github user zhangminglei commented on the issue: https://github.com/apache/flink/pull/5371 Changed! ---

[jira] [Commented] (FLINK-8357) enable rolling in default log settings

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347909#comment-16347909 ] ASF GitHub Bot commented on FLINK-8357: --- Github user zhangminglei commented on a diff in the pull

[jira] [Commented] (FLINK-8357) enable rolling in default log settings

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347910#comment-16347910 ] ASF GitHub Bot commented on FLINK-8357: --- Github user zhangminglei commented on a diff in the pull

[GitHub] flink pull request #5371: [FLINK-8357] [conf] Enable rolling in default log ...

2018-01-31 Thread zhangminglei
Github user zhangminglei commented on a diff in the pull request: https://github.com/apache/flink/pull/5371#discussion_r165244013 --- Diff: flink-dist/src/main/flink-bin/conf/logback-yarn.xml --- @@ -17,8 +17,14 @@ --> - + ${log.file}

[GitHub] flink pull request #5371: [FLINK-8357] [conf] Enable rolling in default log ...

2018-01-31 Thread zhangminglei
Github user zhangminglei commented on a diff in the pull request: https://github.com/apache/flink/pull/5371#discussion_r165244025 --- Diff: flink-dist/src/main/flink-bin/conf/logback.xml --- @@ -17,8 +17,14 @@ --> - + ${log.file}

[jira] [Assigned] (FLINK-8247) Support Hadoop-free variant of Flink on Mesos

2018-01-31 Thread Eron Wright (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eron Wright reassigned FLINK-8247: --- Assignee: Eron Wright > Support Hadoop-free variant of Flink on Mesos >

[jira] [Created] (FLINK-8541) Mesos RM should recover from failover timeout

2018-01-31 Thread Eron Wright (JIRA)
Eron Wright created FLINK-8541: --- Summary: Mesos RM should recover from failover timeout Key: FLINK-8541 URL: https://issues.apache.org/jira/browse/FLINK-8541 Project: Flink Issue Type: Bug

[jira] [Commented] (FLINK-8308) Update yajl-ruby dependency to 1.3.1 or higher

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347693#comment-16347693 ] ASF GitHub Bot commented on FLINK-8308: --- Github user uce commented on the issue:

[GitHub] flink issue #5395: [FLINK-8308] Remove explicit yajl-ruby dependency, update...

2018-01-31 Thread uce
Github user uce commented on the issue: https://github.com/apache/flink/pull/5395 https://ci.apache.org/builders/flink-docs-master/builds/977/steps/Flink%20docs/logs/stdio says ``` Ruby version: ruby 2.0.0p384 (2014-01-12) [x86_64-linux-gnu] ``` I can ask

[jira] [Commented] (FLINK-7477) Use "hadoop classpath" to augment classpath when available

2018-01-31 Thread Ken Krugler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347685#comment-16347685 ] Ken Krugler commented on FLINK-7477: I posted to the mailing list about an issue that this change

[jira] [Commented] (FLINK-8101) Elasticsearch 6.x support

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347610#comment-16347610 ] ASF GitHub Bot commented on FLINK-8101: --- Github user cjolif commented on a diff in the pull request:

[GitHub] flink pull request #5374: [FLINK-8101][flink-connectors] Elasticsearch 5.3+ ...

2018-01-31 Thread cjolif
Github user cjolif commented on a diff in the pull request: https://github.com/apache/flink/pull/5374#discussion_r165186422 --- Diff: flink-connectors/flink-connector-elasticsearch5.3/src/main/java/org/apache/flink/streaming/connectors/elasticsearch53/BulkProcessorIndexer.java ---

[jira] [Commented] (FLINK-7608) LatencyGauge change to histogram metric

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347515#comment-16347515 ] ASF GitHub Bot commented on FLINK-7608: --- Github user zentol commented on the issue:

[GitHub] flink issue #5161: [FLINK-7608][metric] Refactor latency statistics metric

2018-01-31 Thread zentol
Github user zentol commented on the issue: https://github.com/apache/flink/pull/5161 Correction: We still can't display them in the UI since we have no tab for job metrics. ---

[jira] [Commented] (FLINK-7856) Port JobVertexBackPressureHandler to REST endpoint

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347418#comment-16347418 ] ASF GitHub Bot commented on FLINK-7856: --- Github user GJL commented on a diff in the pull request:

[GitHub] flink pull request #5397: [FLINK-7856][flip6] WIP

2018-01-31 Thread GJL
Github user GJL commented on a diff in the pull request: https://github.com/apache/flink/pull/5397#discussion_r165155912 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/legacy/backpressure/BackPressureStatsTracker.java --- @@ -74,7 +74,7 @@

[jira] [Commented] (FLINK-7856) Port JobVertexBackPressureHandler to REST endpoint

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347417#comment-16347417 ] ASF GitHub Bot commented on FLINK-7856: --- Github user GJL commented on a diff in the pull request:

[GitHub] flink pull request #5397: [FLINK-7856][flip6] WIP

2018-01-31 Thread GJL
Github user GJL commented on a diff in the pull request: https://github.com/apache/flink/pull/5397#discussion_r165155639 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/messages/JobVertexBackPressureInfo.java --- @@ -0,0 +1,184 @@ +/* + * Licensed to

[jira] [Commented] (FLINK-7856) Port JobVertexBackPressureHandler to REST endpoint

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347414#comment-16347414 ] ASF GitHub Bot commented on FLINK-7856: --- Github user GJL commented on a diff in the pull request:

[GitHub] flink pull request #5397: [FLINK-7856][flip6] WIP

2018-01-31 Thread GJL
Github user GJL commented on a diff in the pull request: https://github.com/apache/flink/pull/5397#discussion_r165155368 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/TestingTaskExecutorGateway.java --- @@ -75,6 +76,17 @@ public void

[jira] [Commented] (FLINK-7856) Port JobVertexBackPressureHandler to REST endpoint

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347413#comment-16347413 ] ASF GitHub Bot commented on FLINK-7856: --- GitHub user GJL opened a pull request:

[GitHub] flink pull request #5397: [FLINK-7856][flip6] WIP

2018-01-31 Thread GJL
GitHub user GJL opened a pull request: https://github.com/apache/flink/pull/5397 [FLINK-7856][flip6] WIP WIP PR is based on #4893 @tillrohrmann You can merge this pull request into a Git repository by running: $ git pull https://github.com/GJL/flink

[GitHub] flink issue #5395: [FLINK-8308] Remove explicit yajl-ruby dependency, update...

2018-01-31 Thread StevenLangbroek
Github user StevenLangbroek commented on the issue: https://github.com/apache/flink/pull/5395 https://github.com/apache/calcite/tree/master/site#setup ---

[jira] [Commented] (FLINK-8308) Update yajl-ruby dependency to 1.3.1 or higher

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347348#comment-16347348 ] ASF GitHub Bot commented on FLINK-8308: --- Github user StevenLangbroek commented on the issue:

[jira] [Commented] (FLINK-8308) Update yajl-ruby dependency to 1.3.1 or higher

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347314#comment-16347314 ] ASF GitHub Bot commented on FLINK-8308: --- Github user alpinegizmo commented on the issue:

[jira] [Commented] (FLINK-8357) enable rolling in default log settings

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347312#comment-16347312 ] ASF GitHub Bot commented on FLINK-8357: --- Github user XuMingmin commented on a diff in the pull

[jira] [Commented] (FLINK-8357) enable rolling in default log settings

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347311#comment-16347311 ] ASF GitHub Bot commented on FLINK-8357: --- Github user XuMingmin commented on a diff in the pull

[GitHub] flink issue #5395: [FLINK-8308] Remove explicit yajl-ruby dependency, update...

2018-01-31 Thread alpinegizmo
Github user alpinegizmo commented on the issue: https://github.com/apache/flink/pull/5395 It would be lovely to ditch ruby 1.9. However, if I understand correctly, we don't fully control the environment that builds the docs, and the last time we tried this we couldn't get a newer

[GitHub] flink pull request #5371: [FLINK-8357] [conf] Enable rolling in default log ...

2018-01-31 Thread XuMingmin
Github user XuMingmin commented on a diff in the pull request: https://github.com/apache/flink/pull/5371#discussion_r165141492 --- Diff: flink-dist/src/main/flink-bin/conf/logback.xml --- @@ -17,8 +17,14 @@ --> - + ${log.file} +

[GitHub] flink pull request #5371: [FLINK-8357] [conf] Enable rolling in default log ...

2018-01-31 Thread XuMingmin
Github user XuMingmin commented on a diff in the pull request: https://github.com/apache/flink/pull/5371#discussion_r165141378 --- Diff: flink-dist/src/main/flink-bin/conf/logback-yarn.xml --- @@ -17,8 +17,14 @@ --> - + ${log.file}

[jira] [Commented] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347155#comment-16347155 ] zhu.qing commented on FLINK-8534: - And Code is attached as a BFS in graph use native flink api. > if

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu.qing updated FLINK-8534: Description: When insert too much entry into bucket (MutableHashTable insertBucketEntry() line 1054 more

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu.qing updated FLINK-8534: Description: When insert too much entry into bucket (MutableHashTable line 1054 more than 255 ) will cause 

[jira] [Commented] (FLINK-8360) Implement task-local state recovery

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347147#comment-16347147 ] ASF GitHub Bot commented on FLINK-8360: --- Github user tillrohrmann commented on a diff in the pull

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165112749 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/StateSnapshotContextSynchronousImpl.java --- @@ -130,7 +136,7 @@ public

[jira] [Commented] (FLINK-8101) Elasticsearch 6.x support

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347137#comment-16347137 ] ASF GitHub Bot commented on FLINK-8101: --- Github user cjolif commented on a diff in the pull request:

[GitHub] flink pull request #5374: [FLINK-8101][flink-connectors] Elasticsearch 5.3+ ...

2018-01-31 Thread cjolif
Github user cjolif commented on a diff in the pull request: https://github.com/apache/flink/pull/5374#discussion_r165109918 --- Diff: flink-connectors/flink-connector-elasticsearch5.3/src/main/java/org/apache/flink/streaming/connectors/elasticsearch53/Elasticsearch53ApiCallBridge.java

[jira] [Commented] (FLINK-5820) Extend State Backend Abstraction to support Global Cleanup Hooks

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347136#comment-16347136 ] ASF GitHub Bot commented on FLINK-5820: --- GitHub user StephanEwen opened a pull request:

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu.qing updated FLINK-8534: Environment: windows, intellij idea, 8g ram, 4core i5 cpu, Flink 1.4.0, and parallelism = 2 will cause

[jira] [Commented] (FLINK-8308) Update yajl-ruby dependency to 1.3.1 or higher

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347135#comment-16347135 ] ASF GitHub Bot commented on FLINK-8308: --- GitHub user StevenLangbroek opened a pull request:

[GitHub] flink pull request #5396: [FLINK-5820] [state backends] Split shared/exclusi...

2018-01-31 Thread StephanEwen
GitHub user StephanEwen opened a pull request: https://github.com/apache/flink/pull/5396 [FLINK-5820] [state backends] Split shared/exclusive state and properly handle disposal ## What is the purpose of the change This PR contains the final changes needed for [FLINK-5820].

[GitHub] flink pull request #5395: [FLINK-8308] Remove explicit yajl-ruby dependency,...

2018-01-31 Thread StevenLangbroek
GitHub user StevenLangbroek opened a pull request: https://github.com/apache/flink/pull/5395 [FLINK-8308] Remove explicit yajl-ruby dependency, update Jekyll to 3+ ## What is the purpose of the change The docs dependend on `yajl-ruby` 1.2, which had a security defect.

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu.qing updated FLINK-8534: Environment: windows intellij idea 8g ram 4core i5 cpu. Flink 1.4.0 (was: windows ideal 8g ram 4core i5

[jira] [Commented] (FLINK-3089) State API Should Support Data Expiration (State TTL)

2018-01-31 Thread Xavier Fournet (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347125#comment-16347125 ] Xavier Fournet commented on FLINK-3089: --- Thanks [~phoenixjiangnan] for the design documentation. I

[jira] [Commented] (FLINK-8360) Implement task-local state recovery

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347095#comment-16347095 ] ASF GitHub Bot commented on FLINK-8360: --- Github user tillrohrmann commented on a diff in the pull

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu.qing updated FLINK-8534: Description: When insert too much entry into bucket (MutableHashTable line 1054 more than 255 ) will cause 

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347093#comment-16347093 ] ASF GitHub Bot commented on FLINK-8484: --- Github user tzulitai commented on the issue:

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165107880 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotResult.java --- @@ -0,0 +1,83 @@ +package

[GitHub] flink issue #5337: [FLINK-8484][flink-kinesis-connector] Ensure a Kinesis co...

2018-01-31 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5337 Great, thanks for the update! As a side note, I will be making some additional changes to the code regarding the not-so-nice iteration across the `sequenceNumsToRestore` map. It would make

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165107698 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotResult.java --- @@ -0,0 +1,83 @@ +package

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu.qing updated FLINK-8534: Description: When insert too much entry into bucket (more than 255 )will cause   spillPartition(). So 

[jira] [Commented] (FLINK-8360) Implement task-local state recovery

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347090#comment-16347090 ] ASF GitHub Bot commented on FLINK-8360: --- Github user tillrohrmann commented on a diff in the pull

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165106582 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotResult.java --- @@ -0,0 +1,83 @@ +package

[jira] [Commented] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347086#comment-16347086 ] zhu.qing commented on FLINK-8534: - And I failed to use 16g laptop to reproduce the bug. The key to the bug

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347085#comment-16347085 ] ASF GitHub Bot commented on FLINK-8484: --- Github user pluppens commented on the issue:

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu.qing updated FLINK-8534: Description: When insert too much entry into bucket (more than 255 )will cause   spillPartition(). So 

[GitHub] flink issue #5337: [FLINK-8484][flink-kinesis-connector] Ensure a Kinesis co...

2018-01-31 Thread pluppens
Github user pluppens commented on the issue: https://github.com/apache/flink/pull/5337 Thanks - we've been running it in production for the last 5 days without issues, so it seems to work fine. We'll be enabling autoscaling of the streams in the coming hours, so if anything is amiss,

[jira] [Commented] (FLINK-8360) Implement task-local state recovery

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347082#comment-16347082 ] ASF GitHub Bot commented on FLINK-8360: --- Github user tillrohrmann commented on a diff in the pull

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165105360 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotResult.java --- @@ -0,0 +1,83 @@ +package

[jira] [Commented] (FLINK-8360) Implement task-local state recovery

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347080#comment-16347080 ] ASF GitHub Bot commented on FLINK-8360: --- Github user tillrohrmann commented on a diff in the pull

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165104909 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotResult.java --- @@ -0,0 +1,83 @@ +package

[jira] [Commented] (FLINK-8360) Implement task-local state recovery

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347074#comment-16347074 ] ASF GitHub Bot commented on FLINK-8360: --- Github user tillrohrmann commented on a diff in the pull

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165103361 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/Snapshotable.java --- @@ -54,5 +54,5 @@ * * @param state the old

[jira] [Commented] (FLINK-8360) Implement task-local state recovery

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347073#comment-16347073 ] ASF GitHub Bot commented on FLINK-8360: --- Github user StefanRRichter commented on a diff in the pull

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165103021 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DefaultOperatorStateBackend.java --- @@ -325,8 +325,7 @@ public

[jira] [Commented] (FLINK-8360) Implement task-local state recovery

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347072#comment-16347072 ] ASF GitHub Bot commented on FLINK-8360: --- Github user StefanRRichter commented on a diff in the pull

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165102741 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DefaultOperatorStateBackend.java --- @@ -325,8 +325,7 @@ public

[jira] [Commented] (FLINK-8360) Implement task-local state recovery

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347070#comment-16347070 ] ASF GitHub Bot commented on FLINK-8360: --- Github user tillrohrmann commented on a diff in the pull

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu.qing updated FLINK-8534: Description: When insert too much entry into bucket (more than 255 )will cause   spillPartition(). So 

[jira] [Commented] (FLINK-8360) Implement task-local state recovery

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347068#comment-16347068 ] ASF GitHub Bot commented on FLINK-8360: --- Github user tillrohrmann commented on a diff in the pull

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165102131 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DefaultOperatorStateBackend.java --- @@ -325,8 +325,7 @@ public

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165101966 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -1016,7

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread Fabian Hueske (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabian Hueske updated FLINK-8534: - Component/s: Local Runtime > if insert too much BucketEntry into one bucket in join of iteration

[jira] [Commented] (FLINK-8360) Implement task-local state recovery

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347063#comment-16347063 ] ASF GitHub Bot commented on FLINK-8360: --- Github user tillrohrmann commented on a diff in the pull

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-01-31 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r165098878 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/StateObjectCollection.java --- @@ -0,0 +1,179 @@ +/* + * Licensed to

[jira] [Created] (FLINK-8540) FileStateHandles must not attempt to delete their parent directory.

2018-01-31 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-8540: --- Summary: FileStateHandles must not attempt to delete their parent directory. Key: FLINK-8540 URL: https://issues.apache.org/jira/browse/FLINK-8540 Project: Flink

[jira] [Created] (FLINK-8539) Introduce "CompletedCheckpointStorageLocation" to explicitly handle disposal of checkpoint storage locations

2018-01-31 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-8539: --- Summary: Introduce "CompletedCheckpointStorageLocation" to explicitly handle disposal of checkpoint storage locations Key: FLINK-8539 URL:

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu.qing updated FLINK-8534: Description: When insert too much entry into bucket will cause  spillPartition(). So 

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu.qing updated FLINK-8534: Description: When insert too much entry into bucket will cause  spillPartition(). So 

[jira] [Commented] (FLINK-8308) Update yajl-ruby dependency to 1.3.1 or higher

2018-01-31 Thread Ufuk Celebi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347033#comment-16347033 ] Ufuk Celebi commented on FLINK-8308: I don't know how many people use it but we have a Docker image

[jira] [Commented] (FLINK-8526) When use parallelism equals to half of the number of cpu, join and shuffle operators will easly cause deadlock.

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347027#comment-16347027 ] zhu.qing commented on FLINK-8526: - link there is more information in 

[jira] [Updated] (FLINK-8534) if insert too much BucketEntry into one bucket in join of iteration will cause a error (Caused : java.io.FileNotFoundException release file error)

2018-01-31 Thread zhu.qing (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu.qing updated FLINK-8534: Attachment: T2AdjSetBfs.java > if insert too much BucketEntry into one bucket in join of iteration will >

[jira] [Commented] (FLINK-8384) Session Window Assigner with Dynamic Gaps

2018-01-31 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347020#comment-16347020 ] ASF GitHub Bot commented on FLINK-8384: --- Github user dyanarose commented on the issue:

[GitHub] flink issue #5295: [FLINK-8384] [streaming] Session Window Assigner with Dyn...

2018-01-31 Thread dyanarose
Github user dyanarose commented on the issue: https://github.com/apache/flink/pull/5295 Ah, I hadn't thought to keep both in place. So unless the Dynamic SessionWindow classes had withDynamicGap made package private, you would then be able to instantiate them from two different

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

2018-01-31 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347008#comment-16347008 ] Aljoscha Krettek commented on FLINK-8500: - Yes, no I feel stupid.  You analysed it exactly right:

[jira] [Comment Edited] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

2018-01-31 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347008#comment-16347008 ] Aljoscha Krettek edited comment on FLINK-8500 at 1/31/18 3:26 PM: -- Yes,

[jira] [Updated] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

2018-01-31 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aljoscha Krettek updated FLINK-8500: Priority: Blocker (was: Major) > Get the timestamp of the Kafka message from kafka

  1   2   3   4   >