[jira] [Commented] (FLINK-14422) Expose TaskExecutor resource configurations to RESTful API
[ https://issues.apache.org/jira/browse/FLINK-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953424#comment-16953424 ] Xintong Song commented on FLINK-14422: -- [~lining], I've updated the summary and description according to our offline discussion. The original one seems a bit unclear. Please correct me if the updates are not consistent with your original idea. > Expose TaskExecutor resource configurations to RESTful API > -- > > Key: FLINK-14422 > URL: https://issues.apache.org/jira/browse/FLINK-14422 > Project: Flink > Issue Type: Wish > Components: Runtime / Task >Reporter: lining >Priority: Major > > TaskExecutors in the same Flink cluster might be configured with different > resources. It would be helpful to expose TaskExecutors' resource configs to > RESTful API, so it can be displayed on Flink web ui or queried by external > systems. > Currently, resource configurations are used for creating > TaskExecutorServices, which will be passed into the TaskExecutor. The > TaskExecutor, after created, is not aware of its resource configuration. > We propose to also pass the original resource configs into the TaskExecutor, > and report it to ResourceManager on registration. In that way, the rest > server can acquire the resource configs from ResourceManager, as part of > TaskManagerInfo. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9916: [FLINK-14408][Table-Planner]UDF's open method is invoked to initialize when sql is optimized in oldPlanner
flinkbot edited a comment on issue #9916: [FLINK-14408][Table-Planner]UDF's open method is invoked to initialize when sql is optimized in oldPlanner URL: https://github.com/apache/flink/pull/9916#issuecomment-542764294 ## CI report: * cd122b399a8dc12c3c604d7db6dea012ee7c0c7f : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132188836) * 926952d0d3159aa788269aa3e51ce5e0a33088d2 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14422) Expose TaskExecutor resource configurations to RESTful API
[ https://issues.apache.org/jira/browse/FLINK-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-14422: - Description: TaskExecutors in the same Flink cluster might be configured with different resources. It would be helpful to expose TaskExecutors' resource configs to RESTful API, so it can be displayed on Flink web ui or queried by external systems. Currently, resource configurations are used for creating TaskExecutorServices, which will be passed into the TaskExecutor. The TaskExecutor, after created, is not aware of its resource configuration. We propose to also pass the original resource configs into the TaskExecutor, and report it to ResourceManager on registration. In that way, the rest server can acquire the resource configs from ResourceManager, as part of TaskManagerInfo. was:Now user could get network segment size metric, but couldn't calculate it‘s memory size. Because it couldn't get page size which in config. For standalone, each taskmanagers has its config. Should we add TaskmangerRescourceConfig for TaskExecutor, then user could get it from rest. > Expose TaskExecutor resource configurations to RESTful API > -- > > Key: FLINK-14422 > URL: https://issues.apache.org/jira/browse/FLINK-14422 > Project: Flink > Issue Type: Wish > Components: Runtime / Task >Reporter: lining >Priority: Major > > TaskExecutors in the same Flink cluster might be configured with different > resources. It would be helpful to expose TaskExecutors' resource configs to > RESTful API, so it can be displayed on Flink web ui or queried by external > systems. > Currently, resource configurations are used for creating > TaskExecutorServices, which will be passed into the TaskExecutor. The > TaskExecutor, after created, is not aware of its resource configuration. > We propose to also pass the original resource configs into the TaskExecutor, > and report it to ResourceManager on registration. In that way, the rest > server can acquire the resource configs from ResourceManager, as part of > TaskManagerInfo. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9843: [FLINK-14296] [Table SQL] Use Optional for optional parameters in parser module
flinkbot edited a comment on issue #9843: [FLINK-14296] [Table SQL] Use Optional for optional parameters in parser module URL: https://github.com/apache/flink/pull/9843#issuecomment-538717105 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 64f2cbc7910a1c80ed6eb5c09b42cbeb1ac9d2d7 (Thu Oct 17 05:48:25 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-14296) Improve handling of parameters nullabillity in parser module
[
https://issues.apache.org/jira/browse/FLINK-14296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953417#comment-16953417
]
Jark Wu edited comment on FLINK-14296 at 10/17/19 5:47 AM:
---
1.10.0: 456a17685374aaa2f17479c013fde56fc528e7fb
was (Author: jark):
1.10.0: 0a9d287a92144093b6fd648143a4805f6ae6a38d
> Improve handling of parameters nullabillity in parser module
>
>
> Key: FLINK-14296
> URL: https://issues.apache.org/jira/browse/FLINK-14296
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
>Reporter: Dawid Wysakowicz
>Assignee: Jiayi Liao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I want to suggest using Optional for optional parameters in classes such as
> SqlCreateTable/SqlCreateView/SqlTableColumn etc.
> Right now we must check against null at different locations e.g.:
> {code}
> SqlNodeList partitionKey = sqlCreateTable.getPartitionKeyList();
> if (partitionKey != null) {
> partitionKeys = partitionKey
> .getList()
> .stream()
> .map(p -> ((SqlIdentifier) p).getSimple())
> .collect(Collectors.toList());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] wuchong closed pull request #9843: [FLINK-14296] [Table SQL] Use Optional for optional parameters in parser module
wuchong closed pull request #9843: [FLINK-14296] [Table SQL] Use Optional for optional parameters in parser module URL: https://github.com/apache/flink/pull/9843 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex
flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex URL: https://github.com/apache/flink/pull/9798#issuecomment-536255109 ## CI report: * 7231b0df07d82c5c758d744f51401251f6449ea9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129579103) * ac8cc7bcba9e93fbea9374409df581d2959aecb8 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132268257) * 668dce7d962640943a26b3c669cb2f0b413daf8b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132271214) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese URL: https://github.com/apache/flink/pull/9784#issuecomment-535882318 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 096444b219447b1d467fe63ab6278c1c28802eb0 (Thu Oct 17 05:41:19 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] AT-Fieldless commented on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
AT-Fieldless commented on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese URL: https://github.com/apache/flink/pull/9784#issuecomment-543011701 Ok, I agree with you and I hava not problem This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (FLINK-14296) Improve handling of parameters nullabillity in parser module
[
https://issues.apache.org/jira/browse/FLINK-14296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu resolved FLINK-14296.
-
Fix Version/s: 1.10.0
Resolution: Fixed
1.10.0: 0a9d287a92144093b6fd648143a4805f6ae6a38d
> Improve handling of parameters nullabillity in parser module
>
>
> Key: FLINK-14296
> URL: https://issues.apache.org/jira/browse/FLINK-14296
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
>Reporter: Dawid Wysakowicz
>Assignee: Jiayi Liao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I want to suggest using Optional for optional parameters in classes such as
> SqlCreateTable/SqlCreateView/SqlTableColumn etc.
> Right now we must check against null at different locations e.g.:
> {code}
> SqlNodeList partitionKey = sqlCreateTable.getPartitionKeyList();
> if (partitionKey != null) {
> partitionKeys = partitionKey
> .getList()
> .stream()
> .map(p -> ((SqlIdentifier) p).getSimple())
> .collect(Collectors.toList());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9900: [FLINK-14366][tests] Annotate MiniCluster tests in flink-tests with AlsoRunWithSchedulerNG
flinkbot edited a comment on issue #9900: [FLINK-14366][tests] Annotate MiniCluster tests in flink-tests with AlsoRunWithSchedulerNG URL: https://github.com/apache/flink/pull/9900#issuecomment-542033352 ## CI report: * a69bbc4f6eb0eac5509a989ebd5a4f31d8b028d6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/131903595) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14422) Expose TaskExecutor resource configurations to RESTful API
[ https://issues.apache.org/jira/browse/FLINK-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-14422: - Summary: Expose TaskExecutor resource configurations to RESTful API (was: Get taskmanager resource config in rest api) > Expose TaskExecutor resource configurations to RESTful API > -- > > Key: FLINK-14422 > URL: https://issues.apache.org/jira/browse/FLINK-14422 > Project: Flink > Issue Type: Wish > Components: Runtime / Task >Reporter: lining >Priority: Major > > Now user could get network segment size metric, but couldn't calculate it‘s > memory size. Because it couldn't get page size which in config. For > standalone, each taskmanagers has its config. Should we add > TaskmangerRescourceConfig for TaskExecutor, then user could get it from rest. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-14364) Allow comments fail when not ignore parse errors in CsvRowDeserializationSchema
[ https://issues.apache.org/jira/browse/FLINK-14364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953412#comment-16953412 ] Jiayi Liao edited comment on FLINK-14364 at 10/17/19 5:22 AM: -- [~twalthr] I've checked jackson codes and find out that this allowComments only work with multiple lines message by using objectReader.readValues(message) insteand of objectReader.readValue(message). I'm not sure if this is a jackson bug or expected behaviour. But either way I think we should add this into #CsvValidator#validate. What do you think? was (Author: wind_ljy): [~twalthr] I've checked jackson codes and find out that this allowComments only work with multiple lines message by using objectReader.readValues(message) insteand of objectReader.readValue(message). I'm not sure if this is a jackson bug or expected behaviour. But either way I think we should add this into #CsvValidator#validate. > Allow comments fail when not ignore parse errors in > CsvRowDeserializationSchema > --- > > Key: FLINK-14364 > URL: https://issues.apache.org/jira/browse/FLINK-14364 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Reporter: Jingsong Lee >Assignee: Jiayi Liao >Priority: Major > Fix For: 1.10.0 > > > Use CsvRowDeserializationSchema, when setIgnoreParseErrors(false) and > setAllowComments(true). > If there are some comments in msg, will throw MismatchedInputException. > If this a bug? and we should catch MismatchedInputException and return null? > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-14364) Allow comments fail when not ignore parse errors in CsvRowDeserializationSchema
[ https://issues.apache.org/jira/browse/FLINK-14364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953412#comment-16953412 ] Jiayi Liao commented on FLINK-14364: [~twalthr] I've checked jackson codes and find out that this allowComments only work with multiple lines message by using objectReader.readValues(message) insteand of objectReader.readValue(message). I'm not sure if this is a jackson bug or expected behaviour. But either way I think we should add this into #CsvValidator#validate. > Allow comments fail when not ignore parse errors in > CsvRowDeserializationSchema > --- > > Key: FLINK-14364 > URL: https://issues.apache.org/jira/browse/FLINK-14364 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Reporter: Jingsong Lee >Assignee: Jiayi Liao >Priority: Major > Fix For: 1.10.0 > > > Use CsvRowDeserializationSchema, when setIgnoreParseErrors(false) and > setAllowComments(true). > If there are some comments in msg, will throw MismatchedInputException. > If this a bug? and we should catch MismatchedInputException and return null? > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese URL: https://github.com/apache/flink/pull/9784#issuecomment-535882318 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 096444b219447b1d467fe63ab6278c1c28802eb0 (Thu Oct 17 05:11:46 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
xintongsong commented on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese URL: https://github.com/apache/flink/pull/9784#issuecomment-543004945 I would suggest we differentiate the occurrences of 'source' into two categories and handle them differently. - For 'source' that refers to the source operator, or the source vertex of the job graph, we treat it as a proper noun and do not translate it. - For 'source' that refers to where the data comes from in general, we translate it to '数据源'. Above are my personal opinions. Discussions are welcomed if you have different opinions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9917: [FLINK-14421][Table-API]Normalize field definition
flinkbot edited a comment on issue #9917: [FLINK-14421][Table-API]Normalize field definition URL: https://github.com/apache/flink/pull/9917#issuecomment-542773708 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 02a6efc4d621e0d5bc143ffde8abc28faa89186e (Thu Oct 17 05:08:42 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wangxlong commented on issue #9917: [FLINK-14421][Table-API]Normalize field definition
wangxlong commented on issue #9917: [FLINK-14421][Table-API]Normalize field definition URL: https://github.com/apache/flink/pull/9917#issuecomment-543004575 > Thanks for the contribution @wangxlong, btw, please pay more attention on the commit message, you can refer https://flink.apache.org/contributing/code-style-and-quality-pull-requests.html for more details. Oh, i am sorry fot that. I will pay more attention on it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9920: [FLINK-14389][runtime] Restore task state before restarting tasks in DefaultScheduler
flinkbot edited a comment on issue #9920: [FLINK-14389][runtime] Restore task state before restarting tasks in DefaultScheduler URL: https://github.com/apache/flink/pull/9920#issuecomment-542997479 ## CI report: * c7e058ec893217ef5c8c3cf3ae5e893a7ccf4b8f : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132271237) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9890: [FLINK-14272][python][table-planner-blink] Support Blink planner for Python UDF
flinkbot edited a comment on issue #9890: [FLINK-14272][python][table-planner-blink] Support Blink planner for Python UDF URL: https://github.com/apache/flink/pull/9890#issuecomment-541418126 ## CI report: * ec7f1a70fd420d25326e49ac1f4de878d4aae481 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/131691044) * 47706d03f178962d0e0fabc4b4a51d0810be5e91 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/131693072) * 9546adc32393cf7a777a963133a134ea2f5f4fd6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132096206) * ffac224585e0d5001dae23ae1778ecbffc61a7b6 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132271229) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9911: [hotfix][docs-zh] Fix the incorrect link
flinkbot edited a comment on issue #9911: [hotfix][docs-zh] Fix the incorrect link URL: https://github.com/apache/flink/pull/9911#issuecomment-542689902 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 5b6a576cd035a7efe8ffd6663719f5b7c5385403 (Thu Oct 17 04:56:35 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
flinkbot edited a comment on issue #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups. URL: https://github.com/apache/flink/pull/9910#issuecomment-542638190 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8d1696b97872fdd9fd3aca725c8b58ad8227f6f2 (Thu Oct 17 04:56:31 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] 1996fanrui commented on issue #9911: [hotfix][docs-zh] Fix the incorrect link
1996fanrui commented on issue #9911: [hotfix][docs-zh] Fix the incorrect link URL: https://github.com/apache/flink/pull/9911#issuecomment-543001479 Hi @wuchong , Please help to review and merge this PR. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime]
Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory
setups.
URL: https://github.com/apache/flink/pull/9910#discussion_r335809349
##
File path:
flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/nodes/resource/NodeResourceUtil.java
##
@@ -34,8 +34,9 @@
* Build resourceSpec from managedMem.
*/
public static ResourceSpec fromManagedMem(int managedMem) {
- ResourceSpec.Builder builder = ResourceSpec.newBuilder();
- builder.setManagedMemoryInMB(managedMem);
+ ResourceSpec.Builder builder = ResourceSpec.newBuilder(0.0, 0);
+ // TODO: before operators separate on-heap/off-heap managed
memory, we use on-heap managed memory to denote total managed memory
+ builder.setOnHeapManagedMemoryMB(managedMem);
Review comment:
This is an util class for table planner only. It's in the module
`flink-table-planner-blink`. It's not used for DataSet API.
We do not plan to enable fine grained resource management for DataSet API.
It will use `UNKNOWN` resource profiles/specs only. For release 1.10, the plan
is to enable fine grained resource management for blink table/sql jobs only. So
there shouldn't be a problem.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese URL: https://github.com/apache/flink/pull/9784#issuecomment-53526 ## CI report: * 198da66f5d4525d04f69bfed13eedb51652a2f38 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129428277) * be59de38e2c2710ab7a07a7a9ffbf04d92db21d5 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132082361) * 096444b219447b1d467fe63ab6278c1c28802eb0 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132268240) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex
flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex URL: https://github.com/apache/flink/pull/9798#issuecomment-536255109 ## CI report: * 7231b0df07d82c5c758d744f51401251f6449ea9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129579103) * ac8cc7bcba9e93fbea9374409df581d2959aecb8 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132268257) * 668dce7d962640943a26b3c669cb2f0b413daf8b : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132271214) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9920: [FLINK-14389][runtime] Restore task state before restarting tasks in DefaultScheduler
flinkbot commented on issue #9920: [FLINK-14389][runtime] Restore task state before restarting tasks in DefaultScheduler URL: https://github.com/apache/flink/pull/9920#issuecomment-542997479 ## CI report: * c7e058ec893217ef5c8c3cf3ae5e893a7ccf4b8f : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test
flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542991773 ## CI report: * a7539d69ef1d149aaf1b6a749a04ca94b16ef6d2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132269780) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9900: [FLINK-14366][tests] Annotate MiniCluster tests in flink-tests with AlsoRunWithSchedulerNG
flinkbot edited a comment on issue #9900: [FLINK-14366][tests] Annotate MiniCluster tests in flink-tests with AlsoRunWithSchedulerNG URL: https://github.com/apache/flink/pull/9900#issuecomment-542033352 ## CI report: * a69bbc4f6eb0eac5509a989ebd5a4f31d8b028d6 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/131903595) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9890: [FLINK-14272][python][table-planner-blink] Support Blink planner for Python UDF
flinkbot edited a comment on issue #9890: [FLINK-14272][python][table-planner-blink] Support Blink planner for Python UDF URL: https://github.com/apache/flink/pull/9890#issuecomment-541418126 ## CI report: * ec7f1a70fd420d25326e49ac1f4de878d4aae481 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/131691044) * 47706d03f178962d0e0fabc4b4a51d0810be5e91 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/131693072) * 9546adc32393cf7a777a963133a134ea2f5f4fd6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132096206) * ffac224585e0d5001dae23ae1778ecbffc61a7b6 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex
flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex URL: https://github.com/apache/flink/pull/9798#issuecomment-536255109 ## CI report: * 7231b0df07d82c5c758d744f51401251f6449ea9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129579103) * ac8cc7bcba9e93fbea9374409df581d2959aecb8 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132268257) * 668dce7d962640943a26b3c669cb2f0b413daf8b : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9900: [FLINK-14366][tests] Annotate MiniCluster tests in flink-tests with AlsoRunWithSchedulerNG
flinkbot edited a comment on issue #9900: [FLINK-14366][tests] Annotate MiniCluster tests in flink-tests with AlsoRunWithSchedulerNG URL: https://github.com/apache/flink/pull/9900#issuecomment-542026880 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit a69bbc4f6eb0eac5509a989ebd5a4f31d8b028d6 (Thu Oct 17 04:23:54 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex
flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex URL: https://github.com/apache/flink/pull/9798#issuecomment-536254451 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 668dce7d962640943a26b3c669cb2f0b413daf8b (Thu Oct 17 04:22:53 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhuzhurk commented on issue #9900: [FLINK-14366][tests] Annotate MiniCluster tests in flink-tests with AlsoRunWithSchedulerNG
zhuzhurk commented on issue #9900: [FLINK-14366][tests] Annotate MiniCluster tests in flink-tests with AlsoRunWithSchedulerNG URL: https://github.com/apache/flink/pull/9900#issuecomment-542993760 @flinkbot run travis This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] vthinkxie commented on a change in pull request #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex
vthinkxie commented on a change in pull request #9798: [FLINK-14176][web]Web Ui
add log url for taskmanager of vertex
URL: https://github.com/apache/flink/pull/9798#discussion_r335804160
##
File path:
flink-runtime-web/web-dashboard/src/app/pages/job/overview/taskmanagers/job-overview-drawer-taskmanagers.component.html
##
@@ -40,7 +41,13 @@
- {{ taskManager.host }}
+ {{ taskManager.host }}
Review comment:
```suggestion
{{ taskManager.host }}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9920: [FLINK-14389][runtime] Restore task state before restarting tasks in DefaultScheduler
flinkbot commented on issue #9920: [FLINK-14389][runtime] Restore task state before restarting tasks in DefaultScheduler URL: https://github.com/apache/flink/pull/9920#issuecomment-542992891 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit c7e058ec893217ef5c8c3cf3ae5e893a7ccf4b8f (Thu Oct 17 04:18:31 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14389) Restore task state in new DefaultScheduler
[
https://issues.apache.org/jira/browse/FLINK-14389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-14389:
---
Labels: pull-request-available (was: )
> Restore task state in new DefaultScheduler
> --
>
> Key: FLINK-14389
> URL: https://issues.apache.org/jira/browse/FLINK-14389
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: Gary Yao
>Assignee: Zhu Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
>
> The new {{DefaultScheduler}} should restore the state of restarted tasks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] zhuzhurk opened a new pull request #9920: [FLINK-14389][runtime] Restore task state before restarting tasks in DefaultScheduler
zhuzhurk opened a new pull request #9920: [FLINK-14389][runtime] Restore task state before restarting tasks in DefaultScheduler URL: https://github.com/apache/flink/pull/9920 ## What is the purpose of the change This PR is to enabled DefaultScheduler to restore task state before restarting tasks. ## Brief change log - *Added SchedulerBase#restartState for task state restore* - *Changed DefaultScheduler to invoke SchedulerBase#restartState before restarting tasks* ## Verifying this change This change added tests and can be verified as follows: - *Added unit tests* - *KeyedStateCheckpointingITCase is able to pass with this fix when using ng scheduler* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (**yes** / no / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9919: [FLINK-13303][hive]add hive e2e connector test
flinkbot commented on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542991773 ## CI report: * a7539d69ef1d149aaf1b6a749a04ca94b16ef6d2 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9900: [FLINK-14366][tests] Annotate MiniCluster tests in flink-tests with AlsoRunWithSchedulerNG
flinkbot edited a comment on issue #9900: [FLINK-14366][tests] Annotate MiniCluster tests in flink-tests with AlsoRunWithSchedulerNG URL: https://github.com/apache/flink/pull/9900#issuecomment-542033352 ## CI report: * a69bbc4f6eb0eac5509a989ebd5a4f31d8b028d6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/131903595) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex
flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex URL: https://github.com/apache/flink/pull/9798#issuecomment-536254451 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 668dce7d962640943a26b3c669cb2f0b413daf8b (Thu Oct 17 04:07:38 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
flinkbot edited a comment on issue #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering URL: https://github.com/apache/flink/pull/9853#issuecomment-539467530 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 272536f7c0a89d283e76c187ca29cca2b0951318 (Thu Oct 17 04:05:36 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @pnowojski [committer] * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] ifndef-SleePy commented on a change in pull request #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
ifndef-SleePy commented on a change in pull request #9853:
[FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
URL: https://github.com/apache/flink/pull/9853#discussion_r335801615
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
##
@@ -1306,10 +1312,8 @@ private long getRandomInitDelay() {
return
ThreadLocalRandom.current().nextLong(minPauseBetweenCheckpoints, baseInterval +
1L);
}
- private ScheduledFuture scheduleTriggerWithDelay(long initDelay) {
- return timer.scheduleAtFixedRate(
- new ScheduledTrigger(),
- initDelay, baseInterval, TimeUnit.MILLISECONDS);
+ private ScheduledFuture scheduleTriggerWithDelay(long delay) {
+ return timer.schedule(new ScheduledTrigger(), delay,
TimeUnit.MILLISECONDS);
Review comment:
That's a good question. The direct reason is that currently the main thread
executor does not support periodic scheduling, it's not implemented yet. So
there are two options.
1. Support periodic scheduling in main thread executor.
2. Manually triggering checkpoint in `CheckpointCoordinator`.
There is a short discussion of the option 1, see
https://issues.apache.org/jira/browse/FLINK-13848.
I choose option 2 currently. Because I'm a not fan of the way of periodic
scheduling currently. There are a lot of cancellation of periodic scheduling.
Cancellation is not accurate and reliable somewhat.
But to be honest, there is a short-coming of option 2. We have to make sure
the next trigger will be scheduled under all scenarios, successful, failed,
request queued (it's messy of handling queued request now). It's a bit
complicated than I expected.
Do you have any suggestion of this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex
flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex URL: https://github.com/apache/flink/pull/9798#issuecomment-536255109 ## CI report: * 7231b0df07d82c5c758d744f51401251f6449ea9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129579103) * ac8cc7bcba9e93fbea9374409df581d2959aecb8 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132268257) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9890: [FLINK-14272][python][table-planner-blink] Support Blink planner for Python UDF
flinkbot edited a comment on issue #9890: [FLINK-14272][python][table-planner-blink] Support Blink planner for Python UDF URL: https://github.com/apache/flink/pull/9890#issuecomment-541416833 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit ffac224585e0d5001dae23ae1778ecbffc61a7b6 (Thu Oct 17 04:03:33 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese URL: https://github.com/apache/flink/pull/9784#issuecomment-53526 ## CI report: * 198da66f5d4525d04f69bfed13eedb51652a2f38 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129428277) * be59de38e2c2710ab7a07a7a9ffbf04d92db21d5 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132082361) * 096444b219447b1d467fe63ab6278c1c28802eb0 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132268240) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] HuangXingBo commented on issue #9890: [FLINK-14272][python][table-planner-blink] Support Blink planner for Python UDF
HuangXingBo commented on issue #9890: [FLINK-14272][python][table-planner-blink] Support Blink planner for Python UDF URL: https://github.com/apache/flink/pull/9890#issuecomment-542989620 Thanks for @hequn8128 @WeiZhong94 reviewed. I have addressed comments at the latest commit. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
flinkbot edited a comment on issue #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups. URL: https://github.com/apache/flink/pull/9910#issuecomment-542638190 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8d1696b97872fdd9fd3aca725c8b58ad8227f6f2 (Thu Oct 17 04:00:30 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-11491) Support all TPC-DS queries
[ https://issues.apache.org/jira/browse/FLINK-11491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953376#comment-16953376 ] Jark Wu commented on FLINK-11491: - Sure [~Leonard Xu]. > Support all TPC-DS queries > -- > > Key: FLINK-11491 > URL: https://issues.apache.org/jira/browse/FLINK-11491 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue describes the goal of supporting all TPC-DS queries on a unified > runtime for batch and streaming. > Operations might not be executed with the full performance until changes in > other Flink core components have taken place. > This includes external catalog support and Hive integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups. URL: https://github.com/apache/flink/pull/9910#discussion_r335800874 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/types/ResourceProfile.java ## @@ -64,30 +64,30 @@ * A ResourceProfile that indicates infinite resource that matches any resource requirement, for testability purpose only. * */ @VisibleForTesting - public static final ResourceProfile INFINITE = new ResourceProfile(Double.MAX_VALUE, Integer.MAX_VALUE, Integer.MAX_VALUE, Integer.MAX_VALUE, Integer.MAX_VALUE, Integer.MAX_VALUE, Collections.emptyMap()); + public static final ResourceProfile INFINITE = new ResourceProfile(Double.MAX_VALUE, MemorySize.MAX_VALUE, MemorySize.MAX_VALUE, MemorySize.MAX_VALUE, MemorySize.MAX_VALUE, MemorySize.MAX_VALUE, Collections.emptyMap()); /** A ResourceProfile describing zero resources. */ - public static final ResourceProfile ZERO = new ResourceProfile(0, 0); + public static final ResourceProfile ZERO = new ResourceProfile(0, MemorySize.ZERO); // /** How many cpu cores are needed, use double so we can specify cpu like 0.1. */ private final double cpuCores; - /** How many heap memory in mb are needed. */ - private final int heapMemoryInMB; + /** How many task heap memory are needed. */ + private final MemorySize taskHeapMemory; - /** How many direct memory in mb are needed. */ - private final int directMemoryInMB; + /** How many task off-heap memory are needed. */ + private final MemorySize taskOffHeapMemory; Review comment: - `MemorySize` is facing users. Actually, it's annotated with `@PublicEvolving`. - `ResourceProfile` is not facing Flink users. It's used in runtime only. - `ResourceSpec` is annotated `@Internal` for the moment, and we do not have public interfaces allowing users to directly set `ResourceSpec` at the moment. Only blink optimizer is setting it for the users according to their configs. In the future, if we decide to expose interfaces to users for directly setting operator resource requirements (e.g., in DataStream API), it is possible to make `ResourceSpec` user facing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (FLINK-14423) Import teradat.tpcds as test data generator (refer to hive)
[ https://issues.apache.org/jira/browse/FLINK-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu reassigned FLINK-14423: --- Assignee: Leonard Xu > Import teradat.tpcds as test data generator (refer to hive) > --- > > Key: FLINK-14423 > URL: https://issues.apache.org/jira/browse/FLINK-14423 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API, Table SQL / Planner >Reporter: Leonard Xu >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (FLINK-14424) Create tpc-ds end to end test
[ https://issues.apache.org/jira/browse/FLINK-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu reassigned FLINK-14424: --- Assignee: Leonard Xu > Create tpc-ds end to end test > -- > > Key: FLINK-14424 > URL: https://issues.apache.org/jira/browse/FLINK-14424 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API, Table SQL / Planner >Reporter: Leonard Xu >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (FLINK-14425) Pick necessary commits gently that tpc-ds tests need from flink-tpc-ds project
[ https://issues.apache.org/jira/browse/FLINK-14425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu reassigned FLINK-14425: --- Assignee: Leonard Xu > Pick necessary commits gently that tpc-ds tests need from flink-tpc-ds project > -- > > Key: FLINK-14425 > URL: https://issues.apache.org/jira/browse/FLINK-14425 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API, Table SQL / Planner >Reporter: Leonard Xu >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (FLINK-14426) Test and pass 103 tpcds tests
[ https://issues.apache.org/jira/browse/FLINK-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu reassigned FLINK-14426: --- Assignee: Leonard Xu > Test and pass 103 tpcds tests > - > > Key: FLINK-14426 > URL: https://issues.apache.org/jira/browse/FLINK-14426 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API, Table SQL / Planner >Reporter: Leonard Xu >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-11491) Support all TPC-DS queries
[ https://issues.apache.org/jira/browse/FLINK-11491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953372#comment-16953372 ] Leonard Xu edited comment on FLINK-11491 at 10/17/19 3:54 AM: -- [~jark] Thanks，and I have created corresponding sub-tasks. Can you help assign them to me ？ was (Author: leonard xu): [~jark] Thanks，and I have created corresponding sub-tasks. > Support all TPC-DS queries > -- > > Key: FLINK-11491 > URL: https://issues.apache.org/jira/browse/FLINK-11491 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue describes the goal of supporting all TPC-DS queries on a unified > runtime for batch and streaming. > Operations might not be executed with the full performance until changes in > other Flink core components have taken place. > This includes external catalog support and Hive integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9843: [FLINK-14296] [Table SQL] Use Optional for optional parameters in parser module
flinkbot edited a comment on issue #9843: [FLINK-14296] [Table SQL] Use Optional for optional parameters in parser module URL: https://github.com/apache/flink/pull/9843#issuecomment-538717105 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 64f2cbc7910a1c80ed6eb5c09b42cbeb1ac9d2d7 (Thu Oct 17 03:53:28 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test
flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542986145 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit a7539d69ef1d149aaf1b6a749a04ca94b16ef6d2 (Thu Oct 17 03:53:23 UTC 2019) **Warnings:** * **2 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-11491) Support all TPC-DS queries
[ https://issues.apache.org/jira/browse/FLINK-11491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953372#comment-16953372 ] Leonard Xu commented on FLINK-11491: [~jark] Thanks，and I have created corresponding sub-tasks. > Support all TPC-DS queries > -- > > Key: FLINK-11491 > URL: https://issues.apache.org/jira/browse/FLINK-11491 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue describes the goal of supporting all TPC-DS queries on a unified > runtime for batch and streaming. > Operations might not be executed with the full performance until changes in > other Flink core components have taken place. > This includes external catalog support and Hive integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] zjuwangg commented on issue #9919: [FLINK-13303][hive]add hive e2e connector test
zjuwangg commented on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542987273 cc @lirui-apache @bowenli86 @xuefuz to have a preview. Some minor steps should be done soon. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] buptljy commented on issue #9843: [FLINK-14296] [Table SQL] Use Optional for optional parameters in parser module
buptljy commented on issue #9843: [FLINK-14296] [Table SQL] Use Optional for optional parameters in parser module URL: https://github.com/apache/flink/pull/9843#issuecomment-542987335 @danny0405 @wuchong @dawidwys Thanks for you review! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14426) Test and pass 103 tpcds tests
Leonard Xu created FLINK-14426: -- Summary: Test and pass 103 tpcds tests Key: FLINK-14426 URL: https://issues.apache.org/jira/browse/FLINK-14426 Project: Flink Issue Type: Sub-task Components: Table SQL / API, Table SQL / Planner Reporter: Leonard Xu Fix For: 1.10.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14425) Pick necessary commits gently that tpc-ds tests need from flink-tpc-ds project
Leonard Xu created FLINK-14425: -- Summary: Pick necessary commits gently that tpc-ds tests need from flink-tpc-ds project Key: FLINK-14425 URL: https://issues.apache.org/jira/browse/FLINK-14425 Project: Flink Issue Type: Sub-task Components: Table SQL / API, Table SQL / Planner Reporter: Leonard Xu Fix For: 1.10.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
flinkbot edited a comment on issue #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups. URL: https://github.com/apache/flink/pull/9910#issuecomment-542638190 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8d1696b97872fdd9fd3aca725c8b58ad8227f6f2 (Thu Oct 17 03:47:17 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime]
Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory
setups.
URL: https://github.com/apache/flink/pull/9910#discussion_r335798903
##
File path:
flink-core/src/main/java/org/apache/flink/api/common/operators/ResourceSpec.java
##
@@ -310,50 +327,73 @@ private Object readResolve() {
// builder
//
- public static Builder newBuilder() {
- return new Builder();
+ public static Builder newBuilder(double cpuCores, MemorySize
taskHeapMemory) {
+ return new Builder(cpuCores, taskHeapMemory);
+ }
+
+ public static Builder newBuilder(double cpuCores, int taskHeapMemoryMB)
{
Review comment:
I don't quite understand which are the "unrelated changes" that you are
talking about? Do you mean the `Builder` class, or the `Builder#newBuilder`
factory method, or the setters with 'MB' suffix?
If you mean the `Builder` class, I don't understand how changes to it is
unrelated to the changes of `ResourceSpec` that the builder builds.
If you mean the `Builder#newBuilder` factory method, the purpose of them is
to make sure `ResourceSpec` is always constructed with explicit cpu cores and
task heap memory, which is required by FLIP-49.
If you mean the setters with 'MB' suffix, I think it also makes sense that
we remove them and keep those with `MemorySize` arguments only.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (FLINK-14424) Create tpc-ds end to end test
Leonard Xu created FLINK-14424: -- Summary: Create tpc-ds end to end test Key: FLINK-14424 URL: https://issues.apache.org/jira/browse/FLINK-14424 Project: Flink Issue Type: Sub-task Components: Table SQL / API, Table SQL / Planner Reporter: Leonard Xu Fix For: 1.10.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on issue #9919: [FLINK-13303][hive]add hive e2e connector test
flinkbot commented on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542986145 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit a7539d69ef1d149aaf1b6a749a04ca94b16ef6d2 (Thu Oct 17 03:45:40 UTC 2019) **Warnings:** * **2 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zjuwangg opened a new pull request #9919: [FLINK-13303][hive]add hive e2e connector test
zjuwangg opened a new pull request #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919 ## What is the purpose of the change * Setup a Hadoop cluster and hive environment and add a hive e2e connector test example* ## Brief change log - * Setup a Hadoop cluster and hive environment * - *add a hive e2e connector test example* ## Verifying this change This change is already covered by existing tests, such as *HiveReadWriteDataExample*. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes ) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no ) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? ( no) - If yes, how is the feature documented? (docs) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14423) Import teradat.tpcds as test data generator (refer to hive)
Leonard Xu created FLINK-14423: -- Summary: Import teradat.tpcds as test data generator (refer to hive) Key: FLINK-14423 URL: https://issues.apache.org/jira/browse/FLINK-14423 Project: Flink Issue Type: Sub-task Components: Table SQL / API, Table SQL / Planner Reporter: Leonard Xu Fix For: 1.10.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex
flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex URL: https://github.com/apache/flink/pull/9798#issuecomment-536255109 ## CI report: * 7231b0df07d82c5c758d744f51401251f6449ea9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129579103) * ac8cc7bcba9e93fbea9374409df581d2959aecb8 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9025: [FLINK-13149][Web Frontend] support configuration and update Angular to 8.0 version
flinkbot edited a comment on issue #9025: [FLINK-13149][Web Frontend] support configuration and update Angular to 8.0 version URL: https://github.com/apache/flink/pull/9025#issuecomment-512173118 ## CI report: * e2ae53781367b527fd4d77a4a63ae3b8f354a810 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119439998) * 6f2b4c282a8d3f88b58eb6e3946dca5fc6ed9752 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127981266) * a5367c6eff2e56223de725b5fe8b74afd1a0301c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128106324) * beacd0f8376589b9da8b42a80fb008895b7ff4fb : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132266696) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-11491) Support all TPC-DS queries
[ https://issues.apache.org/jira/browse/FLINK-11491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953358#comment-16953358 ] Leonard Xu edited comment on FLINK-11491 at 10/17/19 3:38 AM: -- I start work for this and list main subtask here: 1. Import teradat.tpcds as test data generator (refer to hive) 2. Create tpc-ds end to end test 3. Pick necessary commits that tpc-ds tests need from [flink-tpc-ds|https://github.com/flink-tpc-ds ] (base flink-1.9 and add some features) 4. Test and pass 103 tpcds tests Any suggestion are welcome. was (Author: leonard xu): I start work for this and list main subtask here: 1. Import teradat.tpcds as test data generator (refer to hive) 2. Create tpc-ds end to end test 3. Pick necessary commits that tpc-ds tests need from [flink-tpc-ds|https://github.com/flink-tpc-ds ] (base flink-1.9 and add some) 4. Test and pass 103 tpcds tests Any suggestion are welcome. > Support all TPC-DS queries > -- > > Key: FLINK-11491 > URL: https://issues.apache.org/jira/browse/FLINK-11491 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue describes the goal of supporting all TPC-DS queries on a unified > runtime for batch and streaming. > Operations might not be executed with the full performance until changes in > other Flink core components have taken place. > This includes external catalog support and Hive integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese URL: https://github.com/apache/flink/pull/9784#issuecomment-53526 ## CI report: * 198da66f5d4525d04f69bfed13eedb51652a2f38 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129428277) * be59de38e2c2710ab7a07a7a9ffbf04d92db21d5 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132082361) * 096444b219447b1d467fe63ab6278c1c28802eb0 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-11491) Support all TPC-DS queries
[ https://issues.apache.org/jira/browse/FLINK-11491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu updated FLINK-11491: Parent: (was: FLINK-11439) Issue Type: New Feature (was: Sub-task) > Support all TPC-DS queries > -- > > Key: FLINK-11491 > URL: https://issues.apache.org/jira/browse/FLINK-11491 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue describes the goal of supporting all TPC-DS queries on a unified > runtime for batch and streaming. > Operations might not be executed with the full performance until changes in > other Flink core components have taken place. > This includes external catalog support and Hive integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex
flinkbot edited a comment on issue #9798: [FLINK-14176][web]Web Ui add log url for taskmanager of vertex URL: https://github.com/apache/flink/pull/9798#issuecomment-536254451 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit ac8cc7bcba9e93fbea9374409df581d2959aecb8 (Thu Oct 17 03:29:59 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese URL: https://github.com/apache/flink/pull/9784#issuecomment-535882318 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 096444b219447b1d467fe63ab6278c1c28802eb0 (Thu Oct 17 03:27:57 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese URL: https://github.com/apache/flink/pull/9784#issuecomment-535882318 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 096444b219447b1d467fe63ab6278c1c28802eb0 (Thu Oct 17 03:25:54 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] AT-Fieldless commented on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
AT-Fieldless commented on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese URL: https://github.com/apache/flink/pull/9784#issuecomment-542981901 @xintongsong I change '数据源' and '接收器' with source and sink here, but there remain some '数据源' in other paragraphs, should I change them? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-11491) Support all TPC-DS queries
[ https://issues.apache.org/jira/browse/FLINK-11491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953360#comment-16953360 ] Jark Wu commented on FLINK-11491: - Feel free to create sub-tasks and open pull requests. > Support all TPC-DS queries > -- > > Key: FLINK-11491 > URL: https://issues.apache.org/jira/browse/FLINK-11491 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue describes the goal of supporting all TPC-DS queries on a unified > runtime for batch and streaming. > Operations might not be executed with the full performance until changes in > other Flink core components have taken place. > This includes external catalog support and Hive integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
flinkbot edited a comment on issue #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups. URL: https://github.com/apache/flink/pull/9910#issuecomment-542638190 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8d1696b97872fdd9fd3aca725c8b58ad8227f6f2 (Thu Oct 17 03:22:53 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime]
Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory
setups.
URL: https://github.com/apache/flink/pull/9910#discussion_r335794879
##
File path:
flink-core/src/main/java/org/apache/flink/api/common/operators/ResourceSpec.java
##
@@ -257,12 +273,11 @@ public boolean equals(Object obj) {
} else if (obj != null && obj.getClass() == ResourceSpec.class)
{
ResourceSpec that = (ResourceSpec) obj;
return this.cpuCores == that.cpuCores &&
- this.heapMemoryInMB ==
that.heapMemoryInMB &&
- this.directMemoryInMB ==
that.directMemoryInMB &&
- this.nativeMemoryInMB ==
that.nativeMemoryInMB &&
- this.stateSizeInMB ==
that.stateSizeInMB &&
- this.managedMemoryInMB ==
that.managedMemoryInMB &&
- Objects.equals(this.extendedResources,
that.extendedResources);
+ (this.taskHeapMemory == null ?
that.taskHeapMemory == null : this.taskHeapMemory.equals(that.taskHeapMemory))
&&
Review comment:
If we guarantee the fields cannot be null unless the `ResourceProfile` is
`UNKNOWN`, we do not need to handle the null cases for individual fields.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
flinkbot edited a comment on issue #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups. URL: https://github.com/apache/flink/pull/9910#issuecomment-542638190 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8d1696b97872fdd9fd3aca725c8b58ad8227f6f2 (Thu Oct 17 03:18:49 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups. URL: https://github.com/apache/flink/pull/9910#discussion_r335794168 ## File path: flink-core/src/main/java/org/apache/flink/api/common/operators/ResourceSpec.java ## @@ -70,72 +70,88 @@ /** How many cpu cores are needed, use double so we can specify cpu like 0.1. */ private final double cpuCores; - /** How many java heap memory in mb are needed. */ - private final int heapMemoryInMB; + /** How many task heap memory are needed. */ Review comment: How about we don't return `Optional` for the getters, but throw an `UnsupportedOperationException` when calling the getters on `UNKNOWN`? We have `Memory.ZERO` for no requirement. If a `ResourceProfile` is not `UNKNOWN`, it should always have explicit values for all the fields. I've made the default constructor private, which should be the only way to set the fields to `null`. Maybe add `checkNotNull` for arguments of all the public constructors? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14296) Improve handling of parameters nullabillity in parser module
[
https://issues.apache.org/jira/browse/FLINK-14296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-14296:
Summary: Improve handling of parameters nullabillity in parser module
(was: SqlNodes in the parser module should use an Optional for optional
parameters)
> Improve handling of parameters nullabillity in parser module
>
>
> Key: FLINK-14296
> URL: https://issues.apache.org/jira/browse/FLINK-14296
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
>Reporter: Dawid Wysakowicz
>Assignee: Jiayi Liao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I want to suggest using Optional for optional parameters in classes such as
> SqlCreateTable/SqlCreateView/SqlTableColumn etc.
> Right now we must check against null at different locations e.g.:
> {code}
> SqlNodeList partitionKey = sqlCreateTable.getPartitionKeyList();
> if (partitionKey != null) {
> partitionKeys = partitionKey
> .getList()
> .stream()
> .map(p -> ((SqlIdentifier) p).getSimple())
> .collect(Collectors.toList());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9843: [FLINK-14296] [Table SQL] Use Optional for optional parameters in parser module
flinkbot edited a comment on issue #9843: [FLINK-14296] [Table SQL] Use Optional for optional parameters in parser module URL: https://github.com/apache/flink/pull/9843#issuecomment-538717105 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 64f2cbc7910a1c80ed6eb5c09b42cbeb1ac9d2d7 (Thu Oct 17 03:16:47 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9025: [FLINK-13149][Web Frontend] support configuration and update Angular to 8.0 version
flinkbot edited a comment on issue #9025: [FLINK-13149][Web Frontend] support configuration and update Angular to 8.0 version URL: https://github.com/apache/flink/pull/9025#issuecomment-512173118 ## CI report: * e2ae53781367b527fd4d77a4a63ae3b8f354a810 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119439998) * 6f2b4c282a8d3f88b58eb6e3946dca5fc6ed9752 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/127981266) * a5367c6eff2e56223de725b5fe8b74afd1a0301c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128106324) * beacd0f8376589b9da8b42a80fb008895b7ff4fb : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-11491) Support all TPC-DS queries
[ https://issues.apache.org/jira/browse/FLINK-11491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953358#comment-16953358 ] Leonard Xu commented on FLINK-11491: I start work for this and list main subtask here: 1. Import teradat.tpcds as test data generator (refer to hive) 2. Create tpc-ds end to end test 3. Pick necessary commits that tpc-ds tests need from [flink-tpc-ds|https://github.com/flink-tpc-ds ] (base flink-1.9 and add some) 4. Test and pass 103 tpcds tests Any suggestion are welcome. > Support all TPC-DS queries > -- > > Key: FLINK-11491 > URL: https://issues.apache.org/jira/browse/FLINK-11491 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue describes the goal of supporting all TPC-DS queries on a unified > runtime for batch and streaming. > Operations might not be executed with the full performance until changes in > other Flink core components have taken place. > This includes external catalog support and Hive integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
flinkbot edited a comment on issue #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering URL: https://github.com/apache/flink/pull/9853#issuecomment-539467530 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 272536f7c0a89d283e76c187ca29cca2b0951318 (Thu Oct 17 03:12:43 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @pnowojski [committer] * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] ifndef-SleePy commented on a change in pull request #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
ifndef-SleePy commented on a change in pull request #9853:
[FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
URL: https://github.com/apache/flink/pull/9853#discussion_r335793012
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinatorTriggeringTest.java
##
@@ -289,29 +292,29 @@ public void testStopPeriodicScheduler() throws Exception
{
failureManager);
// Periodic
+ CompletableFuture periodicTriggerResult =
coord.triggerCheckpoint(
+ System.currentTimeMillis(),
+
CheckpointProperties.forCheckpoint(CheckpointRetentionPolicy.NEVER_RETAIN_AFTER_TERMINATION),
+ null,
+ true,
+ false);
try {
- coord.triggerCheckpoint(
- System.currentTimeMillis(),
-
CheckpointProperties.forCheckpoint(CheckpointRetentionPolicy.NEVER_RETAIN_AFTER_TERMINATION),
- null,
- true,
- false);
+ periodicTriggerResult.get();
fail("The triggerCheckpoint call expected an
exception");
- } catch (CheckpointException e) {
-
assertEquals(CheckpointFailureReason.PERIODIC_SCHEDULER_SHUTDOWN,
e.getCheckpointFailureReason());
+ } catch (ExecutionException e) {
+ assertTrue(e.getCause() instanceof CheckpointException);
Review comment:
Good point!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] ifndef-SleePy commented on a change in pull request #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
ifndef-SleePy commented on a change in pull request #9853:
[FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
URL: https://github.com/apache/flink/pull/9853#discussion_r335793060
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinatorTriggeringTest.java
##
@@ -289,29 +292,29 @@ public void testStopPeriodicScheduler() throws Exception
{
failureManager);
// Periodic
+ CompletableFuture periodicTriggerResult =
coord.triggerCheckpoint(
+ System.currentTimeMillis(),
+
CheckpointProperties.forCheckpoint(CheckpointRetentionPolicy.NEVER_RETAIN_AFTER_TERMINATION),
+ null,
+ true,
+ false);
try {
- coord.triggerCheckpoint(
- System.currentTimeMillis(),
-
CheckpointProperties.forCheckpoint(CheckpointRetentionPolicy.NEVER_RETAIN_AFTER_TERMINATION),
- null,
- true,
- false);
+ periodicTriggerResult.get();
fail("The triggerCheckpoint call expected an
exception");
- } catch (CheckpointException e) {
-
assertEquals(CheckpointFailureReason.PERIODIC_SCHEDULER_SHUTDOWN,
e.getCheckpointFailureReason());
+ } catch (ExecutionException e) {
+ assertTrue(e.getCause() instanceof CheckpointException);
+
assertEquals(CheckpointFailureReason.PERIODIC_SCHEDULER_SHUTDOWN,
+ ((CheckpointException)
e.getCause()).getCheckpointFailureReason());
}
// Not periodic
- try {
- coord.triggerCheckpoint(
- System.currentTimeMillis(),
-
CheckpointProperties.forCheckpoint(CheckpointRetentionPolicy.NEVER_RETAIN_AFTER_TERMINATION),
- null,
- false,
- false);
- } catch (CheckpointException e) {
- fail("Unexpected exception : " +
e.getCheckpointFailureReason().message());
- }
+ CompletableFuture nonPeriodicTriggerResult
= coord.triggerCheckpoint(
+ System.currentTimeMillis(),
+
CheckpointProperties.forCheckpoint(CheckpointRetentionPolicy.NEVER_RETAIN_AFTER_TERMINATION),
+ null,
+ false,
+ false);
+
assertFalse(nonPeriodicTriggerResult.isCompletedExceptionally());
Review comment:
Ditto.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
flinkbot edited a comment on issue #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering URL: https://github.com/apache/flink/pull/9853#issuecomment-539467530 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 272536f7c0a89d283e76c187ca29cca2b0951318 (Thu Oct 17 03:10:42 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @pnowojski [committer] * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] ifndef-SleePy commented on a change in pull request #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
ifndef-SleePy commented on a change in pull request #9853:
[FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
URL: https://github.com/apache/flink/pull/9853#discussion_r335792916
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
##
@@ -153,8 +155,8 @@
/** A handle to the current periodic trigger, to cancel it when
necessary. */
private ScheduledFuture currentPeriodicTrigger;
- /** The timestamp (via {@link System#nanoTime()}) when the last
checkpoint completed. */
- private long lastCheckpointCompletionNanos;
+ /** The relative timestamp when the next checkpoint could be triggered.
*/
+ private long earliestRelativeTimeNextCheckpointBeTriggered;
Review comment:
As mentioned before, "earliest" means it's the earliest time next trigger
could be triggered.
"relative" means it's a relative time (`System.nanotime()`).
Maybe `earliestNextCheckpointTriggerRelativeTime`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
flinkbot edited a comment on issue #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering URL: https://github.com/apache/flink/pull/9853#issuecomment-539467530 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 272536f7c0a89d283e76c187ca29cca2b0951318 (Thu Oct 17 03:07:37 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @pnowojski [committer] * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] ifndef-SleePy commented on a change in pull request #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
ifndef-SleePy commented on a change in pull request #9853:
[FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
URL: https://github.com/apache/flink/pull/9853#discussion_r335792379
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
##
@@ -153,8 +155,8 @@
/** A handle to the current periodic trigger, to cancel it when
necessary. */
private ScheduledFuture currentPeriodicTrigger;
- /** The timestamp (via {@link System#nanoTime()}) when the last
checkpoint completed. */
- private long lastCheckpointCompletionNanos;
+ /** The relative timestamp when the next checkpoint could be triggered.
*/
Review comment:
Actually it's not the time which next checkpoint "should" be triggered. The
`earliestRelativeTimeNextCheckpointBeTriggered` means next trigger couldn't be
launched until this time due to the restriction of `minPauseBetweenCheckpoints`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
flinkbot edited a comment on issue #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups. URL: https://github.com/apache/flink/pull/9910#issuecomment-542638190 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8d1696b97872fdd9fd3aca725c8b58ad8227f6f2 (Thu Oct 17 03:04:35 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups. URL: https://github.com/apache/flink/pull/9910#discussion_r335791885 ## File path: flink-core/src/main/java/org/apache/flink/api/common/operators/ResourceSpec.java ## @@ -70,72 +70,88 @@ /** How many cpu cores are needed, use double so we can specify cpu like 0.1. */ private final double cpuCores; - /** How many java heap memory in mb are needed. */ - private final int heapMemoryInMB; + /** How many task heap memory are needed. */ + private final MemorySize taskHeapMemory; - /** How many nio direct memory in mb are needed. */ - private final int directMemoryInMB; + /** How many task off-heap memory are needed. */ + private final MemorySize taskOffHeapMemory; - /** How many native memory in mb are needed. */ - private final int nativeMemoryInMB; + /** How many on-heap managed memory are needed. */ + private final MemorySize onHeapManagedMemory; Review comment: I would like to keep it 'onHeap'. The name _On Heap Managed Memory_ aligns with the FLIP doc, config options, `TaskExecutorResourceSpec`, etc. I'd like to keep them consistent for less confusing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lamber-ken commented on issue #6195: [FLINK-9543][METRICS] Expose JobMaster ID to metric system
lamber-ken commented on issue #6195: [FLINK-9543][METRICS] Expose JobMaster ID
to metric system
URL: https://github.com/apache/flink/pull/6195#issuecomment-542976828
> Closing this PR for now.
Hi, @zentol, maybe it's time to bring up this issue again now.
We can't distinguish some metrics of jobmanager, because these metrics don't
contains `ResourceID` like
`flink_jobmanager_Status_JVM_CPU_Load`、`flink_jobmanager_Status_JVM_CPU_Time`
and so on.
From `org.apache.flink.runtime.metrics.util.MetricUtils`, we can see that
missing ResourceID info when init JobManagerMetricGroup.
```
public static JobManagerMetricGroup instantiateJobManagerMetricGroup(
final MetricRegistry metricRegistry,
final String hostname) {
final JobManagerMetricGroup jobManagerMetricGroup = new
JobManagerMetricGroup(
metricRegistry,
hostname);
...
return jobManagerMetricGroup;
}
public static TaskManagerMetricGroup instantiateTaskManagerMetricGroup(
MetricRegistry metricRegistry,
TaskManagerLocation taskManagerLocation,
NetworkEnvironment network) {
final TaskManagerMetricGroup taskManagerMetricGroup = new
TaskManagerMetricGroup(
metricRegistry,
taskManagerLocation.getHostname(),
taskManagerLocation.getResourceID().toString());
...
return taskManagerMetricGroup;
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] ifndef-SleePy commented on a change in pull request #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
ifndef-SleePy commented on a change in pull request #9853:
[FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
URL: https://github.com/apache/flink/pull/9853#discussion_r335791667
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
##
@@ -535,149 +528,142 @@ public boolean isShutdown() {
Review comment:
I fully understand...
The reason I didn't rework this part is the `triggerCheckpoint` will be
changed in later PRs. I was planning to make the non-functional reworking in
the very end PRs.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering
flinkbot edited a comment on issue #9853: [FLINK-13904][checkpointing] Avoid competition of checkpoint triggering URL: https://github.com/apache/flink/pull/9853#issuecomment-539467530 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 272536f7c0a89d283e76c187ca29cca2b0951318 (Thu Oct 17 02:57:30 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @pnowojski [committer] * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14422) Get taskmanager resource config in rest api
[ https://issues.apache.org/jira/browse/FLINK-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953351#comment-16953351 ] lining commented on FLINK-14422: [~xintongsong] [~trohrm...@apache.org], how about it? > Get taskmanager resource config in rest api > --- > > Key: FLINK-14422 > URL: https://issues.apache.org/jira/browse/FLINK-14422 > Project: Flink > Issue Type: Wish > Components: Runtime / Task >Reporter: lining >Priority: Major > > Now user could get network segment size metric, but couldn't calculate it‘s > memory size. Because it couldn't get page size which in config. For > standalone, each taskmanagers has its config. Should we add > TaskmangerRescourceConfig for TaskExecutor, then user could get it from rest. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.
xintongsong commented on a change in pull request #9910: [FLINK-14405][runtime] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups. URL: https://github.com/apache/flink/pull/9910#discussion_r335790615 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/types/ResourceProfile.java ## @@ -51,11 +52,19 @@ private static final long serialVersionUID = 1L; - /** A ResourceProfile that indicates an unknown set of resources. */ + /** +* A ResourceProfile that indicates an unknown resource requirement. +* This is mainly used for describing resource requirements that the exact amount of resource needed is not specified. +* It can also be used for describing remaining resource of a multi task slot that contains tasks with unknown resource requirements. +* It should not used for describing total resource of a task executor / slot, which should always be specific. +* */ public static final ResourceProfile UNKNOWN = new ResourceProfile(); - /** ResourceProfile which matches any other ResourceProfile. */ - public static final ResourceProfile ANY = new ResourceProfile(Double.MAX_VALUE, Integer.MAX_VALUE, Integer.MAX_VALUE, Integer.MAX_VALUE, Integer.MAX_VALUE, Integer.MAX_VALUE, Collections.emptyMap()); + /** +* A ResourceProfile that indicates infinite resource that matches any resource requirement, for testability purpose only. +* */ + @VisibleForTesting + public static final ResourceProfile INFINITE = new ResourceProfile(Double.MAX_VALUE, Integer.MAX_VALUE, Integer.MAX_VALUE, Integer.MAX_VALUE, Integer.MAX_VALUE, Integer.MAX_VALUE, Collections.emptyMap()); Review comment: I think renaming `ANY` to `INFINITE` is indeed helpful. `ANY` is ambiguous. People may misunderstand this as "requesting a slot with any resource", and thus misuse it for requirements. I see this many times in maintaining our internal version, and whenever I correct them the new misusage comes out after a while. As for `UNKNOWN`, I was also considering renaming it to `UNKNOWN_REQUIREMENT`. Then I discovered that there's one case that it's not describing the resource requirement, but describing the unknown state of a slot. That is, as I explained in the javadocs, for describing remaining resources of a slot that already contains tasks with unknown resource requirements in slot sharing scenarios (`MultiTaskSlot#reservedResources`). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
