[GitHub] [flink] flinkbot edited a comment on issue #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9
flinkbot edited a comment on issue #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9 URL: https://github.com/apache/flink/pull/9526#issuecomment-524521471 ## CI report: * acf89a30196f5f62d6d28f5b498b357db62aae31 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/124451445) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9517: [FLINK-13779][metrics] PrometheusPushGatewayReporter support push metrics with groupingKey
flinkbot edited a comment on issue #9517: [FLINK-13779][metrics] PrometheusPushGatewayReporter support push metrics with groupingKey URL: https://github.com/apache/flink/pull/9517#issuecomment-524185771 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 63209029449bc869bfe957850809564f6682ed56 (Sat Aug 24 05:27:41 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zentol commented on issue #9517: [FLINK-13779][metrics] PrometheusPushGatewayReporter support push metrics with groupingKey
zentol commented on issue #9517: [FLINK-13779][metrics] PrometheusPushGatewayReporter support push metrics with groupingKey URL: https://github.com/apache/flink/pull/9517#issuecomment-524521859 > Could you explain a bit more what the groupingKey actually accomplishes? Is it just a global tag that is applied to all metrics? To clarify, if this is all the groupingKey is then it shouldn't implemented this way; there's no reason to make this a feature specific this reporter. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9
flinkbot commented on issue #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9 URL: https://github.com/apache/flink/pull/9526#issuecomment-524521471 ## CI report: * acf89a30196f5f62d6d28f5b498b357db62aae31 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9
flinkbot commented on issue #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9 URL: https://github.com/apache/flink/pull/9526#issuecomment-524521158 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit acf89a30196f5f62d6d28f5b498b357db62aae31 (Sat Aug 24 05:13:14 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13614) Add MigrationVersion.v1_9
[
https://issues.apache.org/jira/browse/FLINK-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-13614:
---
Labels: pull-request-available (was: )
> Add MigrationVersion.v1_9
> -
>
> Key: FLINK-13614
> URL: https://issues.apache.org/jira/browse/FLINK-13614
> Project: Flink
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 1.10.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.10.0
>
>
> Add {{MigrationVersion.v1_9}}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] yanghua opened a new pull request #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9
yanghua opened a new pull request #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9 URL: https://github.com/apache/flink/pull/9526 ## What is the purpose of the change *This pull request adds a new version(v1_9) for MigrationVersion* ## Brief change log - *Add MigrationVersion.v1_9* ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / **not documented**) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning
flinkbot edited a comment on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning URL: https://github.com/apache/flink/pull/9525#issuecomment-524508878 ## CI report: * 1af1d3eed520394012342a900558baf124bc7f44 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124447039) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning
flinkbot edited a comment on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning URL: https://github.com/apache/flink/pull/9525#issuecomment-524508878 ## CI report: * 1af1d3eed520394012342a900558baf124bc7f44 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/124447039) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning
flinkbot commented on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning URL: https://github.com/apache/flink/pull/9525#issuecomment-524508878 ## CI report: * 1af1d3eed520394012342a900558baf124bc7f44 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning
flinkbot commented on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning URL: https://github.com/apache/flink/pull/9525#issuecomment-524508170 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 1af1d3eed520394012342a900558baf124bc7f44 (Sat Aug 24 02:12:46 UTC 2019) **Warnings:** * Documentation files were touched, but no `.zh.md` files: Update Chinese documentation or file Jira ticket. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13363) Add documentation for streaming aggregate performance tunning.
[ https://issues.apache.org/jira/browse/FLINK-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-13363: --- Labels: pull-request-available (was: ) > Add documentation for streaming aggregate performance tunning. > -- > > Key: FLINK-13363 > URL: https://issues.apache.org/jira/browse/FLINK-13363 > Project: Flink > Issue Type: Sub-task > Components: Documentation, Table SQL / Planner >Reporter: Jark Wu >Assignee: Jark Wu >Priority: Major > Labels: pull-request-available > Fix For: 1.9.1 > > > Add documentation for streaming aggregate performance tuning > - “Performance Tuning / Streaming Aggregation”: Explain how to solve > data-skew problem in streaming aggregation (non-windowed aggregate), the > internals, and the configurations. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] wuchong opened a new pull request #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning
wuchong opened a new pull request #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning URL: https://github.com/apache/flink/pull/9525 ## What is the purpose of the change Add documentation for streaming aggregate performance tuning ## Brief change log - Add introduction for streaming aggregate performance tuning in `dev/table/tuning/streaming_aggregation_optimization.md`. ## Verifying this change only add docs without test ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (not applicable / **docs** / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem
flinkbot edited a comment on issue #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem URL: https://github.com/apache/flink/pull/8468#issuecomment-493155027 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit baae1632aabac35e6e08b402065857c4d67491f2 (Sat Aug 24 02:02:15 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @beyond1920, @godfreyhe, @kurtyoung * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem
walterddr commented on a change in pull request #8468: [FLINK-12399][table]
Adding PushDownTableSource interface to fix FilterableTableSource
applyPredicate problem
URL: https://github.com/apache/flink/pull/8468#discussion_r317340483
##
File path:
flink-table/flink-table-common/src/main/java/org/apache/flink/table/sources/PushDownTableSource.java
##
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.sources;
+
+/**
+ * Base interface for supporting operator push-down to a {@code TableSource}.
+ * A {@code TableSource} extending this interface will be able to acquire
additional
+ * information regarding the push-down information of the table source.
+ *
+ * @param The return type of the {@link TableSource}.
+ */
+public interface PushDownTableSource {
+
+ /**
+* Describes the push down rule for this specific table source.
+*
+* @return A String explaining the {@link PushDownTableSource}.
+*/
+ String explainPushDown();
Review comment:
I figure I might be able to use some sort of default here but it will be
hard without understanding what exactly is the pushdown is. open to any
suggestion.
However I think having this API here alone prevents user from not overriding
the `pushdown` and prevents the problematic pushdown rule application
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve…
flinkbot edited a comment on issue #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve… URL: https://github.com/apache/flink/pull/9524#issuecomment-524486870 ## CI report: * b14e1be6754588ea64242b6ea79a272c78d6728a : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124439358) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9519: [FLINK-13826][table-planner][hive] Support INSERT OVERWRITE for Hive …
flinkbot edited a comment on issue #9519: [FLINK-13826][table-planner][hive] Support INSERT OVERWRITE for Hive … URL: https://github.com/apache/flink/pull/9519#issuecomment-524218956 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 285e200cf872b81b1dec964bf5cb1619e5f6d568 (Fri Aug 23 23:59:01 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13826).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] bowenli86 commented on issue #9519: [FLINK-13826][table-planner][hive] Support INSERT OVERWRITE for Hive …
bowenli86 commented on issue #9519: [FLINK-13826][table-planner][hive] Support INSERT OVERWRITE for Hive … URL: https://github.com/apache/flink/pull/9519#issuecomment-524495691 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve…
flinkbot edited a comment on issue #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve… URL: https://github.com/apache/flink/pull/9524#issuecomment-524485727 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit b14e1be6754588ea64242b6ea79a272c78d6728a (Fri Aug 23 23:27:27 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13841).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] bowenli86 commented on a change in pull request #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve…
bowenli86 commented on a change in pull request #9524: [FLINK-13841][hive]
Extend Hive version support to all 1.2 and 2.3 ve…
URL: https://github.com/apache/flink/pull/9524#discussion_r317328213
##
File path:
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/table/catalog/hive/client/HiveShimV230.java
##
@@ -99,14 +101,35 @@ public void alterTable(IMetaStoreClient client, String
databaseName, String tabl
client.alter_table(databaseName, tableName, table);
}
+ @Override
+ public void alterPartition(IMetaStoreClient client, String
databaseName, String tableName, Partition partition)
+ throws InvalidOperationException, MetaException, TException {
Review comment:
nit: needs an extra tab
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] bowenli86 commented on a change in pull request #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve…
bowenli86 commented on a change in pull request #9524: [FLINK-13841][hive]
Extend Hive version support to all 1.2 and 2.3 ve…
URL: https://github.com/apache/flink/pull/9524#discussion_r317328127
##
File path:
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/table/catalog/hive/client/HiveShimV230.java
##
@@ -86,7 +88,7 @@ public Function getFunction(IMetaStoreClient client, String
dbName, String funct
public boolean moveToTrash(FileSystem fs, Path path, Configuration
conf, boolean purge) throws IOException {
try {
Method method =
FileUtils.class.getDeclaredMethod("moveToTrash", FileSystem.class, Path.class,
- Configuration.class, boolean.class);
Review comment:
revert?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] bowenli86 commented on a change in pull request #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve…
bowenli86 commented on a change in pull request #9524: [FLINK-13841][hive]
Extend Hive version support to all 1.2 and 2.3 ve…
URL: https://github.com/apache/flink/pull/9524#discussion_r317328086
##
File path:
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/table/catalog/hive/client/HiveShimV230.java
##
@@ -99,14 +101,35 @@ public void alterTable(IMetaStoreClient client, String
databaseName, String tabl
client.alter_table(databaseName, tableName, table);
}
+ @Override
+ public void alterPartition(IMetaStoreClient client, String
databaseName, String tableName, Partition partition)
+ throws InvalidOperationException, MetaException, TException {
+ String errorMsg = "Failed to alter partition for table %s in
database %s";
+ try {
+ Method method =
client.getClass().getMethod("alter_partition", String.class, String.class,
+ Partition.class, EnvironmentContext.class);
+ method.invoke(client, databaseName, tableName,
partition, null);
+ } catch (InvocationTargetException ite) {
+ Throwable targetEx = ite.getTargetException();
+ if (targetEx instanceof TException) {
+ throw (TException) targetEx;
+ } else {
+ throw new
CatalogException(String.format(errorMsg, tableName, databaseName), targetEx);
+ }
+ } catch (NoSuchMethodException | IllegalAccessException e) {
+ throw new CatalogException(String.format(errorMsg,
tableName, databaseName), e);
+ }
+ }
+
@Override
public SimpleGenericUDAFParameterInfo
createUDAFParameterInfo(ObjectInspector[] params, boolean isWindowing, boolean
distinct, boolean allColumns) {
try {
Constructor constructor =
SimpleGenericUDAFParameterInfo.class.getConstructor(ObjectInspector[].class,
- boolean.class, boolean.class,
boolean.class);
Review comment:
revert?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] bowenli86 commented on a change in pull request #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve…
bowenli86 commented on a change in pull request #9524: [FLINK-13841][hive]
Extend Hive version support to all 1.2 and 2.3 ve…
URL: https://github.com/apache/flink/pull/9524#discussion_r317328097
##
File path:
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/table/catalog/hive/client/HiveShimV230.java
##
@@ -99,14 +101,35 @@ public void alterTable(IMetaStoreClient client, String
databaseName, String tabl
client.alter_table(databaseName, tableName, table);
}
+ @Override
+ public void alterPartition(IMetaStoreClient client, String
databaseName, String tableName, Partition partition)
+ throws InvalidOperationException, MetaException, TException {
+ String errorMsg = "Failed to alter partition for table %s in
database %s";
+ try {
+ Method method =
client.getClass().getMethod("alter_partition", String.class, String.class,
+ Partition.class, EnvironmentContext.class);
+ method.invoke(client, databaseName, tableName,
partition, null);
+ } catch (InvocationTargetException ite) {
+ Throwable targetEx = ite.getTargetException();
+ if (targetEx instanceof TException) {
+ throw (TException) targetEx;
+ } else {
+ throw new
CatalogException(String.format(errorMsg, tableName, databaseName), targetEx);
+ }
+ } catch (NoSuchMethodException | IllegalAccessException e) {
+ throw new CatalogException(String.format(errorMsg,
tableName, databaseName), e);
+ }
+ }
+
@Override
public SimpleGenericUDAFParameterInfo
createUDAFParameterInfo(ObjectInspector[] params, boolean isWindowing, boolean
distinct, boolean allColumns) {
try {
Constructor constructor =
SimpleGenericUDAFParameterInfo.class.getConstructor(ObjectInspector[].class,
- boolean.class, boolean.class,
boolean.class);
+ boolean.class, boolean.class, boolean.class);
return (SimpleGenericUDAFParameterInfo)
constructor.newInstance(params, isWindowing, distinct, allColumns);
} catch (NoSuchMethodException | IllegalAccessException |
InstantiationException | InvocationTargetException e) {
throw new CatalogException("Failed to create
SimpleGenericUDAFParameterInfo", e);
}
}
+
Review comment:
revert?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Closed] (FLINK-13442) Remove unnecessary notifySubpartitionConsumed method from view reader
[ https://issues.apache.org/jira/browse/FLINK-13442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang closed FLINK-13442. Resolution: Fixed Fixed in master : 482462f2855ea68667146491b61c347d11c52298 > Remove unnecessary notifySubpartitionConsumed method from view reader > -- > > Key: FLINK-13442 > URL: https://issues.apache.org/jira/browse/FLINK-13442 > Project: Flink > Issue Type: Improvement > Components: Runtime / Network >Reporter: zhijiang >Assignee: zhijiang >Priority: Minor > Labels: pull-request-available > Fix For: 1.10.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Currently the methods of > `NetworkSequenceViewReader#notifySubpartitionConsumed` and ` > NetworkSequenceViewReader#releaseAllResources` would be called meanwhile in > netty stack during releasing resources. > As confirmed in FLINK-13245, in order to make this release logic simple and > clean, we could remove the redundant `notifySubpartitionConsumed` from > `NetworkSequenceViewReader` side, and also remove it from > `ResultSubpartitionView` side. In the implementation of > `ResultSubpartitionView#releaseAllResources` it would further notify the > parent subpartition of consumed state via > `ResultSubpartition#notifySubpartitionConsumed` which further feedback to > parent `ResultPartition` layer via `onConsumedSubpartition`. Finally > `ResultPartition` could decide whether to release itself or not. > E.g. for the case of `ReleaseOnConsumptionResultPartition` which is mainly > used for pipelined partition, it would release partition after reference > counter decreased to 0. For the case of `ResultPartition` which would be > generated for blocking partition by default, it would never be released after > notifying consumed. And the JM/scheduler would decide when to release > partition properly. > In addition, `InputChannel#notifySubpartitionConsumed` could also be removed > as a result of above. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] zhijiangW merged pull request #9262: [FLINK-13442][network] Remove unnecessary notifySubpartitionConsumed method from view reader
zhijiangW merged pull request #9262: [FLINK-13442][network] Remove unnecessary notifySubpartitionConsumed method from view reader URL: https://github.com/apache/flink/pull/9262 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve…
flinkbot edited a comment on issue #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve… URL: https://github.com/apache/flink/pull/9524#issuecomment-524486870 ## CI report: * b14e1be6754588ea64242b6ea79a272c78d6728a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/124439358) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve…
flinkbot commented on issue #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve… URL: https://github.com/apache/flink/pull/9524#issuecomment-524486870 ## CI report: * b14e1be6754588ea64242b6ea79a272c78d6728a : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9262: [FLINK-13442][network] Remove unnecessary notifySubpartitionConsumed method from view reader
flinkbot edited a comment on issue #9262: [FLINK-13442][network] Remove unnecessary notifySubpartitionConsumed method from view reader URL: https://github.com/apache/flink/pull/9262#issuecomment-515976800 ## CI report: * bfc3f20979821e9648e5a03ef645998aa2f3587b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121083823) * 6ef24d8fcaccc95217c9ad3061b2f6d967f38abd : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121119879) * 44c5782e54cd82e082bb098027a7bafb044a8899 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121216545) * a42027d32f9c1c27822b2cd479bd25cfccf86851 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124432874) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve…
flinkbot commented on issue #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve… URL: https://github.com/apache/flink/pull/9524#issuecomment-524485727 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 50195b89c11acd498d072c4f8bf9a142d92c0cc9 (Fri Aug 23 22:52:19 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13841).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuefuz commented on issue #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve…
xuefuz commented on issue #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve… URL: https://github.com/apache/flink/pull/9524#issuecomment-524485653 cc @bowenli86 @lirui-apache @zjuwangg This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13841) Extend Hive version support to all 1.2 and 2.3 versions
[ https://issues.apache.org/jira/browse/FLINK-13841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-13841: --- Labels: pull-request-available (was: ) > Extend Hive version support to all 1.2 and 2.3 versions > --- > > Key: FLINK-13841 > URL: https://issues.apache.org/jira/browse/FLINK-13841 > Project: Flink > Issue Type: Improvement > Components: Connectors / Hive >Reporter: Xuefu Zhang >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0 > > > This is to support all 1.2 (1.2.0, 1.2.1, 1.2.2) and 2.3 (2.3.0-5) versions. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] xuefuz opened a new pull request #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve…
xuefuz opened a new pull request #9524: [FLINK-13841][hive] Extend Hive version support to all 1.2 and 2.3 ve… URL: https://github.com/apache/flink/pull/9524 …rsions ## What is the purpose of the change Support all 1.2 and 2.3 minor Hive versions instead of currently 1.2.1 and 2.3.4 only. ## Brief change log *(for example:)* - Added new shims for each version - Made inheritance relationships between the shims - Shimmed methods to deal with API difference ## Verifying this change This change is already covered by existing tests, and manually ran the tests for all versions. This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (100MB)* - *Extended integration test for recovery after master (JobManager) failure* - *Added test that validates that TaskInfo is transferred only once across recoveries* - *Manually verified the change by running a 4 node cluser with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #7713: [FLINK-10995][network] Copy intermediate serialization results only once for broadcast mode
flinkbot edited a comment on issue #7713: [FLINK-10995][network] Copy intermediate serialization results only once for broadcast mode URL: https://github.com/apache/flink/pull/7713#issuecomment-463979858 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 512499aae1ab33c5ae96c8a2100016e40d83b654 (Fri Aug 23 22:45:48 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhijiangW commented on issue #7713: [FLINK-10995][network] Copy intermediate serialization results only once for broadcast mode
zhijiangW commented on issue #7713: [FLINK-10995][network] Copy intermediate serialization results only once for broadcast mode URL: https://github.com/apache/flink/pull/7713#issuecomment-524484356 Yes, we could revisit this PR soon. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (FLINK-10790) Refactor all the StreamPartitioner implementations into runtime module
[
https://issues.apache.org/jira/browse/FLINK-10790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhijiang closed FLINK-10790.
Resolution: Won't Fix
It is not very necessary/feasible to do this now. Close it now and if necessary
we could reopen future.
> Refactor all the StreamPartitioner implementations into runtime module
> --
>
> Key: FLINK-10790
> URL: https://issues.apache.org/jira/browse/FLINK-10790
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
>Affects Versions: 1.8.0
>Reporter: zhijiang
>Assignee: zhijiang
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The motivation is for refactoring {{ChannelSelector}} interface mentioned in
> [FLINK-10662|https://issues.apache.org/jira/browse/FLINK-10662].
> The {{ChannelSelector}} implementations are separate for stream and batch
> currently, resulting in many duplicated logics for selecting channels. The
> root cause is that they are locating in different modules to not reuse the
> existing ones.
> In general, the core runtime codes should be unified for both stream and
> batch jobs as much as possible, then we can reduce the work overhead to
> maintain the duplicated codes.
> This jira would migrate the existing partitioner package from
> {{flink-streaming-jave}} to {{flink-runtime}}. Next we want to de-duplicate
> implementations for {{RoundRobinChannelSelector}} and {{OutputEmitter}}.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #7631: [FLINK-11391][shuffle] Introduce PartitionShuffleDescriptor and ShuffleDeploymentDescriptor
flinkbot edited a comment on issue #7631: [FLINK-11391][shuffle] Introduce PartitionShuffleDescriptor and ShuffleDeploymentDescriptor URL: https://github.com/apache/flink/pull/7631#issuecomment-459738324 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 48e3aacae9724023140ea5d4fc8007a1550477b1 (Fri Aug 23 22:37:40 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhijiangW commented on issue #7631: [FLINK-11391][shuffle] Introduce PartitionShuffleDescriptor and ShuffleDeploymentDescriptor
zhijiangW commented on issue #7631: [FLINK-11391][shuffle] Introduce PartitionShuffleDescriptor and ShuffleDeploymentDescriptor URL: https://github.com/apache/flink/pull/7631#issuecomment-524482975 No problem, it should be closed and I forgot it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-13841) Extend Hive version support to all 1.2 and 2.3 versions
Xuefu Zhang created FLINK-13841: --- Summary: Extend Hive version support to all 1.2 and 2.3 versions Key: FLINK-13841 URL: https://issues.apache.org/jira/browse/FLINK-13841 Project: Flink Issue Type: Improvement Components: Connectors / Hive Reporter: Xuefu Zhang Fix For: 1.10.0 This is to support all 1.2 (1.2.0, 1.2.1, 1.2.2) and 2.3 (2.3.0-5) versions. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] zhijiangW edited a comment on issue #9477: [FLINK-13765][task] Introduce the InputSelectionHandler for selecting next input in StreamTwoInputSelectableProcessor
zhijiangW edited a comment on issue #9477: [FLINK-13765][task] Introduce the
InputSelectionHandler for selecting next input in
StreamTwoInputSelectableProcessor
URL: https://github.com/apache/flink/pull/9477#issuecomment-524481057
Thanks for this review @pnowojski !
Actually this PR would not be a blocker for the following work as I
misleading before. If just from the design aspect, the benefits might not be
very obvious as you mentioned. My previous thought:
- We could make `StreamTwoInputSelectableProcessor` a bit thin after
refactoring, because we remove two non-final vars and two methods from the
processor.
- In previous way, `StreamTwoInputSelectableProcessor` would involve in
three fields {`InputSelectable`, `InputSelection`, `availableInputsMask`} to
expose too many details. After refactoring, the processor only interacts with
one component and some details are hidden behind.
I could understand your concern and it would be better if we could further
reduce some complex interactive logics provided by `TwoInputSelectionHandler`.
So I think this pr can be pending now, and after we find something very
confirmable to be done, we can merge then.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9477: [FLINK-13765][task] Introduce the InputSelectionHandler for selecting next input in StreamTwoInputSelectableProcessor
flinkbot edited a comment on issue #9477: [FLINK-13765][task] Introduce the InputSelectionHandler for selecting next input in StreamTwoInputSelectableProcessor URL: https://github.com/apache/flink/pull/9477#issuecomment-522359714 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6dd3cd3345d771fee15314914869d0e692350689 (Fri Aug 23 22:26:28 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhijiangW commented on issue #9477: [FLINK-13765][task] Introduce the InputSelectionHandler for selecting next input in StreamTwoInputSelectableProcessor
zhijiangW commented on issue #9477: [FLINK-13765][task] Introduce the
InputSelectionHandler for selecting next input in
StreamTwoInputSelectableProcessor
URL: https://github.com/apache/flink/pull/9477#issuecomment-524481057
Thanks for this review @pnowojski !
Actually this PR would not be a blocker for the following work as I
misleading before. If just from the design aspect, the benefits might not be
very obvious as you mentioned. My previous thought:
- We could make `StreamTwoInputSelectableProcessor` a bit thin after
refactoring, because we remove two non-final vars and two methods from the
processor.
- In previous way, `StreamTwoInputSelectableProcessor` would involve in
three fields {InputSelectable, InputSelection, availableInputsMask} to expose
too many details. After refactoring, the processor only interacts with one
component and some details are hidden behind.
I could understand your concern and it would be better if we could further
reduce some complex interactive logics provided by `TwoInputSelectionHandler`.
So I think this pr can be pending now, and after we find something very
confirmable to be done, we can merge then.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9518: [FLINK-13823] [Table] Fix incorrect debug log in CompileUtils
flinkbot edited a comment on issue #9518: [FLINK-13823] [Table] Fix incorrect debug log in CompileUtils URL: https://github.com/apache/flink/pull/9518#issuecomment-524212346 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 1b071229db7ead5285137fa3a2fe90391c68a9fd (Fri Aug 23 22:20:23 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13823).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9262: [FLINK-13442][network] Remove unnecessary notifySubpartitionConsumed method from view reader
flinkbot edited a comment on issue #9262: [FLINK-13442][network] Remove unnecessary notifySubpartitionConsumed method from view reader URL: https://github.com/apache/flink/pull/9262#issuecomment-515976800 ## CI report: * bfc3f20979821e9648e5a03ef645998aa2f3587b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121083823) * 6ef24d8fcaccc95217c9ad3061b2f6d967f38abd : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121119879) * 44c5782e54cd82e082bb098027a7bafb044a8899 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121216545) * a42027d32f9c1c27822b2cd479bd25cfccf86851 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/124432874) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9262: [FLINK-13442][network] Remove unnecessary notifySubpartitionConsumed method from view reader
flinkbot edited a comment on issue #9262: [FLINK-13442][network] Remove unnecessary notifySubpartitionConsumed method from view reader URL: https://github.com/apache/flink/pull/9262#issuecomment-515976800 ## CI report: * bfc3f20979821e9648e5a03ef645998aa2f3587b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121083823) * 6ef24d8fcaccc95217c9ad3061b2f6d967f38abd : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121119879) * 44c5782e54cd82e082bb098027a7bafb044a8899 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/121216545) * a42027d32f9c1c27822b2cd479bd25cfccf86851 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (FLINK-13764) Pass the counter of numRecordsIn into the constructor of StreamInputProcessor
[ https://issues.apache.org/jira/browse/FLINK-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang resolved FLINK-13764. -- Resolution: Fixed Fixed in master : a7f1fee876cde130d3b84848685ba78ba0fb6c11 > Pass the counter of numRecordsIn into the constructor of StreamInputProcessor > - > > Key: FLINK-13764 > URL: https://issues.apache.org/jira/browse/FLINK-13764 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Metrics, Runtime / Task >Reporter: zhijiang >Assignee: zhijiang >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Currently the counter of numRecordsIn is setup while processing input in > processor. In order to integrate the processing logic based on > StreamTaskInput#emitNext(Output) later, we need to pass the counter into > output functions then. So there are three reasons to do this: > * It is the precondition of following integration work. > * We could make the counter as final fields in StreamInputProcessor and > StreamTwoInputSelectableProcessor. > * We could reuse the counter setup logic for all the input processors. > There should be no side effects if we make the counter setup a bit earlier > than the previous way. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor
flinkbot edited a comment on issue #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor URL: https://github.com/apache/flink/pull/9476#issuecomment-522355405 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8effe87ff5154b882216647ac02969d33e3bebfa (Fri Aug 23 21:25:18 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhijiangW merged pull request #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor
zhijiangW merged pull request #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor URL: https://github.com/apache/flink/pull/9476 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9517: [FLINK-13779][metrics] PrometheusPushGatewayReporter support push metrics with groupingKey
flinkbot edited a comment on issue #9517: [FLINK-13779][metrics] PrometheusPushGatewayReporter support push metrics with groupingKey URL: https://github.com/apache/flink/pull/9517#issuecomment-524185771 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 63209029449bc869bfe957850809564f6682ed56 (Fri Aug 23 20:32:30 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zentol commented on a change in pull request #9517: [FLINK-13779][metrics] PrometheusPushGatewayReporter support push metrics with groupingKey
zentol commented on a change in pull request #9517: [FLINK-13779][metrics]
PrometheusPushGatewayReporter support push metrics with groupingKey
URL: https://github.com/apache/flink/pull/9517#discussion_r317291152
##
File path:
flink-metrics/flink-metrics-prometheus/src/main/java/org/apache/flink/metrics/prometheus/PrometheusPushGatewayReporter.java
##
@@ -67,25 +73,44 @@ public void open(MetricConfig config) {
}
pushGateway = new PushGateway(host + ':' + port);
- log.info("Configured PrometheusPushGatewayReporter with
{host:{}, port:{}, jobName: {}, randomJobNameSuffix:{}, deleteOnShutdown:{}}",
host, port, jobName, randomSuffix, deleteOnShutdown);
+ log.info("Configured PrometheusPushGatewayReporter with
{host:{}, port:{}, jobName:{}, randomJobNameSuffix:{}, deleteOnShutdown:{},
groupingKey:{}}",
+ host, port, jobName, randomSuffix, deleteOnShutdown,
groupingKey);
+ }
+
+ @VisibleForTesting
+ static Map parserGroupingKey(final String
groupingKeyConfig) {
+ if (!groupingKeyConfig.isEmpty()) {
+ Map groupingKey = new HashMap<>();
+ String[] kvs = groupingKeyConfig.split(",");
+ for (String kv : kvs) {
+ int idx = kv.indexOf("=");
+ if (idx < 0) {
+ continue;
Review comment:
this case implies an invalid configuration and should be logged as a warning
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol commented on a change in pull request #9517: [FLINK-13779][metrics] PrometheusPushGatewayReporter support push metrics with groupingKey
zentol commented on a change in pull request #9517: [FLINK-13779][metrics]
PrometheusPushGatewayReporter support push metrics with groupingKey
URL: https://github.com/apache/flink/pull/9517#discussion_r317290921
##
File path:
flink-metrics/flink-metrics-prometheus/src/main/java/org/apache/flink/metrics/prometheus/PrometheusPushGatewayReporter.java
##
@@ -67,25 +73,44 @@ public void open(MetricConfig config) {
}
pushGateway = new PushGateway(host + ':' + port);
- log.info("Configured PrometheusPushGatewayReporter with
{host:{}, port:{}, jobName: {}, randomJobNameSuffix:{}, deleteOnShutdown:{}}",
host, port, jobName, randomSuffix, deleteOnShutdown);
+ log.info("Configured PrometheusPushGatewayReporter with
{host:{}, port:{}, jobName:{}, randomJobNameSuffix:{}, deleteOnShutdown:{},
groupingKey:{}}",
+ host, port, jobName, randomSuffix, deleteOnShutdown,
groupingKey);
+ }
+
+ @VisibleForTesting
+ static Map parserGroupingKey(final String
groupingKeyConfig) {
Review comment:
rename to `parseGroupingKey`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol commented on a change in pull request #9517: [FLINK-13779][metrics] PrometheusPushGatewayReporter support push metrics with groupingKey
zentol commented on a change in pull request #9517: [FLINK-13779][metrics]
PrometheusPushGatewayReporter support push metrics with groupingKey
URL: https://github.com/apache/flink/pull/9517#discussion_r317290696
##
File path:
flink-metrics/flink-metrics-prometheus/src/main/java/org/apache/flink/metrics/prometheus/PrometheusPushGatewayReporter.java
##
@@ -67,25 +73,44 @@ public void open(MetricConfig config) {
}
pushGateway = new PushGateway(host + ':' + port);
- log.info("Configured PrometheusPushGatewayReporter with
{host:{}, port:{}, jobName: {}, randomJobNameSuffix:{}, deleteOnShutdown:{}}",
host, port, jobName, randomSuffix, deleteOnShutdown);
+ log.info("Configured PrometheusPushGatewayReporter with
{host:{}, port:{}, jobName:{}, randomJobNameSuffix:{}, deleteOnShutdown:{},
groupingKey:{}}",
+ host, port, jobName, randomSuffix, deleteOnShutdown,
groupingKey);
+ }
+
+ @VisibleForTesting
+ static Map parserGroupingKey(final String
groupingKeyConfig) {
+ if (!groupingKeyConfig.isEmpty()) {
+ Map groupingKey = new HashMap<>();
+ String[] kvs = groupingKeyConfig.split(",");
+ for (String kv : kvs) {
+ int idx = kv.indexOf("=");
+ if (idx < 0) {
+ continue;
+ }
+ groupingKey.put(kv.substring(0, idx),
kv.substring(idx + 1));
+ }
+
+ return groupingKey;
+ }
+ return null;
}
@Override
public void report() {
try {
- pushGateway.push(CollectorRegistry.defaultRegistry,
jobName);
+ pushGateway.push(CollectorRegistry.defaultRegistry,
jobName, groupingKey);
Review comment:
is there a difference in behavior between passing null and an empty
collection?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol commented on a change in pull request #9517: [FLINK-13779][metrics] PrometheusPushGatewayReporter support push metrics with groupingKey
zentol commented on a change in pull request #9517: [FLINK-13779][metrics]
PrometheusPushGatewayReporter support push metrics with groupingKey
URL: https://github.com/apache/flink/pull/9517#discussion_r317291152
##
File path:
flink-metrics/flink-metrics-prometheus/src/main/java/org/apache/flink/metrics/prometheus/PrometheusPushGatewayReporter.java
##
@@ -67,25 +73,44 @@ public void open(MetricConfig config) {
}
pushGateway = new PushGateway(host + ':' + port);
- log.info("Configured PrometheusPushGatewayReporter with
{host:{}, port:{}, jobName: {}, randomJobNameSuffix:{}, deleteOnShutdown:{}}",
host, port, jobName, randomSuffix, deleteOnShutdown);
+ log.info("Configured PrometheusPushGatewayReporter with
{host:{}, port:{}, jobName:{}, randomJobNameSuffix:{}, deleteOnShutdown:{},
groupingKey:{}}",
+ host, port, jobName, randomSuffix, deleteOnShutdown,
groupingKey);
+ }
+
+ @VisibleForTesting
+ static Map parserGroupingKey(final String
groupingKeyConfig) {
+ if (!groupingKeyConfig.isEmpty()) {
+ Map groupingKey = new HashMap<>();
+ String[] kvs = groupingKeyConfig.split(",");
+ for (String kv : kvs) {
+ int idx = kv.indexOf("=");
+ if (idx < 0) {
+ continue;
Review comment:
this cast implies an invalid configuration and should be logged as a warning
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (FLINK-13430) Configure sending travis build notifications to bui...@flink.apache.org
[ https://issues.apache.org/jira/browse/FLINK-13430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-13430: - Fix Version/s: 1.10.0 > Configure sending travis build notifications to bui...@flink.apache.org > --- > > Key: FLINK-13430 > URL: https://issues.apache.org/jira/browse/FLINK-13430 > Project: Flink > Issue Type: Task > Components: Build System >Reporter: Jark Wu >Assignee: Jark Wu >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0, 1.9.1 > > Time Spent: 20m > Remaining Estimate: 0h > > As discussed in the mailing list[1], the community want to send travis build > notifications to bui...@flink.apache.org mailing list. > [1]: > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Setup-a-builds-flink-apache-org-mailing-list-for-travis-builds-tt30778.html -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9479: [FLINK-13768] [Documentation] Add documentation regarding path style access for s3
flinkbot edited a comment on issue #9479: [FLINK-13768] [Documentation] Add documentation regarding path style access for s3 URL: https://github.com/apache/flink/pull/9479#issuecomment-522435214 ## CI report: * f4b4a9e1343252b7783d8fd39d8ab25e39b400d7 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/123671193) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9479: [FLINK-13768] [Documentation] Add documentation regarding path style access for s3
flinkbot edited a comment on issue #9479: [FLINK-13768] [Documentation] Add documentation regarding path style access for s3 URL: https://github.com/apache/flink/pull/9479#issuecomment-522434059 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit f4b4a9e1343252b7783d8fd39d8ab25e39b400d7 (Fri Aug 23 18:32:28 UTC 2019) **Warnings:** * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13768).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] achyuthsamudrala commented on issue #9479: [FLINK-13768] [Documentation] Add documentation regarding path style access for s3
achyuthsamudrala commented on issue #9479: [FLINK-13768] [Documentation] Add documentation regarding path style access for s3 URL: https://github.com/apache/flink/pull/9479#issuecomment-524417655 Hi, can someone look at this? @aljoscha @zentol @NicoK This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (FLINK-13685) YARN deployment hangs when requested container is above the maximum vcore size
[ https://issues.apache.org/jira/browse/FLINK-13685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Márton Balassi reassigned FLINK-13685: -- Assignee: Josh YEH (was: Matyas Orhidi) > YARN deployment hangs when requested container is above the maximum vcore size > -- > > Key: FLINK-13685 > URL: https://issues.apache.org/jira/browse/FLINK-13685 > Project: Flink > Issue Type: New Feature > Components: Deployment / YARN >Affects Versions: 1.8.1 >Reporter: Gyula Fora >Assignee: Josh YEH >Priority: Major > > When deploying on YARN flink explicitly checks whether the requested > container vcores is smaller than yarn.nodemanager.resource.cpu-vcores. > Unfortunately this doesn't guarantee that the request can be fulfilled if > yarn.scheduler.maximum-allocation-vcores is set to a lower number. > In these cases flink will just keep trying to request a container but will > never succeed. > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem
flinkbot edited a comment on issue #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem URL: https://github.com/apache/flink/pull/8468#issuecomment-524372146 ## CI report: * baae1632aabac35e6e08b402065857c4d67491f2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124398771) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9441: [FLINK-13723][docs] use liquid-c for faster doc generation
flinkbot edited a comment on issue #9441: [FLINK-13723][docs] use liquid-c for faster doc generation URL: https://github.com/apache/flink/pull/9441#issuecomment-521435635 ## CI report: * 874bb04d04f1b02cd916682a07303794cc04c7b5 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/123297359) * 16a7099512a9662e8a7574b3ab1d6e3b1f2d7797 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/123298764) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor
flinkbot edited a comment on issue #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor URL: https://github.com/apache/flink/pull/9476#issuecomment-522355867 ## CI report: * df58cd55f5ca284e412ebbfc35c336e4f012974c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123646651) * 870e9199aa5afbbdea0bccffcb2fa4c2fa26ec8f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124293088) * 8effe87ff5154b882216647ac02969d33e3bebfa : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124380448) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13292) NullPointerException when reading a string field in a nested struct from an Orc file.
[
https://issues.apache.org/jira/browse/FLINK-13292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914470#comment-16914470
]
Nithish commented on FLINK-13292:
-
tried in Flink 1.8.0 and null string in `*record1*` are being processed without
any problem
{code:java}
import org.apache.hadoop.conf.Configuration;
import org.apache.flink.orc.OrcRowInputFormat;
final String TEST_SCHEMA_COMPOSITES_NULLS =
"struct<" +
"int1:int," +
"record1:struct," +
"list1:array>>>," +
"list2:array>" +
">";
env.readFile(new OrcRowInputFormat(
"~/path/test-data-composites-with-nulls.orc",
TEST_SCHEMA_COMPOSITES_NULLS,
new Configuration()
),"~/path/test-data-composites-with-nulls.orc").writeAsText("file:///tmp/test/fromOrc");
{code}
I notice that you have an ACID ORC file, I will try to see if the issue is
replicatable by creating a ACID ORC file
> NullPointerException when reading a string field in a nested struct from an
> Orc file.
> -
>
> Key: FLINK-13292
> URL: https://issues.apache.org/jira/browse/FLINK-13292
> Project: Flink
> Issue Type: Bug
> Components: Connectors / ORC
>Affects Versions: 1.8.0
>Reporter: Alejandro Sellero
>Priority: Major
>
> When I try to read an Orc file using flink-orc an NullPointerException
> exception is thrown.
> I think this issue could be related with this closed issue
> https://issues.apache.org/jira/browse/FLINK-8230
> This happens when trying to read the string fields in a nested struct. This
> is my schema:
> {code:java}
> "struct<" +
> "operation:int," +
> "originalTransaction:bigInt," +
> "bucket:int," +
> "rowId:bigInt," +
> "currentTransaction:bigInt," +
> "row:struct<" +
> "id:int," +
> "headline:string," +
> "user_id:int," +
> "company_id:int," +
> "created_at:timestamp," +
> "updated_at:timestamp," +
> "link:string," +
> "is_html:tinyint," +
> "source:string," +
> "company_feed_id:int," +
> "editable:tinyint," +
> "body_clean:string," +
> "activitystream_activity_id:bigint," +
> "uniqueness_checksum:string," +
> "rating:string," +
> "review_id:int," +
> "soft_deleted:tinyint," +
> "type:string," +
> "metadata:string," +
> "url:string," +
> "imagecache_uuid:string," +
> "video_id:int" +
> ">>",{code}
> {code:java}
> [error] Caused by: java.lang.NullPointerException
> [error] at java.lang.String.checkBounds(String.java:384)
> [error] at java.lang.String.(String.java:462)
> [error] at
> org.apache.flink.orc.OrcBatchReader.readString(OrcBatchReader.java:1216)
> [error] at
> org.apache.flink.orc.OrcBatchReader.readNonNullBytesColumnAsString(OrcBatchReader.java:328)
> [error] at
> org.apache.flink.orc.OrcBatchReader.readField(OrcBatchReader.java:215)
> [error] at
> org.apache.flink.orc.OrcBatchReader.readNonNullStructColumn(OrcBatchReader.java:453)
> [error] at
> org.apache.flink.orc.OrcBatchReader.readField(OrcBatchReader.java:250)
> [error] at
> org.apache.flink.orc.OrcBatchReader.fillRows(OrcBatchReader.java:143)
> [error] at
> org.apache.flink.orc.OrcRowInputFormat.ensureBatch(OrcRowInputFormat.java:333)
> [error] at
> org.apache.flink.orc.OrcRowInputFormat.reachedEnd(OrcRowInputFormat.java:313)
> [error] at
> org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:190)
> [error] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
> [error] at java.lang.Thread.run(Thread.java:748){code}
> Instead to use the TableApi I am trying to read the orc files in the Batch
> mode as following:
> {code:java}
> env
> .readFile(
> new OrcRowInputFormat(
> "",
> "SCHEMA_GIVEN_BEFORE",
> new HadoopConfiguration()
> ),
> "PATH_TO_FOLDER"
> )
> .writeAsText("file:///tmp/test/fromOrc")
> {code}
> Thanks for your support
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9440: [FLINK-13724][docs] remove unnecessary whitespace from the generated pages
flinkbot edited a comment on issue #9440: [FLINK-13724][docs] remove unnecessary whitespace from the generated pages URL: https://github.com/apache/flink/pull/9440#issuecomment-521435596 ## CI report: * 9fd30468bb3d2031219dd10956401c410ec81cf0 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/123297374) * fa2d2f9f9bed875ed184cd641bbfaf141ebad364 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123298777) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9441: [FLINK-13723][docs] use liquid-c for faster doc generation
flinkbot edited a comment on issue #9441: [FLINK-13723][docs] use liquid-c for faster doc generation URL: https://github.com/apache/flink/pull/9441#issuecomment-521433294 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 16a7099512a9662e8a7574b3ab1d6e3b1f2d7797 (Fri Aug 23 17:07:56 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] StephanEwen commented on issue #9441: [FLINK-13723][docs] use liquid-c for faster doc generation
StephanEwen commented on issue #9441: [FLINK-13723][docs] use liquid-c for faster doc generation URL: https://github.com/apache/flink/pull/9441#issuecomment-524391054 Looks good, merging this... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9440: [FLINK-13724][docs] remove unnecessary whitespace from the generated pages
flinkbot edited a comment on issue #9440: [FLINK-13724][docs] remove unnecessary whitespace from the generated pages URL: https://github.com/apache/flink/pull/9440#issuecomment-521432353 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit fa2d2f9f9bed875ed184cd641bbfaf141ebad364 (Fri Aug 23 17:01:49 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] StephanEwen commented on issue #9440: [FLINK-13724][docs] remove unnecessary whitespace from the generated pages
StephanEwen commented on issue #9440: [FLINK-13724][docs] remove unnecessary whitespace from the generated pages URL: https://github.com/apache/flink/pull/9440#issuecomment-524389170 Looks good, merging this... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9439: [FLINK-13728][docs] fix wrong closing tag order in sidenav
flinkbot edited a comment on issue #9439: [FLINK-13728][docs] fix wrong closing tag order in sidenav URL: https://github.com/apache/flink/pull/9439#issuecomment-521431522 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 530bf78fdb79319e0a9f28e0fdf679559cee7c4c (Fri Aug 23 16:48:34 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] StephanEwen commented on issue #9439: [FLINK-13728][docs] fix wrong closing tag order in sidenav
StephanEwen commented on issue #9439: [FLINK-13728][docs] fix wrong closing tag order in sidenav URL: https://github.com/apache/flink/pull/9439#issuecomment-524384820 Looks good to me, merging this in the next batch... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (FLINK-13430) Configure sending travis build notifications to bui...@flink.apache.org
[ https://issues.apache.org/jira/browse/FLINK-13430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu resolved FLINK-13430. - Fix Version/s: 1.9.1 Resolution: Fixed master: aa9ac7ea6608c259963aee9b7d9e27cbd940ad67 1.9.1: bb7fcf54bc57f76456312b4d87d4d41902e56dec > Configure sending travis build notifications to bui...@flink.apache.org > --- > > Key: FLINK-13430 > URL: https://issues.apache.org/jira/browse/FLINK-13430 > Project: Flink > Issue Type: Task > Components: Build System >Reporter: Jark Wu >Assignee: Jark Wu >Priority: Major > Labels: pull-request-available > Fix For: 1.9.1 > > Time Spent: 20m > Remaining Estimate: 0h > > As discussed in the mailing list[1], the community want to send travis build > notifications to bui...@flink.apache.org mailing list. > [1]: > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Setup-a-builds-flink-apache-org-mailing-list-for-travis-builds-tt30778.html -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org
flinkbot edited a comment on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org URL: https://github.com/apache/flink/pull/9230#issuecomment-515090400 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 4c78488f133081281a4c5df362b358cdb80002fa (Fri Aug 23 16:45:31 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] asfgit closed pull request #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org
asfgit closed pull request #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org URL: https://github.com/apache/flink/pull/9230 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9502: [FLINK-13814][hive] HiveTableSink should strip quotes from partition …
flinkbot edited a comment on issue #9502: [FLINK-13814][hive] HiveTableSink should strip quotes from partition … URL: https://github.com/apache/flink/pull/9502#issuecomment-523777539 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 5ad14b3392b439f2b63b9fb079bf5546c874607d (Fri Aug 23 16:22:07 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org
flinkbot edited a comment on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org URL: https://github.com/apache/flink/pull/9230#issuecomment-515090400 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 4c78488f133081281a4c5df362b358cdb80002fa (Fri Aug 23 16:21:06 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuefuz commented on a change in pull request #9502: [FLINK-13814][hive] HiveTableSink should strip quotes from partition …
xuefuz commented on a change in pull request #9502: [FLINK-13814][hive]
HiveTableSink should strip quotes from partition …
URL: https://github.com/apache/flink/pull/9502#discussion_r317206172
##
File path:
flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/TableEnvHiveConnectorTest.java
##
@@ -171,6 +172,57 @@ public void testDecimal() throws Exception {
}
}
+ @Test
+ public void testStaticPartition() throws Exception {
+ hiveShell.execute("create database db1");
+ try {
+ hiveShell.execute("create table db1.src (x int)");
+ hiveShell.insertInto("db1",
"src").addRow(1).addRow(2).commit();
+ hiveShell.execute("create table db1.dest (x int)
partitioned by (p1 string, p2 double)");
+ TableEnvironment tableEnv =
getTableEnvWithHiveCatalog();
+ tableEnv.sqlUpdate("insert into db1.dest partition
(p1='1''1', p2=1.1) select x from db1.src");
Review comment:
I'm wondering what this means and why a user might do this: p1='1''1'
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org
wuchong commented on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org URL: https://github.com/apache/flink/pull/9230#issuecomment-524376222 The HTML is enabled. And I tested several emails and looks good now. I will merge this then. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13292) NullPointerException when reading a string field in a nested struct from an Orc file.
[
https://issues.apache.org/jira/browse/FLINK-13292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914427#comment-16914427
]
Nithish commented on FLINK-13292:
-
As far as I see
[https://github.com/apache/flink/blob/master/flink-connectors/flink-orc/src/test/java/org/apache/flink/orc/OrcRowInputFormatTest.java#L684]
Tests that OrcRowInputFormat Handles Null cases in Strings which is why I
believe the failure you have encountered could be due to a data issue
> NullPointerException when reading a string field in a nested struct from an
> Orc file.
> -
>
> Key: FLINK-13292
> URL: https://issues.apache.org/jira/browse/FLINK-13292
> Project: Flink
> Issue Type: Bug
> Components: Connectors / ORC
>Affects Versions: 1.8.0
>Reporter: Alejandro Sellero
>Priority: Major
>
> When I try to read an Orc file using flink-orc an NullPointerException
> exception is thrown.
> I think this issue could be related with this closed issue
> https://issues.apache.org/jira/browse/FLINK-8230
> This happens when trying to read the string fields in a nested struct. This
> is my schema:
> {code:java}
> "struct<" +
> "operation:int," +
> "originalTransaction:bigInt," +
> "bucket:int," +
> "rowId:bigInt," +
> "currentTransaction:bigInt," +
> "row:struct<" +
> "id:int," +
> "headline:string," +
> "user_id:int," +
> "company_id:int," +
> "created_at:timestamp," +
> "updated_at:timestamp," +
> "link:string," +
> "is_html:tinyint," +
> "source:string," +
> "company_feed_id:int," +
> "editable:tinyint," +
> "body_clean:string," +
> "activitystream_activity_id:bigint," +
> "uniqueness_checksum:string," +
> "rating:string," +
> "review_id:int," +
> "soft_deleted:tinyint," +
> "type:string," +
> "metadata:string," +
> "url:string," +
> "imagecache_uuid:string," +
> "video_id:int" +
> ">>",{code}
> {code:java}
> [error] Caused by: java.lang.NullPointerException
> [error] at java.lang.String.checkBounds(String.java:384)
> [error] at java.lang.String.(String.java:462)
> [error] at
> org.apache.flink.orc.OrcBatchReader.readString(OrcBatchReader.java:1216)
> [error] at
> org.apache.flink.orc.OrcBatchReader.readNonNullBytesColumnAsString(OrcBatchReader.java:328)
> [error] at
> org.apache.flink.orc.OrcBatchReader.readField(OrcBatchReader.java:215)
> [error] at
> org.apache.flink.orc.OrcBatchReader.readNonNullStructColumn(OrcBatchReader.java:453)
> [error] at
> org.apache.flink.orc.OrcBatchReader.readField(OrcBatchReader.java:250)
> [error] at
> org.apache.flink.orc.OrcBatchReader.fillRows(OrcBatchReader.java:143)
> [error] at
> org.apache.flink.orc.OrcRowInputFormat.ensureBatch(OrcRowInputFormat.java:333)
> [error] at
> org.apache.flink.orc.OrcRowInputFormat.reachedEnd(OrcRowInputFormat.java:313)
> [error] at
> org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:190)
> [error] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
> [error] at java.lang.Thread.run(Thread.java:748){code}
> Instead to use the TableApi I am trying to read the orc files in the Batch
> mode as following:
> {code:java}
> env
> .readFile(
> new OrcRowInputFormat(
> "",
> "SCHEMA_GIVEN_BEFORE",
> new HadoopConfiguration()
> ),
> "PATH_TO_FOLDER"
> )
> .writeAsText("file:///tmp/test/fromOrc")
> {code}
> Thanks for your support
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem
flinkbot edited a comment on issue #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem URL: https://github.com/apache/flink/pull/8468#issuecomment-524372146 ## CI report: * baae1632aabac35e6e08b402065857c4d67491f2 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/124398771) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem
flinkbot edited a comment on issue #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem URL: https://github.com/apache/flink/pull/8468#issuecomment-493155027 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit baae1632aabac35e6e08b402065857c4d67491f2 (Fri Aug 23 16:09:53 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @beyond1920, @godfreyhe, @kurtyoung * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] walterddr commented on issue #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem
walterddr commented on issue #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem URL: https://github.com/apache/flink/pull/8468#issuecomment-524372656 @twalthr @fhueske can you please help take a look at this PR and see if this can be the right solution? So far we've seen some reporting related to this issue in ML as well as the [JIRA comments](https://issues.apache.org/jira/browse/FLINK-12399). Thanks -Rong This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem
flinkbot commented on issue #8468: [FLINK-12399][table] Adding PushDownTableSource interface to fix FilterableTableSource applyPredicate problem URL: https://github.com/apache/flink/pull/8468#issuecomment-524372146 ## CI report: * baae1632aabac35e6e08b402065857c4d67491f2 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9497: [FLINK-13794][client] Remove unused logic of printStatusDuringExecution
flinkbot edited a comment on issue #9497: [FLINK-13794][client] Remove unused logic of printStatusDuringExecution URL: https://github.com/apache/flink/pull/9497#issuecomment-523361497 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6865ffe41c6c08baa0d61a7c5b5a9fdeafb5d14d (Fri Aug 23 16:00:44 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] TisonKun commented on issue #9497: [FLINK-13794][client] Remove unused logic of printStatusDuringExecution
TisonKun commented on issue #9497: [FLINK-13794][client] Remove unused logic of printStatusDuringExecution URL: https://github.com/apache/flink/pull/9497#issuecomment-524369543 @mxm now we pass api compatibility test also other tests. I think this pull request has reached a mergeable state. Please take another pass of review :-) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9497: [FLINK-13794][client] Remove unused logic of printStatusDuringExecution
flinkbot edited a comment on issue #9497: [FLINK-13794][client] Remove unused logic of printStatusDuringExecution URL: https://github.com/apache/flink/pull/9497#issuecomment-523364967 ## CI report: * b383982c0f9353b799ba85ca2635cd9b5f3e24a8 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124020443) * dec3458bf0f82c673cbcc9b686f590a83062 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124183689) * f97118b164679e2f771ab242c57dcbcff8d51053 : PENDING [Build](TBD) * 22085a6106679b1d1d4ab885890c106a246b0df5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124341963) * 46b618cc661100386c49d87e1ea40a8cec1f6ba7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124354676) * 6865ffe41c6c08baa0d61a7c5b5a9fdeafb5d14d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124380484) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor
flinkbot edited a comment on issue #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor URL: https://github.com/apache/flink/pull/9476#issuecomment-522355867 ## CI report: * df58cd55f5ca284e412ebbfc35c336e4f012974c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123646651) * 870e9199aa5afbbdea0bccffcb2fa4c2fa26ec8f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124293088) * 8effe87ff5154b882216647ac02969d33e3bebfa : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/124380448) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor
flinkbot edited a comment on issue #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor URL: https://github.com/apache/flink/pull/9476#issuecomment-522355405 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8effe87ff5154b882216647ac02969d33e3bebfa (Fri Aug 23 15:38:18 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhijiangW commented on issue #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor
zhijiangW commented on issue #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor URL: https://github.com/apache/flink/pull/9476#issuecomment-524362098 @flinkbot run travis This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor
flinkbot edited a comment on issue #9476: [FLINK-13764][task, metrics] Pass the counter of numRecordsIn into the constructor of StreamInputProcessor URL: https://github.com/apache/flink/pull/9476#issuecomment-522355867 ## CI report: * df58cd55f5ca284e412ebbfc35c336e4f012974c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123646651) * 870e9199aa5afbbdea0bccffcb2fa4c2fa26ec8f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124293088) * 8effe87ff5154b882216647ac02969d33e3bebfa : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124380448) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (FLINK-13573) Merge SubmittedJobGraph into JobGraph
[
https://issues.apache.org/jira/browse/FLINK-13573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann resolved FLINK-13573.
---
Resolution: Fixed
Fixed via 2b9410ee1bd493cca86c479ec0f0a34ccbc842d4
> Merge SubmittedJobGraph into JobGraph
> -
>
> Key: FLINK-13573
> URL: https://issues.apache.org/jira/browse/FLINK-13573
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: TisonKun
>Assignee: TisonKun
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> As time goes on, {{SubmittedJobGraph}} becomes a thin wrapper of {{JobGraph}}
> without any additional information. It is reasonable that we merge
> {{SubmittedJobGraph}} into {{JobGraph}} and use only {{JobGraph}}.
> WDYT? cc [~till.rohrmann] [~GJL]
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9514: [FLINK-13573][coordination] Merge SubmittedJobGraph into JobGraph
flinkbot edited a comment on issue #9514: [FLINK-13573][coordination] Merge SubmittedJobGraph into JobGraph URL: https://github.com/apache/flink/pull/9514#issuecomment-523997004 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6092a6154f8275957d306c9d04b780ccd374b5d9 (Fri Aug 23 15:22:03 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] TisonKun commented on issue #9514: [FLINK-13573][coordination] Merge SubmittedJobGraph into JobGraph
TisonKun commented on issue #9514: [FLINK-13573][coordination] Merge SubmittedJobGraph into JobGraph URL: https://github.com/apache/flink/pull/9514#issuecomment-524356368 Thanks for your review @tillrohrmann ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] tillrohrmann closed pull request #9514: [FLINK-13573][coordination] Merge SubmittedJobGraph into JobGraph
tillrohrmann closed pull request #9514: [FLINK-13573][coordination] Merge SubmittedJobGraph into JobGraph URL: https://github.com/apache/flink/pull/9514 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9520: [hotfix][docs] wrong method name in state-processor-api example
flinkbot edited a comment on issue #9520: [hotfix][docs] wrong method name in state-processor-api example URL: https://github.com/apache/flink/pull/9520#issuecomment-524232071 ## CI report: * b30983f20835f9f4aaddcdd974c9b9664f5ab142 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124339388) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9395: [hotfix][streaming] Fix the wrong word used in the comments in class KeyedStream
flinkbot edited a comment on issue #9395: [hotfix][streaming] Fix the wrong word used in the comments in class KeyedStream URL: https://github.com/apache/flink/pull/9395#issuecomment-519551891 ## CI report: * 6a5450aa95ebeae0a2c011d64f23ec60fd858485 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/122455285) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9477: [FLINK-13765][task] Introduce the InputSelectionHandler for selecting next input in StreamTwoInputSelectableProcessor
flinkbot edited a comment on issue #9477: [FLINK-13765][task] Introduce the InputSelectionHandler for selecting next input in StreamTwoInputSelectableProcessor URL: https://github.com/apache/flink/pull/9477#issuecomment-522359714 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6dd3cd3345d771fee15314914869d0e692350689 (Fri Aug 23 15:06:46 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #9477: [FLINK-13765][task] Introduce the InputSelectionHandler for selecting next input in StreamTwoInputSelectableProcessor
pnowojski commented on a change in pull request #9477: [FLINK-13765][task]
Introduce the InputSelectionHandler for selecting next input in
StreamTwoInputSelectableProcessor
URL: https://github.com/apache/flink/pull/9477#discussion_r317160983
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/io/InputSelectionHandler.java
##
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.runtime.io;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.streaming.api.operators.InputSelectable;
+import org.apache.flink.streaming.api.operators.InputSelection;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * This handler is mainly used for selecting the next available input index
+ * in {@link StreamTwoInputSelectableProcessor}.
+ */
+@Internal
+public class InputSelectionHandler {
+
+ private final InputSelectable inputSelector;
+
+ private InputSelection inputSelection;
+
+ private int availableInputsMask;
+
+ public InputSelectionHandler(InputSelectable inputSelectable) {
+ this.inputSelector = checkNotNull(inputSelectable);
+ this.availableInputsMask = (int) new
InputSelection.Builder().select(1).select(2).build().getInputMask();
+ }
+
+ void nextSelection() {
+ inputSelection = inputSelector.nextSelection();
+ }
+
+ int selectNextInputIndex(int lastReadInputIndex) {
+ return
inputSelection.fairSelectNextIndexOutOf2(availableInputsMask,
lastReadInputIndex);
+ }
+
+ boolean shouldSetAvailableForAnotherInput() {
+ return availableInputsMask < 3 && inputSelection.isALLMaskOf2();
+ }
+
+ void setAvailableInput(int inputIndex) {
+ availableInputsMask |= 1 << inputIndex;
+ }
+
+ void setUnavailableInput(int inputIndex) {
+ availableInputsMask &= ~(1 << inputIndex);
+ }
+
+ boolean isALLInputsSelected() {
Review comment:
Rename to `areAllInputsSelected()`? (with lower case `All`)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #9477: [FLINK-13765][task] Introduce the InputSelectionHandler for selecting next input in StreamTwoInputSelectableProcessor
pnowojski commented on a change in pull request #9477: [FLINK-13765][task]
Introduce the InputSelectionHandler for selecting next input in
StreamTwoInputSelectableProcessor
URL: https://github.com/apache/flink/pull/9477#discussion_r317173222
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/io/InputSelectionHandler.java
##
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.runtime.io;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.streaming.api.operators.InputSelectable;
+import org.apache.flink.streaming.api.operators.InputSelection;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * This handler is mainly used for selecting the next available input index
+ * in {@link StreamTwoInputSelectableProcessor}.
+ */
+@Internal
+public class InputSelectionHandler {
+
+ private final InputSelectable inputSelector;
+
+ private InputSelection inputSelection;
+
+ private int availableInputsMask;
+
+ public InputSelectionHandler(InputSelectable inputSelectable) {
+ this.inputSelector = checkNotNull(inputSelectable);
+ this.availableInputsMask = (int) new
InputSelection.Builder().select(1).select(2).build().getInputMask();
+ }
+
+ void nextSelection() {
+ inputSelection = inputSelector.nextSelection();
+ }
+
+ int selectNextInputIndex(int lastReadInputIndex) {
+ return
inputSelection.fairSelectNextIndexOutOf2(availableInputsMask,
lastReadInputIndex);
+ }
+
+ boolean shouldSetAvailableForAnotherInput() {
+ return availableInputsMask < 3 && inputSelection.isALLMaskOf2();
+ }
+
+ void setAvailableInput(int inputIndex) {
+ availableInputsMask |= 1 << inputIndex;
+ }
+
+ void setUnavailableInput(int inputIndex) {
+ availableInputsMask &= ~(1 << inputIndex);
+ }
+
+ boolean isALLInputsSelected() {
+ checkState(inputSelection != null);
Review comment:
nitty nit: I think there is little value of this `checkState`, as
`NullPointerException` would be thrown one line below and it doesn't safeguard
any arguments. But you can keep this if you prefer.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #9477: [FLINK-13765][task] Introduce the InputSelectionHandler for selecting next input in StreamTwoInputSelectableProcessor
pnowojski commented on a change in pull request #9477: [FLINK-13765][task]
Introduce the InputSelectionHandler for selecting next input in
StreamTwoInputSelectableProcessor
URL: https://github.com/apache/flink/pull/9477#discussion_r317160696
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/io/InputSelectionHandler.java
##
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.runtime.io;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.streaming.api.operators.InputSelectable;
+import org.apache.flink.streaming.api.operators.InputSelection;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * This handler is mainly used for selecting the next available input index
+ * in {@link StreamTwoInputSelectableProcessor}.
+ */
+@Internal
+public class InputSelectionHandler {
Review comment:
nit: Maybe rename to `TwoInputSelectionHandler` to mark that it only
supports two inputs?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Assigned] (FLINK-13632) Update TypeSerializerUpgradeTestBase to restore from 1.9 savepoint
[
https://issues.apache.org/jira/browse/FLINK-13632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann reassigned FLINK-13632:
-
Assignee: vinoyang
> Update TypeSerializerUpgradeTestBase to restore from 1.9 savepoint
> --
>
> Key: FLINK-13632
> URL: https://issues.apache.org/jira/browse/FLINK-13632
> Project: Flink
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 1.10.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Blocker
> Fix For: 1.10.0
>
>
> Update {{TypeSerializerUpgradeTestBase}} to restore from 1.9 savepoint once
> FLINK-11767 has been merged.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (FLINK-13626) Update StatefulJobWBroadcastStateMigrationITCase.scala to restore from 1.9 savepoint
[
https://issues.apache.org/jira/browse/FLINK-13626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann reassigned FLINK-13626:
-
Assignee: vinoyang
> Update StatefulJobWBroadcastStateMigrationITCase.scala to restore from 1.9
> savepoint
>
>
> Key: FLINK-13626
> URL: https://issues.apache.org/jira/browse/FLINK-13626
> Project: Flink
> Issue Type: Sub-task
> Components: Tests
>Affects Versions: 1.10.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Blocker
> Fix For: 1.10.0
>
>
> Update {{StatefulJobWBroadcastStateMigrationITCase.scala}} to restore from
> 1.9 savepoint.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (FLINK-13631) Update FlinkKinesisConsumerMigrationTest to restore from 1.9 savepoint
[
https://issues.apache.org/jira/browse/FLINK-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann reassigned FLINK-13631:
-
Assignee: vinoyang
> Update FlinkKinesisConsumerMigrationTest to restore from 1.9 savepoint
> --
>
> Key: FLINK-13631
> URL: https://issues.apache.org/jira/browse/FLINK-13631
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Kinesis, Tests
>Affects Versions: 1.10.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Blocker
> Fix For: 1.10.0
>
>
> Update {{FlinkKinesisConsumerMigrationTest}} to restore from 1.9 savepoint.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
