[jira] [Commented] (FLINK-13438) Fix Hive connector with DataTypes.DATE/TIME/TIMESTAMP support
[ https://issues.apache.org/jira/browse/FLINK-13438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894616#comment-16894616 ] Rui Li commented on FLINK-13438: [~jark] yes, Hive returns `java.sql.Timestamp` for timestamp columns (at least for the Hive versions we currently support). We can do some conversion between `Timestamp` and `LocalDateTime `. But I suppose there might be issues due to different semantics of these two. E.g. DST has an impact on the string presentation of `Timestamp`, but not `LocalDateTime`. I'll do some investigation to verify. > Fix Hive connector with DataTypes.DATE/TIME/TIMESTAMP support > - > > Key: FLINK-13438 > URL: https://issues.apache.org/jira/browse/FLINK-13438 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Hive >Reporter: Caizhi Weng >Priority: Blocker > Fix For: 1.9.0, 1.10.0 > > > Similar to JDBC connectors, Hive connectors communicate with Flink framework > using TableSchema, which contains DataType. As the time data read from and > write to Hive connectors must be java.sql.* types and the default conversion > class of our time data types are java.time.*, we have to fix Hive connector > with DataTypes.DATE/TIME/TIMESTAMP support. > But currently when reading tables from Hive, the table schema is created > using Hive's schema, so the time types in the created schema will be sql time > type not local time type. If user specifies a local time type in the table > schema when creating a table in Hive, he will get a different schema when > reading it out. This is undesired. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-13344) Translate "How to Contribute" page into Chinese.
[ https://issues.apache.org/jira/browse/FLINK-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894589#comment-16894589 ] WangHengWei commented on FLINK-13344: - [~jark], I started PR, please take a look. > Translate "How to Contribute" page into Chinese. > - > > Key: FLINK-13344 > URL: https://issues.apache.org/jira/browse/FLINK-13344 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Project Website >Reporter: Jark Wu >Assignee: WangHengWei >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The page is https://flink.apache.org/zh/contributing/how-to-contribute.html > The markdown file is located in > https://github.com/apache/flink-web/blob/asf-site/contributing/how-to-contribute.zh.md > Before start working on this, please read translation guideline: > https://cwiki.apache.org/confluence/display/FLINK/Flink+Translation+Specifications -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-5833) Support for Hive GenericUDF
[ https://issues.apache.org/jira/browse/FLINK-5833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894536#comment-16894536 ] MJ Deng commented on FLINK-5833: Hey [~clarkyzl], are you still working on this task? If not, I'm happy to take over. cc [~fhueske] * [|https://issues.apache.org/jira/secure/AddComment!default.jspa?id=13043148] > Support for Hive GenericUDF > --- > > Key: FLINK-5833 > URL: https://issues.apache.org/jira/browse/FLINK-5833 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Hive, Table SQL / Ecosystem >Reporter: Zhuoluo Yang >Assignee: Zhuoluo Yang >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The second step of FLINK-5802 is to support Hive's GenericUDF. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (FLINK-5802) Flink SQL calling Hive User-Defined Functions
[ https://issues.apache.org/jira/browse/FLINK-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894533#comment-16894533 ] MJ Deng edited comment on FLINK-5802 at 7/27/19 7:02 PM: - Hey [~winipanda], are you still working on this task? If not, I'm happy to take over. cc [~fhueske] was (Author: miaoever): Hey [~winipanda], are you still working on this task? If not, I'm happy to work on it. cc [~fhueske] > Flink SQL calling Hive User-Defined Functions > - > > Key: FLINK-5802 > URL: https://issues.apache.org/jira/browse/FLINK-5802 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Hive, Table SQL / Ecosystem >Reporter: Zhuoluo Yang >Assignee: TANG Wen-hui >Priority: Major > Labels: features > > It's important to call hive udf in Flink SQL. A great many udfs were written > in hive since last ten years. > It's really important to reuse the hive udfs. This feature will reduce the > cost of migration and bring more users to flink. > Spark SQL has already supported this function. > https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_spark-guide/content/calling-udfs.html > The Hive UDFs here include both built-in UDFs and customized UDFs. As many > business logic had been written in UDFs, the customized UDFs are more > important than the built-in UDFs. > Generally, there are three kinds of UDFs in Hive: UDF, UDTF and UDAF. > Here is the document of the Spark SQL: > http://spark.apache.org/docs/latest/sql-programming-guide.html#compatibility-with-apache-hive > > Spark code: > https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala > https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-5802) Flink SQL calling Hive User-Defined Functions
[ https://issues.apache.org/jira/browse/FLINK-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894533#comment-16894533 ] Leo Deng commented on FLINK-5802: - Hey [~winipanda], are you still working on this task? If not, I'm happy to work on it. cc [~fhueske] > Flink SQL calling Hive User-Defined Functions > - > > Key: FLINK-5802 > URL: https://issues.apache.org/jira/browse/FLINK-5802 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Hive, Table SQL / Ecosystem >Reporter: Zhuoluo Yang >Assignee: TANG Wen-hui >Priority: Major > Labels: features > > It's important to call hive udf in Flink SQL. A great many udfs were written > in hive since last ten years. > It's really important to reuse the hive udfs. This feature will reduce the > cost of migration and bring more users to flink. > Spark SQL has already supported this function. > https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_spark-guide/content/calling-udfs.html > The Hive UDFs here include both built-in UDFs and customized UDFs. As many > business logic had been written in UDFs, the customized UDFs are more > important than the built-in UDFs. > Generally, there are three kinds of UDFs in Hive: UDF, UDTF and UDAF. > Here is the document of the Spark SQL: > http://spark.apache.org/docs/latest/sql-programming-guide.html#compatibility-with-apache-hive > > Spark code: > https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala > https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-5832) Support for simple hive UDF
[ https://issues.apache.org/jira/browse/FLINK-5832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894531#comment-16894531 ] Leo Deng commented on FLINK-5832: - Hey [~winipanda], are you still working on this task? If not, I'm happy to work on it. cc [~twalthr] > Support for simple hive UDF > --- > > Key: FLINK-5832 > URL: https://issues.apache.org/jira/browse/FLINK-5832 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Hive, Table SQL / Ecosystem >Reporter: Zhuoluo Yang >Assignee: TANG Wen-hui >Priority: Major > Labels: pull-request-available > > The first step of FLINK-5802 is to support simple Hive UDF. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (FLINK-13325) Add test case for FLINK-13249
[ https://issues.apache.org/jira/browse/FLINK-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-13325. -- Resolution: Fixed Fixed in - master in 21621fbcde534969b748f21e9f8983e3f4e0fb1d - 1.9.0 in bb61cbefb41d31d34c4c6d7a260418bc56994db8 > Add test case for FLINK-13249 > - > > Key: FLINK-13325 > URL: https://issues.apache.org/jira/browse/FLINK-13325 > Project: Flink > Issue Type: Sub-task > Components: Tests >Affects Versions: 1.9.0 >Reporter: Stefan Richter >Assignee: Stefan Richter >Priority: Blocker > Labels: pull-request-available > Fix For: 1.9.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Closed] (FLINK-13325) Add test case for FLINK-13249
[ https://issues.apache.org/jira/browse/FLINK-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-13325. > Add test case for FLINK-13249 > - > > Key: FLINK-13325 > URL: https://issues.apache.org/jira/browse/FLINK-13325 > Project: Flink > Issue Type: Sub-task > Components: Tests >Affects Versions: 1.9.0 >Reporter: Stefan Richter >Assignee: Stefan Richter >Priority: Blocker > Labels: pull-request-available > Fix For: 1.9.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] TisonKun commented on issue #9245: [FLINK-13334][coordination] Remove legacy implementation of slot
TisonKun commented on issue #9245: [FLINK-13334][coordination] Remove legacy implementation of slot URL: https://github.com/apache/flink/pull/9245#issuecomment-515698692 The first travis [failure](https://api.travis-ci.com/v3/job/220006881/log.txt) is on > 19:02:56.248 [ERROR] Failures: 19:02:56.248 [ERROR] StandaloneResourceManagerTest.testStartupPeriod:67->assertHappensUntil:117 condition was not fulfilled before the deadline which is definitely a filed issue and IIRC @azagrebin is processing it. Thus I try to retest this but this time it fails on [1. ZooKeeperHighAvailabilityITCase.testRestoreBehaviourWithFaultyStateHandles](https://api.travis-ci.com/v3/job/220118314/log.txt) [2. KvStateServerHandlerTest.testQueryExecutorShutDown](https://api.travis-ci.com/v3/job/220118312/log.txt) >00:33:49.839 [ERROR] Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 10.98 s <<< FAILURE! - in org.apache.flink.queryablestate.network.KvStateServerHandlerTest 00:33:49.839 [ERROR] testQueryExecutorShutDown(org.apache.flink.queryablestate.network.KvStateServerHandlerTest) Time elapsed: 10.052 s <<< ERROR! java.util.concurrent.TimeoutException at org.apache.flink.queryablestate.network.KvStateServerHandlerTest.readInboundBlocking(KvStateServerHandlerTest.java:720) at org.apache.flink.queryablestate.network.KvStateServerHandlerTest.testQueryExecutorShutDown(KvStateServerHandlerTest.java:451) which in my opinion should have nothing to do with this pr. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9249: [FLINK-13447][table-api] Change default planner to legacy planner instead of any one
flinkbot edited a comment on issue #9249: [FLINK-13447][table-api] Change default planner to legacy planner instead of any one URL: https://github.com/apache/flink/pull/9249#issuecomment-515691656 ## CI report: * be35c02b78c6116a023cfba8a09ba0c2e5c4fda6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120946181) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9249: [FLINK-13447][table-api] Change default planner to legacy planner instead of any one
flinkbot commented on issue #9249: [FLINK-13447][table-api] Change default planner to legacy planner instead of any one URL: https://github.com/apache/flink/pull/9249#issuecomment-515691656 ## CI report: * be35c02b78c6116a023cfba8a09ba0c2e5c4fda6 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120946181) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9249: [FLINK-13447][table-api] Change default planner to legacy planner instead of any one
flinkbot commented on issue #9249: [FLINK-13447][table-api] Change default planner to legacy planner instead of any one URL: https://github.com/apache/flink/pull/9249#issuecomment-515691363 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13447) Change default planner to legacy planner instead of any one
[
https://issues.apache.org/jira/browse/FLINK-13447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-13447:
---
Labels: pull-request-available (was: )
> Change default planner to legacy planner instead of any one
> ---
>
> Key: FLINK-13447
> URL: https://issues.apache.org/jira/browse/FLINK-13447
> Project: Flink
> Issue Type: Task
> Components: Table SQL / API
>Reporter: Jark Wu
>Assignee: Jark Wu
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.9.0, 1.10.0
>
>
> As discussed in FLINK-13399, we will change the default behavior of the
> {{EnvironmentSettings}} to use old planner instead of any planner. This will
> enable us to have both planner in the classpath. This will also enable
> users/connectors to have both planner in dependency and without using
> {{EnvironmentSettings}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] wuchong opened a new pull request #9249: [FLINK-13447][table-api] Change default planner to legacy planner instead of any one
wuchong opened a new pull request #9249: [FLINK-13447][table-api] Change default planner to legacy planner instead of any one URL: https://github.com/apache/flink/pull/9249 ## What is the purpose of the change We want to change the default behavior of the `EnvironmentSettings` to use old planner instead of any planner. This will enable us to have both planner in the classpath by default. This will also enable users/connectors to have both planner in dependency and without using EnvironmentSettings explicitly. ## Brief change log Update `EnvironmentSettings` to use old planner instead of any planner by default. ## Verifying this change This change is a trivial rework without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13447) Change default planner to legacy planner instead of any one
[
https://issues.apache.org/jira/browse/FLINK-13447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-13447:
Priority: Blocker (was: Major)
> Change default planner to legacy planner instead of any one
> ---
>
> Key: FLINK-13447
> URL: https://issues.apache.org/jira/browse/FLINK-13447
> Project: Flink
> Issue Type: Task
> Components: Table SQL / API
>Reporter: Jark Wu
>Assignee: Jark Wu
>Priority: Blocker
> Fix For: 1.9.0, 1.10.0
>
>
> As discussed in FLINK-13399, we will change the default behavior of the
> {{EnvironmentSettings}} to use old planner instead of any planner. This will
> enable us to have both planner in the classpath. This will also enable
> users/connectors to have both planner in dependency and without using
> {{EnvironmentSettings}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (FLINK-13447) Change default planner to legacy planner instead of any one
Jark Wu created FLINK-13447:
---
Summary: Change default planner to legacy planner instead of any
one
Key: FLINK-13447
URL: https://issues.apache.org/jira/browse/FLINK-13447
Project: Flink
Issue Type: Task
Components: Table SQL / API
Reporter: Jark Wu
Assignee: Jark Wu
Fix For: 1.9.0, 1.10.0
As discussed in FLINK-13399, we will change the default behavior of the
{{EnvironmentSettings}} to use old planner instead of any planner. This will
enable us to have both planner in the classpath. This will also enable
users/connectors to have both planner in dependency and without using
{{EnvironmentSettings}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] lamber-ken closed pull request #7396: [FLINK-11250][streaming] fix thread lack when StreamTask switched from DEPLOYING to CANCELING
lamber-ken closed pull request #7396: [FLINK-11250][streaming] fix thread lack when StreamTask switched from DEPLOYING to CANCELING URL: https://github.com/apache/flink/pull/7396 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org
wuchong commented on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org URL: https://github.com/apache/flink/pull/9230#issuecomment-515689276 I checked the bui...@travis-ci.org has already been in the subscriber list and allow list. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org
wuchong commented on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org URL: https://github.com/apache/flink/pull/9230#issuecomment-515689189 Yes @zentol . I have read it and tried to reply **-accept** and **-allow** address. But it still doesn't work. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zentol commented on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org
zentol commented on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org URL: https://github.com/apache/flink/pull/9230#issuecomment-515688909 @wuchong see https://apache.org/dev/committers.html#mail-moderate This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zentol commented on a change in pull request #9244: [FLINK-13396] Deprecate the BucketingSink.
zentol commented on a change in pull request #9244: [FLINK-13396] Deprecate the BucketingSink. URL: https://github.com/apache/flink/pull/9244#discussion_r307967133 ## File path: docs/dev/connectors/filesystem_sink.md ## @@ -23,6 +23,10 @@ specific language governing permissions and limitations under the License. --> + +The BucketingSink has been deprecated since Flink 1.9 and will be removed in subsequent releases. Please use the StreamingFileSink instead. Review comment: let'S add a link to the documentation of the StreamingFileSink This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9248: [FLINK-13446][table-runtime-blink] Fix assign logic for row count sliding window
flinkbot edited a comment on issue #9248: [FLINK-13446][table-runtime-blink] Fix assign logic for row count sliding window URL: https://github.com/apache/flink/pull/9248#issuecomment-515679460 ## CI report: * 70ee1121f9855cce916e7a4de8d198a3cc6b1c47 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120941632) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org
wuchong commented on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org URL: https://github.com/apache/flink/pull/9230#issuecomment-515681135 > Well that sounds perfect then, +1. However, we still need to figure out how to accept builds emails. Currently, all notification from bui...@travis-ci.org are rejected. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zentol commented on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org
zentol commented on issue #9230: [FLINK-13430][build] Configure sending travis build notifications to bui...@flink.apache.org URL: https://github.com/apache/flink/pull/9230#issuecomment-515680060 Well that sounds perfect then, +1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zentol commented on issue #9238: [FLINK-13387][WebUI] Fix log download for old UI
zentol commented on issue #9238: [FLINK-13387][WebUI] Fix log download for old UI URL: https://github.com/apache/flink/pull/9238#issuecomment-515680020 I don't really want to change the behavior of the UI at this point. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9248: [FLINK-13446][table-runtime-blink] Fix assign logic for row count sliding window
flinkbot commented on issue #9248: [FLINK-13446][table-runtime-blink] Fix assign logic for row count sliding window URL: https://github.com/apache/flink/pull/9248#issuecomment-515679460 ## CI report: * 70ee1121f9855cce916e7a4de8d198a3cc6b1c47 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120941632) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9248: [FLINK-13446][table-runtime-blink] Fix assign logic for row count sliding window
flinkbot commented on issue #9248: [FLINK-13446][table-runtime-blink] Fix assign logic for row count sliding window URL: https://github.com/apache/flink/pull/9248#issuecomment-515679172 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] hequn8128 opened a new pull request #9248: [FLINK-13446][table-runtime-blink] Fix assign logic for row count sliding window
hequn8128 opened a new pull request #9248: [FLINK-13446][table-runtime-blink] Fix assign logic for row count sliding window URL: https://github.com/apache/flink/pull/9248 ## What is the purpose of the change For blink planner, the Row count sliding window outputs incorrectly. The window assigner assigns less window than what expected. This means the window outputs fewer data. This pull request fixes the window assigner and trigger logic, i.e., correct the window start and end calculation. ## Brief change log - Add trigger size property to `CountWindow`, because in some cases, the count window fires with a trigger size less than the window size. - Correct the window start and end calculation in `CountSlidingWindowAssigner`. - Correct existing test cases. ## Verifying this change This change is already covered by existing tests, such as GroupWindowITCase and WindowOperatorTest. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13446) Row count sliding window outputs incorrectly in blink planner
[
https://issues.apache.org/jira/browse/FLINK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-13446:
---
Labels: pull-request-available (was: )
> Row count sliding window outputs incorrectly in blink planner
> -
>
> Key: FLINK-13446
> URL: https://issues.apache.org/jira/browse/FLINK-13446
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
>Affects Versions: 1.9.0
>Reporter: Hequn Cheng
>Assignee: Hequn Cheng
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.9.0, 1.10.0
>
>
> For blink planner, the Row count sliding window outputs incorrectly. The
> window assigner assigns less window than what expected. This means the window
> outputs fewer data. The bug can be reproduced by the following test:
> {code:java}
> @Test
> def testGroupWindowWithoutKeyInProjection(): Unit = {
> val data = List(
> (1L, 1, "Hi", 1, 1),
> (2L, 2, "Hello", 2, 2),
> (4L, 2, "Hello", 2, 2),
> (8L, 3, "Hello world", 3, 3),
> (16L, 3, "Hello world", 3, 3))
> val stream = failingDataSource(data)
> val table = stream.toTable(tEnv, 'long, 'int, 'string, 'int2, 'int3,
> 'proctime.proctime)
> val weightAvgFun = new WeightedAvg
> val countDistinct = new CountDistinct
> val windowedTable = table
> .window(Slide over 2.rows every 1.rows on 'proctime as 'w)
> .groupBy('w, 'int2, 'int3, 'string)
> .select(weightAvgFun('long, 'int), countDistinct('long))
> val sink = new TestingAppendSink
> windowedTable.toAppendStream[Row].addSink(sink)
> env.execute()
> val expected = Seq("12,2", "8,1", "2,1", "3,2", "1,1")
> assertEquals(expected.sorted, sink.getAppendResults.sorted)
> }
> {code}
> The expected output is Seq("12,2", "8,1", "2,1", "3,2", "1,1") while the
> actual output is Seq("12,2", "3,2")
> To fix the problem, we can correct the assign logic in
> CountSlidingWindowAssigner.assignWindows.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515660312 ## CI report: * dbe883c57e689ed544de09423192843c758bfa54 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120935623) * 706ebbe34b99bc2c9e14dfc92ad3c683b566f147 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120936069) * 12fa022f25d91065fd2eeb91e29118611b8ac5c6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120936600) * 314bdcc4b411ad226124d42704412d2c176c3648 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120937528) * b56ba2a95d8f0fdf705977058adb9ef9f08d17c0 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120937670) * cbe85dd799b686faf1d6a1b67f51cac2cd1c94fe : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120938960) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515660312 ## CI report: * dbe883c57e689ed544de09423192843c758bfa54 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120935623) * 706ebbe34b99bc2c9e14dfc92ad3c683b566f147 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120936069) * 12fa022f25d91065fd2eeb91e29118611b8ac5c6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120936600) * 314bdcc4b411ad226124d42704412d2c176c3648 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120937528) * b56ba2a95d8f0fdf705977058adb9ef9f08d17c0 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120937670) * cbe85dd799b686faf1d6a1b67f51cac2cd1c94fe : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120938960) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515660312 ## CI report: * dbe883c57e689ed544de09423192843c758bfa54 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120935623) * 706ebbe34b99bc2c9e14dfc92ad3c683b566f147 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120936069) * 12fa022f25d91065fd2eeb91e29118611b8ac5c6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120936600) * 314bdcc4b411ad226124d42704412d2c176c3648 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120937528) * b56ba2a95d8f0fdf705977058adb9ef9f08d17c0 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120937670) * cbe85dd799b686faf1d6a1b67f51cac2cd1c94fe : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120938960) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515660312 ## CI report: * dbe883c57e689ed544de09423192843c758bfa54 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120935623) * 706ebbe34b99bc2c9e14dfc92ad3c683b566f147 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120936069) * 12fa022f25d91065fd2eeb91e29118611b8ac5c6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120936600) * 314bdcc4b411ad226124d42704412d2c176c3648 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120937528) * b56ba2a95d8f0fdf705977058adb9ef9f08d17c0 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120937670) * cbe85dd799b686faf1d6a1b67f51cac2cd1c94fe : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120938960) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13386) Frictions in the new default Web Frontend
[ https://issues.apache.org/jira/browse/FLINK-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894386#comment-16894386 ] Yadong Xie commented on FLINK-13386: Hi [~dawidwys] and [~till.rohrmann] I have fixed the bugs below * the jobmanager configuration is not sorted * different sorting of the operators (the old UI showed the sources first) * the drop-down list for choosing operator/tasks metrics is not sorted, which makes it super hard to screen through available metrics and add these two features * can't see watermarks for all operators at once * no numeric metrics (only graphs) in [https://github.com/apache/flink/pull/9247] > Frictions in the new default Web Frontend > - > > Key: FLINK-13386 > URL: https://issues.apache.org/jira/browse/FLINK-13386 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.9.0 >Reporter: Dawid Wysakowicz >Priority: Critical > Labels: pull-request-available > Fix For: 1.9.0 > > Attachments: bug.png, repro.png > > Time Spent: 10m > Remaining Estimate: 0h > > While manually testing the new WebUI I found a few frictions. > * when using the UI the left panel hides unexpectedly at random moments > * mouse wheel does not work on the logs (taskmanager, jobmanager) pane > * the jobmanager configuration is not sorted > * different sorting of the operators (the old UI showed the sources first) > * the drop-down list for choosing operator/tasks metrics is not sorted, which > makes it super hard to screen through available metrics > * arrow does not touch the rectangles in Chrome (see attached screenshot) > There are also some views missing in the new UI that I personally found > useful in the old UI: > * can't see watermarks for all operators at once > * no numeric metrics (only graphs) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515660312 ## CI report: * dbe883c57e689ed544de09423192843c758bfa54 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120935623) * 706ebbe34b99bc2c9e14dfc92ad3c683b566f147 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120936069) * 12fa022f25d91065fd2eeb91e29118611b8ac5c6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120936600) * 314bdcc4b411ad226124d42704412d2c176c3648 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120937528) * b56ba2a95d8f0fdf705977058adb9ef9f08d17c0 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120937670) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13446) Row count sliding window outputs incorrectly in blink planner
[
https://issues.apache.org/jira/browse/FLINK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894373#comment-16894373
]
Hequn Cheng commented on FLINK-13446:
-
Thank you. I will open the PR soon. Would be great if you can take a look.
> Row count sliding window outputs incorrectly in blink planner
> -
>
> Key: FLINK-13446
> URL: https://issues.apache.org/jira/browse/FLINK-13446
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
>Affects Versions: 1.9.0
>Reporter: Hequn Cheng
>Assignee: Hequn Cheng
>Priority: Major
> Fix For: 1.9.0, 1.10.0
>
>
> For blink planner, the Row count sliding window outputs incorrectly. The
> window assigner assigns less window than what expected. This means the window
> outputs fewer data. The bug can be reproduced by the following test:
> {code:java}
> @Test
> def testGroupWindowWithoutKeyInProjection(): Unit = {
> val data = List(
> (1L, 1, "Hi", 1, 1),
> (2L, 2, "Hello", 2, 2),
> (4L, 2, "Hello", 2, 2),
> (8L, 3, "Hello world", 3, 3),
> (16L, 3, "Hello world", 3, 3))
> val stream = failingDataSource(data)
> val table = stream.toTable(tEnv, 'long, 'int, 'string, 'int2, 'int3,
> 'proctime.proctime)
> val weightAvgFun = new WeightedAvg
> val countDistinct = new CountDistinct
> val windowedTable = table
> .window(Slide over 2.rows every 1.rows on 'proctime as 'w)
> .groupBy('w, 'int2, 'int3, 'string)
> .select(weightAvgFun('long, 'int), countDistinct('long))
> val sink = new TestingAppendSink
> windowedTable.toAppendStream[Row].addSink(sink)
> env.execute()
> val expected = Seq("12,2", "8,1", "2,1", "3,2", "1,1")
> assertEquals(expected.sorted, sink.getAppendResults.sorted)
> }
> {code}
> The expected output is Seq("12,2", "8,1", "2,1", "3,2", "1,1") while the
> actual output is Seq("12,2", "3,2")
> To fix the problem, we can correct the assign logic in
> CountSlidingWindowAssigner.assignWindows.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13446) Row count sliding window outputs incorrectly in blink planner
[
https://issues.apache.org/jira/browse/FLINK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894372#comment-16894372
]
Jark Wu commented on FLINK-13446:
-
Thanks [~hequn8128], I assigned this issue to you.
> Row count sliding window outputs incorrectly in blink planner
> -
>
> Key: FLINK-13446
> URL: https://issues.apache.org/jira/browse/FLINK-13446
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
>Affects Versions: 1.9.0
>Reporter: Hequn Cheng
>Assignee: Hequn Cheng
>Priority: Major
> Fix For: 1.9.0, 1.10.0
>
>
> For blink planner, the Row count sliding window outputs incorrectly. The
> window assigner assigns less window than what expected. This means the window
> outputs fewer data. The bug can be reproduced by the following test:
> {code:java}
> @Test
> def testGroupWindowWithoutKeyInProjection(): Unit = {
> val data = List(
> (1L, 1, "Hi", 1, 1),
> (2L, 2, "Hello", 2, 2),
> (4L, 2, "Hello", 2, 2),
> (8L, 3, "Hello world", 3, 3),
> (16L, 3, "Hello world", 3, 3))
> val stream = failingDataSource(data)
> val table = stream.toTable(tEnv, 'long, 'int, 'string, 'int2, 'int3,
> 'proctime.proctime)
> val weightAvgFun = new WeightedAvg
> val countDistinct = new CountDistinct
> val windowedTable = table
> .window(Slide over 2.rows every 1.rows on 'proctime as 'w)
> .groupBy('w, 'int2, 'int3, 'string)
> .select(weightAvgFun('long, 'int), countDistinct('long))
> val sink = new TestingAppendSink
> windowedTable.toAppendStream[Row].addSink(sink)
> env.execute()
> val expected = Seq("12,2", "8,1", "2,1", "3,2", "1,1")
> assertEquals(expected.sorted, sink.getAppendResults.sorted)
> }
> {code}
> The expected output is Seq("12,2", "8,1", "2,1", "3,2", "1,1") while the
> actual output is Seq("12,2", "3,2")
> To fix the problem, we can correct the assign logic in
> CountSlidingWindowAssigner.assignWindows.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (FLINK-13446) Row count sliding window outputs incorrectly in blink planner
[
https://issues.apache.org/jira/browse/FLINK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-13446:
Fix Version/s: 1.10.0
1.9.0
> Row count sliding window outputs incorrectly in blink planner
> -
>
> Key: FLINK-13446
> URL: https://issues.apache.org/jira/browse/FLINK-13446
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
>Affects Versions: 1.9.0
>Reporter: Hequn Cheng
>Assignee: Hequn Cheng
>Priority: Major
> Fix For: 1.9.0, 1.10.0
>
>
> For blink planner, the Row count sliding window outputs incorrectly. The
> window assigner assigns less window than what expected. This means the window
> outputs fewer data. The bug can be reproduced by the following test:
> {code:java}
> @Test
> def testGroupWindowWithoutKeyInProjection(): Unit = {
> val data = List(
> (1L, 1, "Hi", 1, 1),
> (2L, 2, "Hello", 2, 2),
> (4L, 2, "Hello", 2, 2),
> (8L, 3, "Hello world", 3, 3),
> (16L, 3, "Hello world", 3, 3))
> val stream = failingDataSource(data)
> val table = stream.toTable(tEnv, 'long, 'int, 'string, 'int2, 'int3,
> 'proctime.proctime)
> val weightAvgFun = new WeightedAvg
> val countDistinct = new CountDistinct
> val windowedTable = table
> .window(Slide over 2.rows every 1.rows on 'proctime as 'w)
> .groupBy('w, 'int2, 'int3, 'string)
> .select(weightAvgFun('long, 'int), countDistinct('long))
> val sink = new TestingAppendSink
> windowedTable.toAppendStream[Row].addSink(sink)
> env.execute()
> val expected = Seq("12,2", "8,1", "2,1", "3,2", "1,1")
> assertEquals(expected.sorted, sink.getAppendResults.sorted)
> }
> {code}
> The expected output is Seq("12,2", "8,1", "2,1", "3,2", "1,1") while the
> actual output is Seq("12,2", "3,2")
> To fix the problem, we can correct the assign logic in
> CountSlidingWindowAssigner.assignWindows.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (FLINK-13446) Row count sliding window outputs incorrectly in blink planner
[
https://issues.apache.org/jira/browse/FLINK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu reassigned FLINK-13446:
---
Assignee: Hequn Cheng
> Row count sliding window outputs incorrectly in blink planner
> -
>
> Key: FLINK-13446
> URL: https://issues.apache.org/jira/browse/FLINK-13446
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
>Affects Versions: 1.9.0
>Reporter: Hequn Cheng
>Assignee: Hequn Cheng
>Priority: Major
>
> For blink planner, the Row count sliding window outputs incorrectly. The
> window assigner assigns less window than what expected. This means the window
> outputs fewer data. The bug can be reproduced by the following test:
> {code:java}
> @Test
> def testGroupWindowWithoutKeyInProjection(): Unit = {
> val data = List(
> (1L, 1, "Hi", 1, 1),
> (2L, 2, "Hello", 2, 2),
> (4L, 2, "Hello", 2, 2),
> (8L, 3, "Hello world", 3, 3),
> (16L, 3, "Hello world", 3, 3))
> val stream = failingDataSource(data)
> val table = stream.toTable(tEnv, 'long, 'int, 'string, 'int2, 'int3,
> 'proctime.proctime)
> val weightAvgFun = new WeightedAvg
> val countDistinct = new CountDistinct
> val windowedTable = table
> .window(Slide over 2.rows every 1.rows on 'proctime as 'w)
> .groupBy('w, 'int2, 'int3, 'string)
> .select(weightAvgFun('long, 'int), countDistinct('long))
> val sink = new TestingAppendSink
> windowedTable.toAppendStream[Row].addSink(sink)
> env.execute()
> val expected = Seq("12,2", "8,1", "2,1", "3,2", "1,1")
> assertEquals(expected.sorted, sink.getAppendResults.sorted)
> }
> {code}
> The expected output is Seq("12,2", "8,1", "2,1", "3,2", "1,1") while the
> actual output is Seq("12,2", "3,2")
> To fix the problem, we can correct the assign logic in
> CountSlidingWindowAssigner.assignWindows.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (FLINK-13446) Row count sliding window outputs incorrectly in blink planner
Hequn Cheng created FLINK-13446:
---
Summary: Row count sliding window outputs incorrectly in blink
planner
Key: FLINK-13446
URL: https://issues.apache.org/jira/browse/FLINK-13446
Project: Flink
Issue Type: Bug
Components: Table SQL / Runtime
Affects Versions: 1.9.0
Reporter: Hequn Cheng
For blink planner, the Row count sliding window outputs incorrectly. The window
assigner assigns less window than what expected. This means the window outputs
fewer data. The bug can be reproduced by the following test:
{code:java}
@Test
def testGroupWindowWithoutKeyInProjection(): Unit = {
val data = List(
(1L, 1, "Hi", 1, 1),
(2L, 2, "Hello", 2, 2),
(4L, 2, "Hello", 2, 2),
(8L, 3, "Hello world", 3, 3),
(16L, 3, "Hello world", 3, 3))
val stream = failingDataSource(data)
val table = stream.toTable(tEnv, 'long, 'int, 'string, 'int2, 'int3,
'proctime.proctime)
val weightAvgFun = new WeightedAvg
val countDistinct = new CountDistinct
val windowedTable = table
.window(Slide over 2.rows every 1.rows on 'proctime as 'w)
.groupBy('w, 'int2, 'int3, 'string)
.select(weightAvgFun('long, 'int), countDistinct('long))
val sink = new TestingAppendSink
windowedTable.toAppendStream[Row].addSink(sink)
env.execute()
val expected = Seq("12,2", "8,1", "2,1", "3,2", "1,1")
assertEquals(expected.sorted, sink.getAppendResults.sorted)
}
{code}
The expected output is Seq("12,2", "8,1", "2,1", "3,2", "1,1") while the actual
output is Seq("12,2", "3,2")
To fix the problem, we can correct the assign logic in
CountSlidingWindowAssigner.assignWindows.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13444) Translate English content of FLINK-13396 into Chinese
[
https://issues.apache.org/jira/browse/FLINK-13444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894367#comment-16894367
]
Jark Wu commented on FLINK-13444:
-
Sure. I assigned this issue to you. But please start work until FLINK-13396 is
merged.
> Translate English content of FLINK-13396 into Chinese
> -
>
> Key: FLINK-13444
> URL: https://issues.apache.org/jira/browse/FLINK-13444
> Project: Flink
> Issue Type: Sub-task
> Components: chinese-translation, Documentation
>Reporter: Jark Wu
>Assignee: Ke Li
>Priority: Major
>
> Translate new added English content of FLINK-13396 to Chinese.
> The markdown file is located in {{docs/dev/connectors/filesystem_sink.zh.md}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (FLINK-13444) Translate English content of FLINK-13396 into Chinese
[
https://issues.apache.org/jira/browse/FLINK-13444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu reassigned FLINK-13444:
---
Assignee: Ke Li
> Translate English content of FLINK-13396 into Chinese
> -
>
> Key: FLINK-13444
> URL: https://issues.apache.org/jira/browse/FLINK-13444
> Project: Flink
> Issue Type: Sub-task
> Components: chinese-translation, Documentation
>Reporter: Jark Wu
>Assignee: Ke Li
>Priority: Major
>
> Translate new added English content of FLINK-13396 to Chinese.
> The markdown file is located in {{docs/dev/connectors/filesystem_sink.zh.md}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515660312 ## CI report: * dbe883c57e689ed544de09423192843c758bfa54 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120935623) * 706ebbe34b99bc2c9e14dfc92ad3c683b566f147 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120936069) * 12fa022f25d91065fd2eeb91e29118611b8ac5c6 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120936600) * 314bdcc4b411ad226124d42704412d2c176c3648 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120937528) * b56ba2a95d8f0fdf705977058adb9ef9f08d17c0 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120937670) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515660312 ## CI report: * dbe883c57e689ed544de09423192843c758bfa54 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120935623) * 706ebbe34b99bc2c9e14dfc92ad3c683b566f147 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120936069) * 12fa022f25d91065fd2eeb91e29118611b8ac5c6 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120936600) * 314bdcc4b411ad226124d42704412d2c176c3648 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120937528) * b56ba2a95d8f0fdf705977058adb9ef9f08d17c0 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120937670) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515660312 ## CI report: * dbe883c57e689ed544de09423192843c758bfa54 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120935623) * 706ebbe34b99bc2c9e14dfc92ad3c683b566f147 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120936069) * 12fa022f25d91065fd2eeb91e29118611b8ac5c6 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120936600) * 314bdcc4b411ad226124d42704412d2c176c3648 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120937528) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515660312 ## CI report: * dbe883c57e689ed544de09423192843c758bfa54 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120935623) * 706ebbe34b99bc2c9e14dfc92ad3c683b566f147 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120936069) * 12fa022f25d91065fd2eeb91e29118611b8ac5c6 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120936600) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13444) Translate English content of FLINK-13396 into Chinese
[
https://issues.apache.org/jira/browse/FLINK-13444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894347#comment-16894347
]
Ke Li commented on FLINK-13444:
---
Hi [~jark] , Can I work on this issue?
> Translate English content of FLINK-13396 into Chinese
> -
>
> Key: FLINK-13444
> URL: https://issues.apache.org/jira/browse/FLINK-13444
> Project: Flink
> Issue Type: Sub-task
> Components: chinese-translation, Documentation
>Reporter: Jark Wu
>Priority: Major
>
> Translate new added English content of FLINK-13396 to Chinese.
> The markdown file is located in {{docs/dev/connectors/filesystem_sink.zh.md}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance.
flinkbot edited a comment on issue #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance. URL: https://github.com/apache/flink/pull/9105#issuecomment-511499675 ## CI report: * 7b6a81ed94056dd217f7feba9b155158fcfbfc4a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119404317) * 1e289e4de143a390eac564e38c6d5439c650f17a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441215) * dceeac0b80e6695d534e55fbbe3e83aa4262dbc9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119568870) * a2e403e357690465dc8ca1b3b266989d25ac155f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119766340) * 70d2ed3eebc9874eb70559fc5a72b5366392b0df : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120817288) * 14ff63f68d5e683d6ba15a02e340e9d867ed7742 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120934246) * a41ad84cacf54c7d4001448e501924c286233622 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120934632) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515660312 ## CI report: * dbe883c57e689ed544de09423192843c758bfa54 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120935623) * 706ebbe34b99bc2c9e14dfc92ad3c683b566f147 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120936069) * 12fa022f25d91065fd2eeb91e29118611b8ac5c6 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120936600) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13386) Frictions in the new default Web Frontend
[ https://issues.apache.org/jira/browse/FLINK-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Xie updated FLINK-13386: --- Attachment: repro.png > Frictions in the new default Web Frontend > - > > Key: FLINK-13386 > URL: https://issues.apache.org/jira/browse/FLINK-13386 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.9.0 >Reporter: Dawid Wysakowicz >Priority: Critical > Labels: pull-request-available > Fix For: 1.9.0 > > Attachments: bug.png, repro.png > > Time Spent: 10m > Remaining Estimate: 0h > > While manually testing the new WebUI I found a few frictions. > * when using the UI the left panel hides unexpectedly at random moments > * mouse wheel does not work on the logs (taskmanager, jobmanager) pane > * the jobmanager configuration is not sorted > * different sorting of the operators (the old UI showed the sources first) > * the drop-down list for choosing operator/tasks metrics is not sorted, which > makes it super hard to screen through available metrics > * arrow does not touch the rectangles in Chrome (see attached screenshot) > There are also some views missing in the new UI that I personally found > useful in the old UI: > * can't see watermarks for all operators at once > * no numeric metrics (only graphs) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (FLINK-13386) Frictions in the new default Web Frontend
[ https://issues.apache.org/jira/browse/FLINK-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894306#comment-16894306 ] Yadong Xie edited comment on FLINK-13386 at 7/27/19 7:51 AM: - Hi [~dawidwys] * when using the UI the left panel hides unexpectedly at random moments the left panel will collapse when you first enter the job detail page, I can remove this behavior if it confuses the users * mouse wheel does not work on the logs (taskmanager, jobmanager) pane I can not reproduce it, could you provide more info about your browser version and operation system(windows or mac) * the jobmanager configuration is not sorted I will fix this * different sorting of the operators (the old UI showed the sources first) I will fix this * the drop-down list for choosing operator/tasks metrics is not sorted, which makes it super hard to screen through available metrics I will fix this * arrow does not touch the rectangles in Chrome (see attached screenshot) I can not reproduce it, could you provide more info about your browser version and operation system(windows or mac) see the repro.png There are also some views missing in the new UI that I personally found useful in the old UI: * can't see watermarks for all operators at once I will fix this * no numeric metrics (only graphs) I will fix this was (Author: vthinkxie): Hi [~dawidwys] * when using the UI the left panel hides unexpectedly at random moments the left panel will collapse when you first enter the job detail page, I can remove this behavior if it confuses the users * mouse wheel does not work on the logs (taskmanager, jobmanager) pane I can not reproduce it, could you provide more info about your browser version and operation system(windows or mac) * the jobmanager configuration is not sorted I will fix this * different sorting of the operators (the old UI showed the sources first) I will fix this * the drop-down list for choosing operator/tasks metrics is not sorted, which makes it super hard to screen through available metrics I will fix this * arrow does not touch the rectangles in Chrome (see attached screenshot) I can not reproduce it, could you provide more info about your browser version and operation system(windows or mac) There are also some views missing in the new UI that I personally found useful in the old UI: * can't see watermarks for all operators at once I will fix this * no numeric metrics (only graphs) I will fix this > Frictions in the new default Web Frontend > - > > Key: FLINK-13386 > URL: https://issues.apache.org/jira/browse/FLINK-13386 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.9.0 >Reporter: Dawid Wysakowicz >Priority: Critical > Labels: pull-request-available > Fix For: 1.9.0 > > Attachments: bug.png, repro.png > > Time Spent: 10m > Remaining Estimate: 0h > > While manually testing the new WebUI I found a few frictions. > * when using the UI the left panel hides unexpectedly at random moments > * mouse wheel does not work on the logs (taskmanager, jobmanager) pane > * the jobmanager configuration is not sorted > * different sorting of the operators (the old UI showed the sources first) > * the drop-down list for choosing operator/tasks metrics is not sorted, which > makes it super hard to screen through available metrics > * arrow does not touch the rectangles in Chrome (see attached screenshot) > There are also some views missing in the new UI that I personally found > useful in the old UI: > * can't see watermarks for all operators at once > * no numeric metrics (only graphs) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (FLINK-13386) Frictions in the new default Web Frontend
[ https://issues.apache.org/jira/browse/FLINK-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Xie updated FLINK-13386: --- Attachment: 下载 (2).png > Frictions in the new default Web Frontend > - > > Key: FLINK-13386 > URL: https://issues.apache.org/jira/browse/FLINK-13386 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.9.0 >Reporter: Dawid Wysakowicz >Priority: Critical > Labels: pull-request-available > Fix For: 1.9.0 > > Attachments: bug.png, repro.png > > Time Spent: 10m > Remaining Estimate: 0h > > While manually testing the new WebUI I found a few frictions. > * when using the UI the left panel hides unexpectedly at random moments > * mouse wheel does not work on the logs (taskmanager, jobmanager) pane > * the jobmanager configuration is not sorted > * different sorting of the operators (the old UI showed the sources first) > * the drop-down list for choosing operator/tasks metrics is not sorted, which > makes it super hard to screen through available metrics > * arrow does not touch the rectangles in Chrome (see attached screenshot) > There are also some views missing in the new UI that I personally found > useful in the old UI: > * can't see watermarks for all operators at once > * no numeric metrics (only graphs) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (FLINK-13386) Frictions in the new default Web Frontend
[ https://issues.apache.org/jira/browse/FLINK-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Xie updated FLINK-13386: --- Attachment: (was: 下载 (2).png) > Frictions in the new default Web Frontend > - > > Key: FLINK-13386 > URL: https://issues.apache.org/jira/browse/FLINK-13386 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.9.0 >Reporter: Dawid Wysakowicz >Priority: Critical > Labels: pull-request-available > Fix For: 1.9.0 > > Attachments: bug.png, repro.png > > Time Spent: 10m > Remaining Estimate: 0h > > While manually testing the new WebUI I found a few frictions. > * when using the UI the left panel hides unexpectedly at random moments > * mouse wheel does not work on the logs (taskmanager, jobmanager) pane > * the jobmanager configuration is not sorted > * different sorting of the operators (the old UI showed the sources first) > * the drop-down list for choosing operator/tasks metrics is not sorted, which > makes it super hard to screen through available metrics > * arrow does not touch the rectangles in Chrome (see attached screenshot) > There are also some views missing in the new UI that I personally found > useful in the old UI: > * can't see watermarks for all operators at once > * no numeric metrics (only graphs) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9246: [FLINK-13241][Yarn/Mesos] Duplication of PR#9105 on master branch.
flinkbot edited a comment on issue #9246: [FLINK-13241][Yarn/Mesos] Duplication of PR#9105 on master branch. URL: https://github.com/apache/flink/pull/9246#issuecomment-515656507 ## CI report: * 89ca6ef411dda57c14ef27cb90e8cfbb4ca4fb99 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120934490) * bc47897725c5ae01d840c805023117037cfd9e5b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120934627) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9203: [FLINK-13375][table-api] Improve config names in ExecutionConfigOptions and OptimizerConfigOptions
flinkbot edited a comment on issue #9203: [FLINK-13375][table-api] Improve config names in ExecutionConfigOptions and OptimizerConfigOptions URL: https://github.com/apache/flink/pull/9203#issuecomment-514046368 ## CI report: * f5e680b52e6a85e85642fc22a41724c5a452505c : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120109078) * c1388ab2867ad134b2300ccad6ca519eff547ccb : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120216941) * b70392e5d7337da2a416316068397f92925a0d7e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120455507) * 46e98c30bbe0709b2261ddd5ddb2963bf91ccfcc : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120935409) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot edited a comment on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515660312 ## CI report: * dbe883c57e689ed544de09423192843c758bfa54 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120935623) * 706ebbe34b99bc2c9e14dfc92ad3c683b566f147 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120936069) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13445) Distinguishing Memory Configuration for TaskManager and JobManager
[ https://issues.apache.org/jira/browse/FLINK-13445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] madong updated FLINK-13445: --- Priority: Major (was: Minor) > Distinguishing Memory Configuration for TaskManager and JobManager > -- > > Key: FLINK-13445 > URL: https://issues.apache.org/jira/browse/FLINK-13445 > Project: Flink > Issue Type: New Feature > Components: Runtime / Configuration >Affects Versions: 1.8.1 >Reporter: madong >Priority: Major > > we use flink to run some job build in non-java language, so we increase the > value of `containerized.heap-cutoff-ratio` to reserve more memory for > non-java process , which would affect memory allocation for jobManager. > Considering the different behaviors of taskManager and jobManager, should we > use this configuration separately？ -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (FLINK-13386) Frictions in the new default Web Frontend
[ https://issues.apache.org/jira/browse/FLINK-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894306#comment-16894306 ] Yadong Xie edited comment on FLINK-13386 at 7/27/19 7:37 AM: - Hi [~dawidwys] * when using the UI the left panel hides unexpectedly at random moments the left panel will collapse when you first enter the job detail page, I can remove this behavior if it confuses the users * mouse wheel does not work on the logs (taskmanager, jobmanager) pane I can not reproduce it, could you provide more info about your browser version and operation system(windows or mac) * the jobmanager configuration is not sorted I will fix this * different sorting of the operators (the old UI showed the sources first) I will fix this * the drop-down list for choosing operator/tasks metrics is not sorted, which makes it super hard to screen through available metrics I will fix this * arrow does not touch the rectangles in Chrome (see attached screenshot) I can not reproduce it, could you provide more info about your browser version and operation system(windows or mac) There are also some views missing in the new UI that I personally found useful in the old UI: * can't see watermarks for all operators at once I will fix this * no numeric metrics (only graphs) I will fix this was (Author: vthinkxie): Hi [~dawidwys] * when using the UI the left panel hides unexpectedly at random moments the left panel will collapse when you first enter the job detail page * mouse wheel does not work on the logs (taskmanager, jobmanager) pane I can not reproduce it, could you provide more info about your browser version and operation system(windows or mac) * the jobmanager configuration is not sorted I will fix this * different sorting of the operators (the old UI showed the sources first) I will fix this * the drop-down list for choosing operator/tasks metrics is not sorted, which makes it super hard to screen through available metrics I will fix this * arrow does not touch the rectangles in Chrome (see attached screenshot) I can not reproduce it, could you provide more info about your browser version and operation system(windows or mac) There are also some views missing in the new UI that I personally found useful in the old UI: * can't see watermarks for all operators at once I will fix this * no numeric metrics (only graphs) I will fix this > Frictions in the new default Web Frontend > - > > Key: FLINK-13386 > URL: https://issues.apache.org/jira/browse/FLINK-13386 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.9.0 >Reporter: Dawid Wysakowicz >Priority: Critical > Labels: pull-request-available > Fix For: 1.9.0 > > Attachments: bug.png > > Time Spent: 10m > Remaining Estimate: 0h > > While manually testing the new WebUI I found a few frictions. > * when using the UI the left panel hides unexpectedly at random moments > * mouse wheel does not work on the logs (taskmanager, jobmanager) pane > * the jobmanager configuration is not sorted > * different sorting of the operators (the old UI showed the sources first) > * the drop-down list for choosing operator/tasks metrics is not sorted, which > makes it super hard to screen through available metrics > * arrow does not touch the rectangles in Chrome (see attached screenshot) > There are also some views missing in the new UI that I personally found > useful in the old UI: > * can't see watermarks for all operators at once > * no numeric metrics (only graphs) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] hongtao12310 closed pull request #9237: [FLINK-13431][hive] NameNode HA configuration was not loaded when running HiveConnector on Yarn
hongtao12310 closed pull request #9237: [FLINK-13431][hive] NameNode HA configuration was not loaded when running HiveConnector on Yarn URL: https://github.com/apache/flink/pull/9237 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] hongtao12310 commented on issue #9237: [FLINK-13431][hive] NameNode HA configuration was not loaded when running HiveConnector on Yarn
hongtao12310 commented on issue #9237: [FLINK-13431][hive] NameNode HA configuration was not loaded when running HiveConnector on Yarn URL: https://github.com/apache/flink/pull/9237#issuecomment-515662053 miss the description of the PR and failed the CI build. will close the pr and launch a new one This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-13445) Distinguishing Memory Configuration for TaskManager and JobManager
madong created FLINK-13445: -- Summary: Distinguishing Memory Configuration for TaskManager and JobManager Key: FLINK-13445 URL: https://issues.apache.org/jira/browse/FLINK-13445 Project: Flink Issue Type: New Feature Components: Runtime / Configuration Affects Versions: 1.8.1 Reporter: madong we use flink to run some job build in non-java language, so we increase the value of `containerized.heap-cutoff-ratio` to reserve more memory for non-java process , which would affect memory allocation for jobManager. Considering the different behaviors of taskManager and jobManager, should we use this configuration separately？ -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot commented on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot commented on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515660312 ## CI report: * dbe883c57e689ed544de09423192843c758bfa54 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120935623) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
flinkbot commented on issue #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247#issuecomment-515659573 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13386) Frictions in the new default Web Frontend
[ https://issues.apache.org/jira/browse/FLINK-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-13386: --- Labels: pull-request-available (was: ) > Frictions in the new default Web Frontend > - > > Key: FLINK-13386 > URL: https://issues.apache.org/jira/browse/FLINK-13386 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.9.0 >Reporter: Dawid Wysakowicz >Priority: Critical > Labels: pull-request-available > Fix For: 1.9.0 > > Attachments: bug.png > > > While manually testing the new WebUI I found a few frictions. > * when using the UI the left panel hides unexpectedly at random moments > * mouse wheel does not work on the logs (taskmanager, jobmanager) pane > * the jobmanager configuration is not sorted > * different sorting of the operators (the old UI showed the sources first) > * the drop-down list for choosing operator/tasks metrics is not sorted, which > makes it super hard to screen through available metrics > * arrow does not touch the rectangles in Chrome (see attached screenshot) > There are also some views missing in the new UI that I personally found > useful in the old UI: > * can't see watermarks for all operators at once > * no numeric metrics (only graphs) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] vthinkxie opened a new pull request #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend
vthinkxie opened a new pull request #9247: [FLINK-13386][web]: Fix frictions in the new default Web Frontend URL: https://github.com/apache/flink/pull/9247 Fix frictions in the new default Web Frontend ## What is the purpose of the change - [ ] the jobmanager configuration is not sorted - [ ] different sorting of the operators (the old UI showed the sources first) - [ ] the drop-down list for choosing operator/tasks metrics is not sorted, which makes it super hard to screen through available metrics There are also some views missing in the new UI that I personally found useful in the old UI: - [ ] can't see watermarks for all operators at once - [ ] no numeric metrics (only graphs) ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no) - The serializers: (yes / no / don't know) - The runtime per-record code paths (performance sensitive): (yes / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know) - The S3 file system connector: (yes / no / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9203: [FLINK-13375][table-api] Improve config names in ExecutionConfigOptions and OptimizerConfigOptions
flinkbot edited a comment on issue #9203: [FLINK-13375][table-api] Improve config names in ExecutionConfigOptions and OptimizerConfigOptions URL: https://github.com/apache/flink/pull/9203#issuecomment-514046368 ## CI report: * f5e680b52e6a85e85642fc22a41724c5a452505c : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120109078) * c1388ab2867ad134b2300ccad6ca519eff547ccb : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120216941) * b70392e5d7337da2a416316068397f92925a0d7e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120455507) * 46e98c30bbe0709b2261ddd5ddb2963bf91ccfcc : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120935409) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on issue #9203: [FLINK-13375][table-api] Improve config names in ExecutionConfigOptions and OptimizerConfigOptions
wuchong commented on issue #9203: [FLINK-13375][table-api] Improve config names in ExecutionConfigOptions and OptimizerConfigOptions URL: https://github.com/apache/flink/pull/9203#issuecomment-515659009 Hi @JingsongLi @godfreyhe , I have updated the pull request to adapt blink planner to the new config names. In order for easy reviewing, I split it into 2 commits: 1. the first one is just renaming config field names. It touches many files but all is renaming. 2. the second one is renaming config key names, improving description, changing `Integer` option -> `String` option (memory size). Feel free to leave comments. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13386) Frictions in the new default Web Frontend
[ https://issues.apache.org/jira/browse/FLINK-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894306#comment-16894306 ] Yadong Xie commented on FLINK-13386: Hi [~dawidwys] * when using the UI the left panel hides unexpectedly at random moments the left panel will collapse when you first enter the job detail page * mouse wheel does not work on the logs (taskmanager, jobmanager) pane I can not reproduce it, could you provide more info about your browser version and operation system(windows or mac) * the jobmanager configuration is not sorted I will fix this * different sorting of the operators (the old UI showed the sources first) I will fix this * the drop-down list for choosing operator/tasks metrics is not sorted, which makes it super hard to screen through available metrics I will fix this * arrow does not touch the rectangles in Chrome (see attached screenshot) I can not reproduce it, could you provide more info about your browser version and operation system(windows or mac) There are also some views missing in the new UI that I personally found useful in the old UI: * can't see watermarks for all operators at once I will fix this * no numeric metrics (only graphs) I will fix this > Frictions in the new default Web Frontend > - > > Key: FLINK-13386 > URL: https://issues.apache.org/jira/browse/FLINK-13386 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.9.0 >Reporter: Dawid Wysakowicz >Priority: Critical > Fix For: 1.9.0 > > Attachments: bug.png > > > While manually testing the new WebUI I found a few frictions. > * when using the UI the left panel hides unexpectedly at random moments > * mouse wheel does not work on the logs (taskmanager, jobmanager) pane > * the jobmanager configuration is not sorted > * different sorting of the operators (the old UI showed the sources first) > * the drop-down list for choosing operator/tasks metrics is not sorted, which > makes it super hard to screen through available metrics > * arrow does not touch the rectangles in Chrome (see attached screenshot) > There are also some views missing in the new UI that I personally found > useful in the old UI: > * can't see watermarks for all operators at once > * no numeric metrics (only graphs) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-13386) Frictions in the new default Web Frontend
[ https://issues.apache.org/jira/browse/FLINK-13386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894299#comment-16894299 ] Yadong Xie commented on FLINK-13386: [~till.rohrmann] sure, I will check this, thanks > Frictions in the new default Web Frontend > - > > Key: FLINK-13386 > URL: https://issues.apache.org/jira/browse/FLINK-13386 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.9.0 >Reporter: Dawid Wysakowicz >Priority: Critical > Fix For: 1.9.0 > > Attachments: bug.png > > > While manually testing the new WebUI I found a few frictions. > * when using the UI the left panel hides unexpectedly at random moments > * mouse wheel does not work on the logs (taskmanager, jobmanager) pane > * the jobmanager configuration is not sorted > * different sorting of the operators (the old UI showed the sources first) > * the drop-down list for choosing operator/tasks metrics is not sorted, which > makes it super hard to screen through available metrics > * arrow does not touch the rectangles in Chrome (see attached screenshot) > There are also some views missing in the new UI that I personally found > useful in the old UI: > * can't see watermarks for all operators at once > * no numeric metrics (only graphs) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance.
flinkbot edited a comment on issue #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance. URL: https://github.com/apache/flink/pull/9105#issuecomment-511499675 ## CI report: * 7b6a81ed94056dd217f7feba9b155158fcfbfc4a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119404317) * 1e289e4de143a390eac564e38c6d5439c650f17a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441215) * dceeac0b80e6695d534e55fbbe3e83aa4262dbc9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119568870) * a2e403e357690465dc8ca1b3b266989d25ac155f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119766340) * 70d2ed3eebc9874eb70559fc5a72b5366392b0df : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120817288) * 14ff63f68d5e683d6ba15a02e340e9d867ed7742 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120934246) * a41ad84cacf54c7d4001448e501924c286233622 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120934632) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9246: [FLINK-13241][Yarn/Mesos] Duplication of PR#9105 on master branch.
flinkbot edited a comment on issue #9246: [FLINK-13241][Yarn/Mesos] Duplication of PR#9105 on master branch. URL: https://github.com/apache/flink/pull/9246#issuecomment-515656507 ## CI report: * 89ca6ef411dda57c14ef27cb90e8cfbb4ca4fb99 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120934490) * bc47897725c5ae01d840c805023117037cfd9e5b : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120934627) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9246: [FLINK-13241][Yarn/Mesos] Duplication of PR#9105 on master branch.
flinkbot commented on issue #9246: [FLINK-13241][Yarn/Mesos] Duplication of PR#9105 on master branch. URL: https://github.com/apache/flink/pull/9246#issuecomment-515656507 ## CI report: * 89ca6ef411dda57c14ef27cb90e8cfbb4ca4fb99 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120934490) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9246: [FLINK-13241][Yarn/Mesos] Duplication of PR#9105 on master branch.
flinkbot commented on issue #9246: [FLINK-13241][Yarn/Mesos] Duplication of PR#9105 on master branch. URL: https://github.com/apache/flink/pull/9246#issuecomment-515656023 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong opened a new pull request #9246: [FLINK-13241][Yarn/Mesos] Duplication of PR#9105 on master branch.
xintongsong opened a new pull request #9246: [FLINK-13241][Yarn/Mesos] Duplication of PR#9105 on master branch. URL: https://github.com/apache/flink/pull/9246 This PR is a duplication of #9105 on master branch. The original PR #9105 was opened on release-1.9 branch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance.
flinkbot edited a comment on issue #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance. URL: https://github.com/apache/flink/pull/9105#issuecomment-511499675 ## CI report: * 7b6a81ed94056dd217f7feba9b155158fcfbfc4a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119404317) * 1e289e4de143a390eac564e38c6d5439c650f17a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441215) * dceeac0b80e6695d534e55fbbe3e83aa4262dbc9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119568870) * a2e403e357690465dc8ca1b3b266989d25ac155f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119766340) * 70d2ed3eebc9874eb70559fc5a72b5366392b0df : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120817288) * 14ff63f68d5e683d6ba15a02e340e9d867ed7742 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120934246) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
