[GitHub] [flink] flinkbot edited a comment on pull request #13559: [Hotfix][Docs] Fix broken link of flink-architecture.md

[GitBox]

flinkbot edited a comment on pull request #13559:
URL: https://github.com/apache/flink/pull/13559#issuecomment-705309728


   
   ## CI report:
   
   * 4de3c79dd9c54c4ce6fd1b68dc9d4b0a953080c0 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7281)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

flinkbot edited a comment on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-704987370


   
   ## CI report:
   
   * 84dcff98907a214d6e2e2b4490ed7f66816bc396 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7282)
 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7280)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (FLINK-19532) FlinkZooKeeperQuorumPeer ignores purge settings

[M L (Jira)]

M L created FLINK-19532:
---

 Summary: FlinkZooKeeperQuorumPeer ignores purge settings
 Key: FLINK-19532
 URL: https://issues.apache.org/jira/browse/FLINK-19532
 Project: Flink
  Issue Type: Bug
Reporter: M L


After noticing the snapshot/log buildup in our ZooKeeper {{dataDir}}, we tried 
enabling the purge settings ({{autopurge.snapRetainCount}} and 
autopurge.purgeInterval).

Flink's {{FlinkZooKeeperQuorumPeer}} ignores the portion of {{QuorumPeerMain}} 
which uses those settings and starts up the {{DatadirCleanupManager}}, using 
only the {{runFromConfig}} method.

Is this intentional and is the recommended way to handle snapshot buildup with 
the built in ZooKeeper to use the {{PurgeTxnLog}} utility?

It would make sense to enable to support the {{DatadirCleanupManager}} as well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot edited a comment on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

flinkbot edited a comment on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-704987370


   
   ## CI report:
   
   * 84dcff98907a214d6e2e2b4490ed7f66816bc396 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7280)
 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7282)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] scottxusayhi commented on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

scottxusayhi commented on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-705325854


   @flinkbot run azure



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13559: [Hotfix][Docs] Fix broken link of flink-architecture.md

[GitBox]

flinkbot edited a comment on pull request #13559:
URL: https://github.com/apache/flink/pull/13559#issuecomment-705309728


   
   ## CI report:
   
   * 4de3c79dd9c54c4ce6fd1b68dc9d4b0a953080c0 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7281)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #13559: [Hotfix][Docs] Fix broken link of flink-architecture.md

[GitBox]

flinkbot commented on pull request #13559:
URL: https://github.com/apache/flink/pull/13559#issuecomment-705309728


   
   ## CI report:
   
   * 4de3c79dd9c54c4ce6fd1b68dc9d4b0a953080c0 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] scottxusayhi removed a comment on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

scottxusayhi removed a comment on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-705299515


   @flinkbot run travis



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #13559: [Hotfix][Docs] Fix broken link of flink-architecture.md

[GitBox]

flinkbot commented on pull request #13559:
URL: https://github.com/apache/flink/pull/13559#issuecomment-705307598


   Thanks a lot for your contribution to the Apache Flink project. I'm the 
@flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress 
of the review.
   
   
   ## Automated Checks
   Last check on commit 4de3c79dd9c54c4ce6fd1b68dc9d4b0a953080c0 (Thu Oct 08 
03:28:28 UTC 2020)
   
   **Warnings:**
* **Invalid pull request title: No valid Jira ID provided**
   
   
   Mention the bot in a comment to re-run the automated checks.
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review 
Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full 
explanation of the review process.
The Bot is tracking the review progress through labels. Labels are applied 
according to the order of the review items. For consensus, approval by a Flink 
committer of PMC member is required Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot approve description` to approve one or more aspects (aspects: 
`description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until 
`architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's 
attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] wangxlong opened a new pull request #13559: [Hotfix][Docs] Fix broken link of flink-architecture.md

[GitBox]

wangxlong opened a new pull request #13559:
URL: https://github.com/apache/flink/pull/13559


   
   ## What is the purpose of the change
   
   Fix broken link of flink-architecture.md
   
   ## Brief change log
   
   Fix broken link of flink-architecture.md
   
   ## Verifying this change
   
   This change is a trivial rework / code cleanup without any test coverage.
   
   ## Does this pull request potentially affect one of the following parts:
   
 - Dependencies (does it add or upgrade a dependency): (no)
 - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: ( no)
 - The serializers: (no / don't know)
 - The runtime per-record code paths (performance sensitive): (no / don't 
know)
 - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no / don't know)
 - The S3 file system connector: ( no / don't know)
   
   ## Documentation
   
 - Does this pull request introduce a new feature? ( no)
 - If yes, how is the feature documented? (not applicable / docs / JavaDocs 
/ not documented)
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

flinkbot edited a comment on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-704987370


   
   ## CI report:
   
   *  Unknown: [CANCELED](TBD) 
   * 84dcff98907a214d6e2e2b4490ed7f66816bc396 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7280)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

flinkbot edited a comment on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-704987370


   
   ## CI report:
   
   *  Unknown: [CANCELED](TBD) 
   * 84dcff98907a214d6e2e2b4490ed7f66816bc396 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13558: [FLINK-19511] Rename the 'SinkTransformation' to 'LegacySinkTransformation'

[GitBox]

flinkbot edited a comment on pull request #13558:
URL: https://github.com/apache/flink/pull/13558#issuecomment-705296235


   
   ## CI report:
   
   * 49ecba9b70a2face6928f6db7c0e35c66ac260cb Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7279)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] gaoyunhaii commented on pull request #13558: [FLINK-19511] Rename the 'SinkTransformation' to 'LegacySinkTransformation'

[GitBox]

gaoyunhaii commented on pull request #13558:
URL: https://github.com/apache/flink/pull/13558#issuecomment-705300208


   Very thanks @guoweiM for opening this PR! It seems that are still some 
references to `SinkTransformation` in the method comments:
   
   `PartitionTransformation`: line 79
   `LegacySinkTransformation` line 40, 81, 93
   
   after the modification.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] scottxusayhi commented on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

scottxusayhi commented on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-705299515


   @flinkbot run travis



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] scottxusayhi removed a comment on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

scottxusayhi removed a comment on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-705271213


   @flinkbot run azure



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #13558: [FLINK-19511] Rename the 'SinkTransformation' to 'LegacySinkTransformation'

[GitBox]

flinkbot commented on pull request #13558:
URL: https://github.com/apache/flink/pull/13558#issuecomment-705296235


   
   ## CI report:
   
   * 49ecba9b70a2face6928f6db7c0e35c66ac260cb UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (FLINK-19531) Implement the `WriterOperator`

[Guowei Ma (Jira)]

Guowei Ma created FLINK-19531:
-

 Summary: Implement the `WriterOperator`
 Key: FLINK-19531
 URL: https://issues.apache.org/jira/browse/FLINK-19531
 Project: Flink
  Issue Type: Sub-task
  Components: API / DataStream
Reporter: Guowei Ma






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot commented on pull request #13558: [FLINK-19511] Rename the 'SinkTransformation' to 'LegacySinkTransformation'

[GitBox]

flinkbot commented on pull request #13558:
URL: https://github.com/apache/flink/pull/13558#issuecomment-705290770


   Thanks a lot for your contribution to the Apache Flink project. I'm the 
@flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress 
of the review.
   
   
   ## Automated Checks
   Last check on commit 49ecba9b70a2face6928f6db7c0e35c66ac260cb (Thu Oct 08 
02:20:47 UTC 2020)
   
   **Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up 
to date!
   
   
   Mention the bot in a comment to re-run the automated checks.
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review 
Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full 
explanation of the review process.
The Bot is tracking the review progress through labels. Labels are applied 
according to the order of the review items. For consensus, approval by a Flink 
committer of PMC member is required Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot approve description` to approve one or more aspects (aspects: 
`description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until 
`architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's 
attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (FLINK-19511) Rename the `SinkTransformation` to `LegacySinkTransformation`

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-19511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-19511:
---
Labels: pull-request-available  (was: )

> Rename the `SinkTransformation` to `LegacySinkTransformation`
> -
>
> Key: FLINK-19511
> URL: https://issues.apache.org/jira/browse/FLINK-19511
> Project: Flink
>  Issue Type: Sub-task
>  Components: API / DataStream
>Reporter: Guowei Ma
>Assignee: Guowei Ma
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] guoweiM opened a new pull request #13558: [FLINK-19511] Rename the 'SinkTransformation' to 'LegacySinkTransformation'

[GitBox]

guoweiM opened a new pull request #13558:
URL: https://github.com/apache/flink/pull/13558


   
   
   ## What is the purpose of the change
   
   We use the 'SinkTransformation' to represent the new sink API after the 
FLIP-143. So this commit renames the 'SinkTransformation' to 
'LegacySinkTransformation'.
   ## Brief change log
   
   ## Verifying this change
   
   This change is a trivial rework / code cleanup without any test coverage.
   
   ## Does this pull request potentially affect one of the following parts:
   
 - Dependencies (does it add or upgrade a dependency): (no)
 - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
 - The serializers: (no)
 - The runtime per-record code paths (performance sensitive): (no)
 - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
 - The S3 file system connector: (no)
   
   ## Documentation
   
 - Does this pull request introduce a new feature? (no)
 - If yes, how is the feature documented? (not applicable)
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] guoweiM commented on pull request #6613: [FLINK-9940] [API/DataStream][File source] out-of-order files were missed in continuous monitoring

[GitBox]

guoweiM commented on pull request #6613:
URL: https://github.com/apache/flink/pull/6613#issuecomment-705278656


   Do not worry. Take your time.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

flinkbot edited a comment on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-704987370


   
   ## CI report:
   
   * 7d7d685a6d87ddc77424bf0e8e07ce491e717f0e Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7278)
 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7269)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] scottxusayhi removed a comment on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

scottxusayhi removed a comment on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-705273593


   @flinkbot run azure
   
   On Wed, 7 Oct 2020 at 22:47, flinkbot  wrote:
   
   > CI report:
   >
   >- 7d7d685
   >

   >UNKNOWN
   >
   > Bot commands The @flinkbot  bot supports the
   > following commands:
   >
   >- @flinkbot run travis re-run the last Travis build
   >- @flinkbot run azure re-run the last Azure build
   >
   > —
   > You are receiving this because you authored the thread.
   > Reply to this email directly, view it on GitHub
   > , or
   > unsubscribe
   > 

   > .
   >
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] scottxusayhi commented on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

scottxusayhi commented on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-705273593


   @flinkbot run azure
   
   On Wed, 7 Oct 2020 at 22:47, flinkbot  wrote:
   
   > CI report:
   >
   >- 7d7d685
   >

   >UNKNOWN
   >
   > Bot commands The @flinkbot  bot supports the
   > following commands:
   >
   >- @flinkbot run travis re-run the last Travis build
   >- @flinkbot run azure re-run the last Azure build
   >
   > —
   > You are receiving this because you authored the thread.
   > Reply to this email directly, view it on GitHub
   > , or
   > unsubscribe
   > 

   > .
   >
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

flinkbot edited a comment on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-704987370


   
   ## CI report:
   
   * 7d7d685a6d87ddc77424bf0e8e07ce491e717f0e Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7269)
 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7278)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] scottxusayhi removed a comment on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

scottxusayhi removed a comment on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-704987085


   @wangyang0918 could you help to review?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] scottxusayhi commented on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

scottxusayhi commented on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-705271213


   @flinkbot run azure



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

flinkbot edited a comment on pull request #13535:
URL: https://github.com/apache/flink/pull/13535#issuecomment-702896138


   
   ## CI report:
   
   * 4ed984adbb753d9287a4d5d71d8f2d149f86bba0 UNKNOWN
   * b79adda4c4eae4533e7d521b0c52b94f25db4075 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7277)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

flinkbot edited a comment on pull request #13535:
URL: https://github.com/apache/flink/pull/13535#issuecomment-702896138


   
   ## CI report:
   
   * 4ed984adbb753d9287a4d5d71d8f2d149f86bba0 UNKNOWN
   * 505ec8554bfb64d4f53a9f619ffc0729e3e645f3 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7273)
 
   * b79adda4c4eae4533e7d521b0c52b94f25db4075 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7277)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

flinkbot edited a comment on pull request #13535:
URL: https://github.com/apache/flink/pull/13535#issuecomment-702896138


   
   ## CI report:
   
   * 97e855d306d15d90259b65d3db38b7a19c62c6e9 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7271)
 
   * 4ed984adbb753d9287a4d5d71d8f2d149f86bba0 UNKNOWN
   * 505ec8554bfb64d4f53a9f619ffc0729e3e645f3 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7273)
 
   * b79adda4c4eae4533e7d521b0c52b94f25db4075 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7277)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (FLINK-19527) Update SQL Pages

[Seth Wiesman (Jira)]

Seth Wiesman created FLINK-19527:


 Summary: Update SQL Pages
 Key: FLINK-19527
 URL: https://issues.apache.org/jira/browse/FLINK-19527
 Project: Flink
  Issue Type: Sub-task
  Components: Documentation
Reporter: Seth Wiesman


SQL
Goal: Show users the main features early and link to concepts if necessary.
How to use SQL? Intended for users with SQL knowledge.

Overview
Getting started with link to more detailed execution section.
Full Reference
Available operations in SQL as a table. This location allows to further 
split the page in the future if we think an operation needs more space without 
affecting the top-level structure.
Data Definition
Explain special SQL syntax around DDL.
Pattern Matching
Make pattern matching more visible.
... more features in the future 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-19529) Table connectors docs

[Seth Wiesman (Jira)]

Seth Wiesman created FLINK-19529:


 Summary: Table connectors docs
 Key: FLINK-19529
 URL: https://issues.apache.org/jira/browse/FLINK-19529
 Project: Flink
  Issue Type: Sub-task
  Components: Documentation
Reporter: Seth Wiesman


Connect to External Systems
How to connect to other systems for data or metadata?

Overview
What are available sources and sinks?
What are catalogs? What can I manage with it?
Available Connectors
Available Catalogs
Hive



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-19530) Table Concepts Page

[Seth Wiesman (Jira)]

Seth Wiesman created FLINK-19530:


 Summary: Table Concepts Page
 Key: FLINK-19530
 URL: https://issues.apache.org/jira/browse/FLINK-19530
 Project: Flink
  Issue Type: Sub-task
  Components: Documentation
Reporter: Seth Wiesman


Concepts
What are the general concepts (independent of API/SQL Client) any user should 
know about?
We put this at the end and link from the main pages to pages here if necessary.

Planners
What is a Planner? Temporary docs. Removed in the future.
Blink Planner Features and Limitations
Flink Planner Features and Limitations
Data Types
Which data can we process?
Unbounded Data Processing
Which operation needs special attention when working with unbounded data?
Dynamic Tables (with all update modes)
Time Attributes
Query Configuration
Joins in Continuous Queries
Temporal Tables
Explain the concept of a temporal table.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-19526) Update Table API specific section

[Seth Wiesman (Jira)]

Seth Wiesman created FLINK-19526:


 Summary: Update Table API specific section
 Key: FLINK-19526
 URL: https://issues.apache.org/jira/browse/FLINK-19526
 Project: Flink
  Issue Type: Sub-task
  Components: Documentation
Reporter: Seth Wiesman


Table API
Goal: Show users the main table features early and link to concepts if 
necessary.
How to use the API? Intended for users with programming knowledge.

Overview
Short getting started with link to more detailed execution section.
Explain the most important methods in unified Table Environment
Present sqlUpdate/sqlQuery etc.
Querying, Execution, Optimization internals behind the API
Full Reference
Available operations in the API. This location allows us to further split 
the page in the future if we think an operation needs more space without 
affecting the top-level structure.
Present the API operations
... more features in the future 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-19528) Setup and SQL execution

[Seth Wiesman (Jira)]

Seth Wiesman created FLINK-19528:


 Summary: Setup and SQL execution
 Key: FLINK-19528
 URL: https://issues.apache.org/jira/browse/FLINK-19528
 Project: Flink
  Issue Type: Sub-task
Reporter: Seth Wiesman


Setup & Execution

Programmatic
How to setup a project and submit a job?
Dependency structure for using the API
TableEnvironments
Features and limitations of table environments
How to setup Python projects?
SQL Client
How to use the SQL Client? Docs for pure SQL users.
… Notebooks, JDBC, and more executions in the future



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-19525) Update Table API Overview page

[Seth Wiesman (Jira)]

Seth Wiesman created FLINK-19525:


 Summary: Update Table API Overview page
 Key: FLINK-19525
 URL: https://issues.apache.org/jira/browse/FLINK-19525
 Project: Flink
  Issue Type: Sub-task
Reporter: Seth Wiesman


Overview
What is the Table & SQL Ecosystem?
Might be a big page but is a nice executive summary and informative.

Main features like schema awareness, abstraction, connectors, catalogs etc.
How do we achieve unified data processing? Dynamic Tables
Quickly mention planners
Advantages/Disadvantages over DataStream API
E2E example for SQL
E2E example for Table API in Java/Scala/Python
E2E example for Table API in Java/Scala/Python with DataStream API
Short presentation of the SQL Client



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-19524) Google Season of Docs - FLIP 60

[Seth Wiesman (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Seth Wiesman updated FLINK-19524:
-
Issue Type: Improvement  (was: Bug)

> Google Season of Docs - FLIP 60
> ---
>
> Key: FLINK-19524
> URL: https://issues.apache.org/jira/browse/FLINK-19524
> Project: Flink
>  Issue Type: Improvement
>  Components: Documentation, Table SQL / API
>Reporter: Seth Wiesman
>Assignee: Seth Wiesman
>Priority: Major
>
> This is an umbrella ticket to track the implementation of FLIP-60 that will 
> occur as part of the Flink community's involvement in Google Season of Docs. 
> Please refer to the FLIP for a full outline of work: 
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-19524) Google Season of Docs - FLIP 60

[Seth Wiesman (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Seth Wiesman updated FLINK-19524:
-
Description: 
This is an umbrella ticket to track the implementation of FLIP-60 that will 
occur as part of the Flink community's involvement in Google Season of Docs. 

Please refer to the FLIP for a full outline of work: 
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685

  was:This is an umbrella ticket to track the implementation of FLIP-60 that 
will occur as part of the Flink community's involvement in Google Season of 
Docs. 


> Google Season of Docs - FLIP 60
> ---
>
> Key: FLINK-19524
> URL: https://issues.apache.org/jira/browse/FLINK-19524
> Project: Flink
>  Issue Type: Bug
>  Components: Documentation, Table SQL / API
>Reporter: Seth Wiesman
>Assignee: Seth Wiesman
>Priority: Major
>
> This is an umbrella ticket to track the implementation of FLIP-60 that will 
> occur as part of the Flink community's involvement in Google Season of Docs. 
> Please refer to the FLIP for a full outline of work: 
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-19524) Google Season of Docs - FLIP 60

[Seth Wiesman (Jira)]

Seth Wiesman created FLINK-19524:


 Summary: Google Season of Docs - FLIP 60
 Key: FLINK-19524
 URL: https://issues.apache.org/jira/browse/FLINK-19524
 Project: Flink
  Issue Type: Bug
  Components: Documentation, Table SQL / API
Reporter: Seth Wiesman
Assignee: Seth Wiesman


This is an umbrella ticket to track the implementation of FLIP-60 that will 
occur as part of the Flink community's involvement in Google Season of Docs. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-19523) Secrets should be filtered out from logs

[Qinghui Xu (Jira)]

Qinghui Xu created FLINK-19523:
--

 Summary: Secrets should be filtered out from logs
 Key: FLINK-19523
 URL: https://issues.apache.org/jira/browse/FLINK-19523
 Project: Flink
  Issue Type: Improvement
Reporter: Qinghui Xu


When jobmanager starts up, it logs configurations from jvm options and 
application arguments. This includes secrets (eg. "s3.secret-key").
Ideally, secrets should be masked as it is when loading from configuration file.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot edited a comment on pull request #13557: [FLINK-19516] Let MiniClusterJobClient.jobResultFuture complete after MiniCluster shut down

[GitBox]

flinkbot edited a comment on pull request #13557:
URL: https://github.com/apache/flink/pull/13557#issuecomment-705029634


   
   ## CI report:
   
   * c9fbc4b0d96ce887beebd324ebf5c3fdd6736a8c Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7272)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

flinkbot edited a comment on pull request #13535:
URL: https://github.com/apache/flink/pull/13535#issuecomment-702896138


   
   ## CI report:
   
   * 97e855d306d15d90259b65d3db38b7a19c62c6e9 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7271)
 
   * 4ed984adbb753d9287a4d5d71d8f2d149f86bba0 UNKNOWN
   * 505ec8554bfb64d4f53a9f619ffc0729e3e645f3 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7273)
 
   * b79adda4c4eae4533e7d521b0c52b94f25db4075 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] sjwiesman commented on pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

sjwiesman commented on pull request #13535:
URL: https://github.com/apache/flink/pull/13535#issuecomment-705166886


   @knaufk thanks for taking a look, I've pushed a fix that should address all 
of your comments. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

flinkbot edited a comment on pull request #13535:
URL: https://github.com/apache/flink/pull/13535#issuecomment-702896138


   
   ## CI report:
   
   * 97e855d306d15d90259b65d3db38b7a19c62c6e9 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7271)
 
   * 4ed984adbb753d9287a4d5d71d8f2d149f86bba0 UNKNOWN
   * 505ec8554bfb64d4f53a9f619ffc0729e3e645f3 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7273)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (FLINK-19461) yarn-sesson.sh -jm -tm arguments have no effect

[Robert Metzger (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Metzger updated FLINK-19461:
---
Attachment: yarn-debug.log

> yarn-sesson.sh -jm -tm arguments have no effect
> ---
>
> Key: FLINK-19461
> URL: https://issues.apache.org/jira/browse/FLINK-19461
> Project: Flink
>  Issue Type: Bug
>  Components: Command Line Client, Deployment / YARN
>Affects Versions: 1.12.0
>Reporter: Robert Metzger
>Priority: Major
>  Labels: usability
> Fix For: 1.12.0
>
> Attachments: jobmanager.log, yarn-debug.log
>
>
> It seems that I can set arbitrary values for the documented {{-jm}} and 
> {{-tm}} arguments, not leading to any effects.
> Example: {{./bin/yarn-session -jm 512m -tm 512m}} should fail in my opinion, 
> but it starts with the default memory configuration (1280mb / 1200mb? or so).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-19461) yarn-sesson.sh -jm -tm arguments have no effect

[Robert Metzger (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Metzger updated FLINK-19461:
---
Attachment: jobmanager.log

> yarn-sesson.sh -jm -tm arguments have no effect
> ---
>
> Key: FLINK-19461
> URL: https://issues.apache.org/jira/browse/FLINK-19461
> Project: Flink
>  Issue Type: Bug
>  Components: Command Line Client, Deployment / YARN
>Affects Versions: 1.12.0
>Reporter: Robert Metzger
>Priority: Major
>  Labels: usability
> Fix For: 1.12.0
>
> Attachments: jobmanager.log
>
>
> It seems that I can set arbitrary values for the documented {{-jm}} and 
> {{-tm}} arguments, not leading to any effects.
> Example: {{./bin/yarn-session -jm 512m -tm 512m}} should fail in my opinion, 
> but it starts with the default memory configuration (1280mb / 1200mb? or so).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-19461) yarn-sesson.sh -jm -tm arguments have no effect

[Robert Metzger (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209817#comment-17209817
 ] 

Robert Metzger commented on FLINK-19461:


1) -ytm and -yjm has the same (erroneous) behavior.

2) I started the session with: 
{code}./bin/yarn-session.sh -ytm 512m -ytm 512m{code}
I attached the jobmanager.log on debug level.

> yarn-sesson.sh -jm -tm arguments have no effect
> ---
>
> Key: FLINK-19461
> URL: https://issues.apache.org/jira/browse/FLINK-19461
> Project: Flink
>  Issue Type: Bug
>  Components: Command Line Client, Deployment / YARN
>Affects Versions: 1.12.0
>Reporter: Robert Metzger
>Priority: Major
>  Labels: usability
> Fix For: 1.12.0
>
> Attachments: jobmanager.log
>
>
> It seems that I can set arbitrary values for the documented {{-jm}} and 
> {{-tm}} arguments, not leading to any effects.
> Example: {{./bin/yarn-session -jm 512m -tm 512m}} should fail in my opinion, 
> but it starts with the default memory configuration (1280mb / 1200mb? or so).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Closed] (FLINK-19505) Implement distributed DSTL (DFS-based)

[Roman Khachatryan (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-19505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Khachatryan closed FLINK-19505.
-
Resolution: Invalid

> Implement distributed DSTL (DFS-based)
> --
>
> Key: FLINK-19505
> URL: https://issues.apache.org/jira/browse/FLINK-19505
> Project: Flink
>  Issue Type: New Feature
>  Components: Runtime / Checkpointing, Runtime / Task
>Reporter: Roman Khachatryan
>Assignee: Roman Khachatryan
>Priority: Major
>  Labels: DSTL
> Fix For: 1.12.0
>
>
> Sub-tasks:
>  # Implement scheduling and deployment (no rescaling)
>  # Implement RPC between the tasks and DSTL (including load balancing)
>  # Implement re-scaling
>  # Implement filtering of records on replay to support up-scaling



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Closed] (FLINK-19503) Add DSTL interface and its in-memory implementation for testing

[Roman Khachatryan (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-19503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Khachatryan closed FLINK-19503.
-
Resolution: Invalid

> Add DSTL interface and its in-memory implementation for testing
> ---
>
> Key: FLINK-19503
> URL: https://issues.apache.org/jira/browse/FLINK-19503
> Project: Flink
>  Issue Type: New Feature
>  Components: Runtime / Checkpointing, Runtime / Task
>Reporter: Roman Khachatryan
>Assignee: Roman Khachatryan
>Priority: Major
>  Labels: DSTL
> Fix For: 1.12.0
>
>
> Durable Short-term Log is a proposed component to store state changes and 
> later output.
> The goal of this issue is to allow:
>  # Integration with changes to the state backends
>  # Implementation first production DSTL version
> The code should include interfaces, test implementations, means to access 
> from state backends, tests.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Closed] (FLINK-19504) Implement persistence for DSTL via DFS

[Roman Khachatryan (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-19504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Khachatryan closed FLINK-19504.
-
Resolution: Invalid

> Implement persistence for DSTL via DFS
> --
>
> Key: FLINK-19504
> URL: https://issues.apache.org/jira/browse/FLINK-19504
> Project: Flink
>  Issue Type: New Feature
>  Components: Runtime / Checkpointing, Runtime / Task
>Reporter: Roman Khachatryan
>Assignee: Roman Khachatryan
>Priority: Major
>  Labels: DSTL
> Fix For: 1.12.0
>
>
> Durable Short-term Log is a proposed component to store state changes and 
> later output.
> The goal of this issue is to add a persistent version of FLINK-19503 (backed 
> by DFS).
> No scheduling and RPC.
>  
> If absent, add request hedging capability.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot edited a comment on pull request #13551: [FLINK-19520][configuration] Add randomization of checkpoint config.

[GitBox]

flinkbot edited a comment on pull request #13551:
URL: https://github.com/apache/flink/pull/13551#issuecomment-704858051


   
   ## CI report:
   
   * fe4a6228d31108e71a6e9faab97598ffa9a09b8b Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7268)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

flinkbot edited a comment on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-704987370


   
   ## CI report:
   
   * 7d7d685a6d87ddc77424bf0e8e07ce491e717f0e Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7269)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13556: [FLINK-19485] Decouple runtime operator implementation from DataStream operations

[GitBox]

flinkbot edited a comment on pull request #13556:
URL: https://github.com/apache/flink/pull/13556#issuecomment-704987982


   
   ## CI report:
   
   * 40f3f7d289665256ea5a8a938cbbf4ded64396f1 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7270)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (FLINK-19491) AvroSerializerSnapshot cannot handle large schema

[Arvid Heise (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209778#comment-17209778
 ] 

Arvid Heise commented on FLINK-19491:
-

Hi [~nicholasjiang] , there is no inherent reason to believe that Avro schemas 
are only 64kb. It's an unfortunate limitation of the current implementation 
that can be easily mended by added a V3 of the snapshot version that uses a 
different version to store the string, for example by using 
{{DataOutput#writeChars}} instead of {{#writeUTF}}.

> AvroSerializerSnapshot cannot handle large schema
> -
>
> Key: FLINK-19491
> URL: https://issues.apache.org/jira/browse/FLINK-19491
> Project: Flink
>  Issue Type: Bug
>  Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
>Affects Versions: 1.10.2, 1.12.0, 1.11.2
>Reporter: Arvid Heise
>Priority: Major
>
> Flink can only handle schemas up to a size of 64kb.
>  
> {noformat}
> Caused by: java.io.UTFDataFormatException: encoded string too long: 223502 
> bytes
>   at java.io.DataOutputStream.writeUTF(DataOutputStream.java:364)
>   at java.io.DataOutputStream.writeUTF(DataOutputStream.java:323)
>   at 
> org.apache.flink.formats.avro.typeutils.AvroSerializerSnapshot.writeSnapshot(AvroSerializerSnapshot.java:75)
>   at 
> org.apache.flink.api.common.typeutils.TypeSerializerSnapshot.writeVersionedSnapshot(TypeSerializerSnapshot.java:153)
>   at 
> org.apache.flink.api.common.typeutils.NestedSerializersSnapshotDelegate.writeNestedSerializerSnapshots(NestedSerializersSnapshotDelegate.java:159)
>   at 
> org.apache.flink.api.common.typeutils.CompositeTypeSerializerSnapshot.writeSnapshot(CompositeTypeSerializerSnapshot.java:148)
>   at 
> org.apache.flink.api.common.typeutils.TypeSerializerSnapshot.writeVersionedSnapshot(TypeSerializerSnapshot.java:153)
>   at 
> org.apache.flink.api.common.typeutils.TypeSerializerSnapshotSerializationUtil$TypeSerializerSnapshotSerializationProxy.write(TypeSerializerSnapshotSerializationUtil.java:138)
>   at 
> org.apache.flink.api.common.typeutils.TypeSerializerSnapshotSerializationUtil.writeSerializerSnapshot(TypeSerializerSnapshotSerializationUtil.java:55)
>   at 
> org.apache.flink.runtime.state.metainfo.StateMetaInfoSnapshotReadersWriters$CurrentWriterImpl.writeStateMetaInfoSnapshot(StateMetaInfoSnapshotReadersWriters.java:183)
>   at 
> org.apache.flink.runtime.state.KeyedBackendSerializationProxy.write(KeyedBackendSerializationProxy.java:126)
>   at 
> org.apache.flink.runtime.state.heap.HeapSnapshotStrategy$1.callInternal(HeapSnapshotStrategy.java:171)
>   at 
> org.apache.flink.runtime.state.heap.HeapSnapshotStrategy$1.callInternal(HeapSnapshotStrategy.java:158)
>   at 
> org.apache.flink.runtime.state.AsyncSnapshotCallable.call(AsyncSnapshotCallable.java:75)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.flink.runtime.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:510)
>   ... 5 common frames omitted{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot edited a comment on pull request #13529: [FLINK-19473] Implement multi inputs sorting DataInput

[GitBox]

flinkbot edited a comment on pull request #13529:
URL: https://github.com/apache/flink/pull/13529#issuecomment-702193043


   
   ## CI report:
   
   * 0678c7621f0f261144a624044d7055a4687b98d2 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7266)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] rkhachatryan commented on pull request #13040: [FLINK-17073] [checkpointing] checkpointing backpressure if there are too many checkpoints to clean

[GitBox]

rkhachatryan commented on pull request #13040:
URL: https://github.com/apache/flink/pull/13040#issuecomment-705120586


   Hey @echauchot, thanks a lot for the contribution. The changes look mostly 
good to me, I will take care of some follow up clean ups and merge it in a 
couple of days.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13554: [FLINK-19493] In CliFrontend, make flow of Configuration more obvious

[GitBox]

flinkbot edited a comment on pull request #13554:
URL: https://github.com/apache/flink/pull/13554#issuecomment-704950126


   
   ## CI report:
   
   * 7c4621df4ac510fa1458cb6bbf34e2c8a127f42e Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7265)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink-playgrounds] shuiqiangchen commented on a change in pull request #16: [FLINK-19145][walkthroughs] Add PyFlink-walkthrough to Flink playground.

[GitBox]

shuiqiangchen commented on a change in pull request #16:
URL: https://github.com/apache/flink-playgrounds/pull/16#discussion_r501192510



##
File path: pyflink-walkthrough/README.md
##
@@ -0,0 +1,140 @@
+# pyflink-walkthrough
+
+## Background
+
+In this playground, you will learn how to build and run an end-to-end PyFlink 
pipeline for data analytics, covering the following steps:
+
+* Reading data from a Kafka source;
+* Creating data using a 
[UDF](https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/python/table-api-users-guide/udfs/python_udfs.html);
+* Performing a simple aggregation over the source data;
+* Writing the results to Elasticsearch and visualizing them in Kibana.
+
+The environment is based on Docker Compose, so the only requirement is that 
you have [Docker](https://docs.docker.com/get-docker/) 
+installed in your machine.
+
+### Kafka
+You will be using Kafka to store sample input data about payment transactions. 
A simple data generator 
[generate_source_data.py](generator/generate_source_data.py) is provided to
+continuously write new records to the `payment_msg` Kafka topic. Each record 
is structured as follows:
+ 
+`{"createTime": "2020-08-12 06:29:02", "orderId": 1597213797, "payAmount": 
28306.44976403719, "payPlatform": 0, "provinceId": 4}`
+
+* `createTime`: The creation time of the transaction. 
+* `orderId`: The id of the current transaction.
+* `payAmount`: The amount being paid with this transaction.
+* `payPlatform`: The platform used to create this payment: pc or mobile.
+* `provinceId`: The id of the province for the user. 
+
+You can use the following command to read data from the Kafka topic and check 
whether it's generated correctly:
+```shell script
+$ docker-compose exec kafka kafka-console-consumer.sh --bootstrap-server 
kafka:9092 --topic payment_msg
+{"createTime":"2020-07-27 
09:25:32.77","orderId":1595841867217,"payAmount":7732.44,"payPlatform":0,"provinceId":3}
+{"createTime":"2020-07-27 
09:25:33.231","orderId":1595841867218,"payAmount":75774.05,"payPlatform":0,"provinceId":3}
+{"createTime":"2020-07-27 
09:25:33.72","orderId":1595841867219,"payAmount":65908.55,"payPlatform":0,"provinceId":0}
+{"createTime":"2020-07-27 
09:25:34.216","orderId":1595841867220,"payAmount":15341.11,"payPlatform":0,"provinceId":1}
+{"createTime":"2020-07-27 
09:25:34.698","orderId":1595841867221,"payAmount":37504.42,"payPlatform":0,"provinceId":0}
+```
+You can also create a new topic by executing the following command:
+```shell script
+$ docker-compose exec kafka kafka-topics.sh --bootstrap-server kafka:9092 
--create --topic  --partitions 8 --replication-factor 1
+```

Review comment:
   I will move this part to the later section between before starting the 
PyFlink job after docker compose up. 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (FLINK-19497) Implement mutator methods for FlinkCounterWrapper

[Chesnay Schepler (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209701#comment-17209701
 ] 

Chesnay Schepler commented on FLINK-19497:
--

I would consider it unlikely that we release another patch release before 1.12 .

> Implement mutator methods for FlinkCounterWrapper
> -
>
> Key: FLINK-19497
> URL: https://issues.apache.org/jira/browse/FLINK-19497
> Project: Flink
>  Issue Type: Improvement
>  Components: Runtime / Metrics
>Affects Versions: 1.10.2, 1.11.2
>Reporter: Richard Moorhead
>Assignee: Richard Moorhead
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.12.0
>
>
> Looking at the dropwizard wrapper classes in flink-metrics-dropwizard, it 
> appears that all of them have mutator methods defined with the exception of 
> FlinkCounterWrapper. We have a use case wherein we mutate counters from a 
> dropwizard context but wish the underlying metrics in the flink registry to 
> be updated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot edited a comment on pull request #13547: [FLINK-14406][runtime] Exposes managed memory usage through the REST API

[GitBox]

flinkbot edited a comment on pull request #13547:
URL: https://github.com/apache/flink/pull/13547#issuecomment-704265372


   
   ## CI report:
   
   * 4239645b3f019ae4059f181c470bf9aa916f61e7 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7274)
 
   * 268d260399e598943726839a254f004a61f5ba66 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

flinkbot edited a comment on pull request #13535:
URL: https://github.com/apache/flink/pull/13535#issuecomment-702896138


   
   ## CI report:
   
   * bd5834786fbff909ae1a20c5045c0750f63adc4e Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7177)
 
   * 97e855d306d15d90259b65d3db38b7a19c62c6e9 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7271)
 
   * 4ed984adbb753d9287a4d5d71d8f2d149f86bba0 UNKNOWN
   * 505ec8554bfb64d4f53a9f619ffc0729e3e645f3 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7273)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (FLINK-10653) Introduce Pluggable Shuffle Service Architecture

[Till Rohrmann (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-10653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209682#comment-17209682
 ] 

Till Rohrmann commented on FLINK-10653:
---

[~zjwang] should we close this issue and make the open issues follow up issues?

> Introduce Pluggable Shuffle Service Architecture
> 
>
> Key: FLINK-10653
> URL: https://issues.apache.org/jira/browse/FLINK-10653
> Project: Flink
>  Issue Type: New Feature
>  Components: Runtime / Network
>Reporter: Zhijiang
>Assignee: Zhijiang
>Priority: Major
>
> This is the umbrella issue for improving shuffle architecture.
> Shuffle is the process of data transfer between stages, which involves in 
> writing outputs on sender side and reading data on receiver side. In flink 
> implementation, it covers three parts of writer, transport layer and reader 
> separately which are uniformed for both streaming and batch jobs.
> In detail, the current ResultPartitionWriter interface on upstream side only 
> supports in-memory outputs for streaming job and local persistent file 
> outputs for batch job. If we extend to implement another writer such as 
> DfsWriter, RdmaWriter, SortMergeWriter, etc based on ResultPartitionWriter 
> interface, it has not the unified mechanism to extend the reader side 
> accordingly. 
> In order to make the shuffle architecture more flexible and support more 
> scenarios especially for batch jobs, a high level shuffle architecture is 
> necessary to manage and extend both writer and reader sides together.
> Refer to the design doc for more details.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot edited a comment on pull request #13547: [FLINK-14406][runtime] Exposes managed memory usage through the REST API

[GitBox]

flinkbot edited a comment on pull request #13547:
URL: https://github.com/apache/flink/pull/13547#issuecomment-704265372


   
   ## CI report:
   
   * 20c70c0fec1ceb77e60e138bb334bb48f5bcadcf Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7238)
 
   * 4239645b3f019ae4059f181c470bf9aa916f61e7 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] sjwiesman commented on a change in pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

sjwiesman commented on a change in pull request #13535:
URL: https://github.com/apache/flink/pull/13535#discussion_r501158416



##
File path: 
flink-table/flink-table-api-java-bridge/src/main/java/org/apache/flink/table/factories/datagen/types/DecimalDataRandomGenerator.java
##
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.factories.datagen.types;
+
+import org.apache.flink.api.common.functions.RuntimeContext;
+import org.apache.flink.runtime.state.FunctionInitializationContext;
+import org.apache.flink.streaming.api.functions.source.datagen.DataGenerator;
+import org.apache.flink.table.data.DecimalData;
+
+import java.math.BigDecimal;
+import java.math.MathContext;
+import java.math.RoundingMode;
+import java.util.concurrent.ThreadLocalRandom;
+
+/**
+ * Generates random {@link DecimalData} values.
+ */
+public class DecimalDataRandomGenerator implements DataGenerator {
+
+   private final int precision;
+
+   private final int scale;
+
+   private final double min;
+
+   private final double max;
+
+   private final String fmt;
+
+   public DecimalDataRandomGenerator(int precision, int scale, double min, 
double max) {
+   double largest = largestValue(precision, scale);
+   this.precision = precision;
+   this.scale = scale;
+   this.min = Math.min(-1 * largest, min);
+   this.max = Math.min(largest, max);
+   this.fmt = "%" + precision + "." + scale + "f";
+   }
+
+   @Override
+   public void open(String name, FunctionInitializationContext context, 
RuntimeContext runtimeContext) throws Exception {
+   }
+
+   @Override
+   public boolean hasNext() {
+   return true;
+   }
+
+   @Override
+   public DecimalData next() {
+   String value = String.format(fmt, 
ThreadLocalRandom.current().nextDouble(min, max));
+
+   BigDecimal decimal = new BigDecimal(value.trim(), new 
MathContext(precision, RoundingMode.DOWN));

Review comment:
   yes





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] knaufk commented on a change in pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

knaufk commented on a change in pull request #13535:
URL: https://github.com/apache/flink/pull/13535#discussion_r501146816



##
File path: 
flink-table/flink-table-api-java-bridge/src/main/java/org/apache/flink/table/factories/datagen/types/DecimalDataRandomGenerator.java
##
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.factories.datagen.types;
+
+import org.apache.flink.api.common.functions.RuntimeContext;
+import org.apache.flink.runtime.state.FunctionInitializationContext;
+import org.apache.flink.streaming.api.functions.source.datagen.DataGenerator;
+import org.apache.flink.table.data.DecimalData;
+
+import java.math.BigDecimal;
+import java.math.MathContext;
+import java.math.RoundingMode;
+import java.util.concurrent.ThreadLocalRandom;
+
+/**
+ * Generates random {@link DecimalData} values.
+ */
+public class DecimalDataRandomGenerator implements DataGenerator {
+
+   private final int precision;
+
+   private final int scale;
+
+   private final double min;
+
+   private final double max;
+
+   private final String fmt;
+
+   public DecimalDataRandomGenerator(int precision, int scale, double min, 
double max) {
+   double largest = largestValue(precision, scale);
+   this.precision = precision;
+   this.scale = scale;
+   this.min = Math.min(-1 * largest, min);
+   this.max = Math.min(largest, max);
+   this.fmt = "%" + precision + "." + scale + "f";
+   }
+
+   @Override
+   public void open(String name, FunctionInitializationContext context, 
RuntimeContext runtimeContext) throws Exception {
+   }
+
+   @Override
+   public boolean hasNext() {
+   return true;
+   }
+
+   @Override
+   public DecimalData next() {
+   String value = String.format(fmt, 
ThreadLocalRandom.current().nextDouble(min, max));
+
+   BigDecimal decimal = new BigDecimal(value.trim(), new 
MathContext(precision, RoundingMode.DOWN));

Review comment:
   ```suggestion
BigDecimal decimal = new 
BigDecimal(ThreadLocalRandom.current().nextDouble(min, max), new 
MathContext(precision, RoundingMode.DOWN));
   ```

##
File path: 
flink-table/flink-table-api-java-bridge/src/main/java/org/apache/flink/table/factories/datagen/types/DecimalDataRandomGenerator.java
##
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.factories.datagen.types;
+
+import org.apache.flink.api.common.functions.RuntimeContext;
+import org.apache.flink.runtime.state.FunctionInitializationContext;
+import org.apache.flink.streaming.api.functions.source.datagen.DataGenerator;
+import org.apache.flink.table.data.DecimalData;
+
+import java.math.BigDecimal;
+import java.math.MathContext;
+import java.math.RoundingMode;
+import java.util.concurrent.ThreadLocalRandom;
+
+/**
+ * Generates random {@link DecimalData} values.
+ */
+public class DecimalDataRandomGenerator implements DataGenerator {
+
+   private final int precision;
+
+   private final int scale;
+
+   private final double min;
+
+   private final double max;
+
+   private final String fmt;
+
+   public DecimalDataRandomGenerator(int precision, int scale, double min, 
double max) {
+   double largest = largestValue(precision, 

[jira] [Commented] (FLINK-19497) Implement mutator methods for FlinkCounterWrapper

[Richard Moorhead (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209662#comment-17209662
 ] 

Richard Moorhead commented on FLINK-19497:
--

I see November as the timeframe for 1.12; in your estimation, would the patch 
release occur before then?

> Implement mutator methods for FlinkCounterWrapper
> -
>
> Key: FLINK-19497
> URL: https://issues.apache.org/jira/browse/FLINK-19497
> Project: Flink
>  Issue Type: Improvement
>  Components: Runtime / Metrics
>Affects Versions: 1.10.2, 1.11.2
>Reporter: Richard Moorhead
>Assignee: Richard Moorhead
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.12.0
>
>
> Looking at the dropwizard wrapper classes in flink-metrics-dropwizard, it 
> appears that all of them have mutator methods defined with the exception of 
> FlinkCounterWrapper. We have a use case wherein we mutate counters from a 
> dropwizard context but wish the underlying metrics in the flink registry to 
> be updated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot edited a comment on pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

flinkbot edited a comment on pull request #13535:
URL: https://github.com/apache/flink/pull/13535#issuecomment-702896138


   
   ## CI report:
   
   * bd5834786fbff909ae1a20c5045c0750f63adc4e Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7177)
 
   * 97e855d306d15d90259b65d3db38b7a19c62c6e9 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7271)
 
   * 4ed984adbb753d9287a4d5d71d8f2d149f86bba0 UNKNOWN
   * 505ec8554bfb64d4f53a9f619ffc0729e3e645f3 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13552: [FLINK-19482][format-orc] Define serialVersionUID for OrcRowInputFormat

[GitBox]

flinkbot edited a comment on pull request #13552:
URL: https://github.com/apache/flink/pull/13552#issuecomment-704894754


   
   ## CI report:
   
   * c13db0be1302b5b60d13992ae5960422604e1299 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7263)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (FLINK-19517) Support for Confluent Kafka of Table Creation in Flink SQL Client

[Jark Wu (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209658#comment-17209658
 ] 

Jark Wu commented on FLINK-19517:
-

Hi [~ksp0422], never mind. It's not easy to track the development progress :)

I think you can set enable.idempotence and transactional.id, but note these 
properties are only work for kafka sink.

Yes. We should add the missing documentation for the "properties.*", there is 
already an issue for this, FLINK-18768.

> Support for Confluent Kafka of Table Creation in Flink SQL Client
> -
>
> Key: FLINK-19517
> URL: https://issues.apache.org/jira/browse/FLINK-19517
> Project: Flink
>  Issue Type: Wish
>  Components: Connectors / Kafka, Table SQL / Ecosystem
>Affects Versions: 1.12.0
>Reporter: Kevin Kwon
>Priority: Major
>
> Currently, table creation from SQL client such as below works well
> {code:sql}
> CREATE TABLE kafkaTable (
>   user_id BIGINT,
>   item_id BIGINT,
>   category_id BIGINT,
>   behavior STRING,
>   ts TIMESTAMP(3)
> ) WITH (
>   'connector' = 'kafka',
>   'topic' = 'user_behavior',
>   'properties.bootstrap.servers' = 'localhost:9092',
>   'properties.group.id' = 'testGroup',
>   'format' = 'avro',
>   'scan.startup.mode' = 'earliest-offset'
> )
> {code}
> Although I would wish for the table creation to support Confluent Kafka 
> configuration as well. For example something like
> {code:sql}
> CREATE TABLE kafkaTable (
>   user_id BIGINT,
>   item_id BIGINT,
>   category_id BIGINT,
>   behavior STRING,
>   ts TIMESTAMP(3)
> ) WITH (
>   'connector' = 'confluent-kafka',
>   'topic' = 'user_behavior',
>   'properties.bootstrap.servers' = 'localhost:9092',
>   'properties.group.id' = 'testGroup',
>   'schema-registry' = 'http://schema-registry.com',
>   'scan.startup.mode' = 'earliest-offset'
> )
> {code}
> If this is enabled, it will be much more convenient to test queries 
> on-the-fly that business analysts want to test against with 'Confluent Kafka'
> Additionally, it will be better if we can
>  - specify 'parallelism' within WITH clause to support parallel partition 
> processing
>  - specify custom properties within WITH clause specified in 
> [https://docs.confluent.io/5.4.2/installation/configuration/consumer-configs.html]
>  - have remote access to SQL client in cluster from local environment



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-19522) Add ability to set auto commit on jdbc driver from Table/SQL API

[Dylan Forciea (Jira)]

Dylan Forciea created FLINK-19522:
-

 Summary: Add ability to set auto commit on jdbc driver from 
Table/SQL API
 Key: FLINK-19522
 URL: https://issues.apache.org/jira/browse/FLINK-19522
 Project: Flink
  Issue Type: Improvement
  Components: Connectors / JDBC
Affects Versions: 1.11.2
Reporter: Dylan Forciea
 Attachments: Screen Shot 2020-10-01 at 5.03.24 PM.png, Screen Shot 
2020-10-01 at 5.03.31 PM.png

When I tried to stream data from postgres via the JDBC source connector in the 
SQL api, it was loading the entirety of the table into memory before starting 
streaming. This is due to the postgres JDBC driver requiring the autoCommit 
flag to be set to true for streaming to take place.

FLINK-12198 provided the means to do this with the JDBCInputSource, but this 
did not extend to the SQL description. This option should be added.

To reproduce, create a very large table and try to read it in with the SQL api. 
You will see a large spike of memory usage and no data streaming, and then it 
will start all at once. I will attach a couple of graphs before and after I 
made a patch to the code myself to set auto-commit.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-18768) Imporve SQL kafka connector docs about passing kafka properties

[Jark Wu (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-18768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jark Wu updated FLINK-18768:

Component/s: Table SQL / Ecosystem

> Imporve SQL kafka connector docs about passing kafka properties
> ---
>
> Key: FLINK-18768
> URL: https://issues.apache.org/jira/browse/FLINK-18768
> Project: Flink
>  Issue Type: Improvement
>  Components: Connectors / Kafka, Documentation, Table SQL / Ecosystem
>Reporter: Leonard Xu
>Priority: Major
>  Labels: starter
>
> SQL kafka connector support passthrough kafka properties 
> like：'message.max.bytes', 'replica.fetch.max.bytes' and other kafka 
> properties, but current docs and test missed this case.
> I think we can add some test and explanation  in docs. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot edited a comment on pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

flinkbot edited a comment on pull request #13535:
URL: https://github.com/apache/flink/pull/13535#issuecomment-702896138


   
   ## CI report:
   
   * bd5834786fbff909ae1a20c5045c0750f63adc4e Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7177)
 
   * 97e855d306d15d90259b65d3db38b7a19c62c6e9 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7271)
 
   * 4ed984adbb753d9287a4d5d71d8f2d149f86bba0 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13557: [FLINK-19516] Let MiniClusterJobClient.jobResultFuture complete after MiniCluster shut down

[GitBox]

flinkbot edited a comment on pull request #13557:
URL: https://github.com/apache/flink/pull/13557#issuecomment-705029634


   
   ## CI report:
   
   * c9fbc4b0d96ce887beebd324ebf5c3fdd6736a8c Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7272)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #13557: [FLINK-19516] Let MiniClusterJobClient.jobResultFuture complete after MiniCluster shut down

[GitBox]

flinkbot commented on pull request #13557:
URL: https://github.com/apache/flink/pull/13557#issuecomment-705029634


   
   ## CI report:
   
   * c9fbc4b0d96ce887beebd324ebf5c3fdd6736a8c UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

flinkbot edited a comment on pull request #13535:
URL: https://github.com/apache/flink/pull/13535#issuecomment-702896138


   
   ## CI report:
   
   * bd5834786fbff909ae1a20c5045c0750f63adc4e Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7177)
 
   * 97e855d306d15d90259b65d3db38b7a19c62c6e9 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7271)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] HuangZhenQiu commented on pull request #8952: [FLINK-10868][flink-yarn] Add failure rater for resource manager

[GitBox]

HuangZhenQiu commented on pull request #8952:
URL: https://github.com/apache/flink/pull/8952#issuecomment-705022743


   > @HuangZhenQiu
   > I am closing this PR for now. Please, reopen it if you decide to work on 
it again.
   
   @azagrebin 
   Sorry for late reply. I will start to work on it after flink forward. Would 
you please reopen it for me?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (FLINK-19491) AvroSerializerSnapshot cannot handle large schema

[Lian Jiang (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209632#comment-17209632
 ] 

Lian Jiang commented on FLINK-19491:


Is it possible to support large schemas (>64KB) by having a new design working 
around the string/UTF 64KB limit? Thanks.

> AvroSerializerSnapshot cannot handle large schema
> -
>
> Key: FLINK-19491
> URL: https://issues.apache.org/jira/browse/FLINK-19491
> Project: Flink
>  Issue Type: Bug
>  Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
>Affects Versions: 1.10.2, 1.12.0, 1.11.2
>Reporter: Arvid Heise
>Priority: Major
>
> Flink can only handle schemas up to a size of 64kb.
>  
> {noformat}
> Caused by: java.io.UTFDataFormatException: encoded string too long: 223502 
> bytes
>   at java.io.DataOutputStream.writeUTF(DataOutputStream.java:364)
>   at java.io.DataOutputStream.writeUTF(DataOutputStream.java:323)
>   at 
> org.apache.flink.formats.avro.typeutils.AvroSerializerSnapshot.writeSnapshot(AvroSerializerSnapshot.java:75)
>   at 
> org.apache.flink.api.common.typeutils.TypeSerializerSnapshot.writeVersionedSnapshot(TypeSerializerSnapshot.java:153)
>   at 
> org.apache.flink.api.common.typeutils.NestedSerializersSnapshotDelegate.writeNestedSerializerSnapshots(NestedSerializersSnapshotDelegate.java:159)
>   at 
> org.apache.flink.api.common.typeutils.CompositeTypeSerializerSnapshot.writeSnapshot(CompositeTypeSerializerSnapshot.java:148)
>   at 
> org.apache.flink.api.common.typeutils.TypeSerializerSnapshot.writeVersionedSnapshot(TypeSerializerSnapshot.java:153)
>   at 
> org.apache.flink.api.common.typeutils.TypeSerializerSnapshotSerializationUtil$TypeSerializerSnapshotSerializationProxy.write(TypeSerializerSnapshotSerializationUtil.java:138)
>   at 
> org.apache.flink.api.common.typeutils.TypeSerializerSnapshotSerializationUtil.writeSerializerSnapshot(TypeSerializerSnapshotSerializationUtil.java:55)
>   at 
> org.apache.flink.runtime.state.metainfo.StateMetaInfoSnapshotReadersWriters$CurrentWriterImpl.writeStateMetaInfoSnapshot(StateMetaInfoSnapshotReadersWriters.java:183)
>   at 
> org.apache.flink.runtime.state.KeyedBackendSerializationProxy.write(KeyedBackendSerializationProxy.java:126)
>   at 
> org.apache.flink.runtime.state.heap.HeapSnapshotStrategy$1.callInternal(HeapSnapshotStrategy.java:171)
>   at 
> org.apache.flink.runtime.state.heap.HeapSnapshotStrategy$1.callInternal(HeapSnapshotStrategy.java:158)
>   at 
> org.apache.flink.runtime.state.AsyncSnapshotCallable.call(AsyncSnapshotCallable.java:75)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.flink.runtime.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:510)
>   ... 5 common frames omitted{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot edited a comment on pull request #13535: [FLINK-19496][table] DataGen source DECIMAL always returns null

[GitBox]

flinkbot edited a comment on pull request #13535:
URL: https://github.com/apache/flink/pull/13535#issuecomment-702896138


   
   ## CI report:
   
   * bd5834786fbff909ae1a20c5045c0750f63adc4e Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7177)
 
   * 97e855d306d15d90259b65d3db38b7a19c62c6e9 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #13557: [FLINK-19516] Let MiniClusterJobClient.jobResultFuture complete after MiniCluster shut down

[GitBox]

flinkbot commented on pull request #13557:
URL: https://github.com/apache/flink/pull/13557#issuecomment-705018818


   Thanks a lot for your contribution to the Apache Flink project. I'm the 
@flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress 
of the review.
   
   
   ## Automated Checks
   Last check on commit c9fbc4b0d96ce887beebd324ebf5c3fdd6736a8c (Wed Oct 07 
15:35:40 UTC 2020)
   
   **Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up 
to date!
   
   
   Mention the bot in a comment to re-run the automated checks.
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review 
Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full 
explanation of the review process.
The Bot is tracking the review progress through labels. Labels are applied 
according to the order of the review items. For consensus, approval by a Flink 
committer of PMC member is required Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot approve description` to approve one or more aspects (aspects: 
`description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until 
`architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's 
attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13553: [FLINK-19306][coordination] Add DeclarativeSlotManager

[GitBox]

flinkbot edited a comment on pull request #13553:
URL: https://github.com/apache/flink/pull/13553#issuecomment-704936985


   
   ## CI report:
   
   * 15d4c353c4c0a646b2bdf34bbedbd81359cb5b5f Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7264)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (FLINK-19516) PerJobMiniClusterFactoryTest.testJobClient()

[Till Rohrmann (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209626#comment-17209626
 ] 

Till Rohrmann commented on FLINK-19516:
---

[~sewen] if you post the link to the Azure pipeline run and not the directly 
logs, then it is a tad easier to retrieve the uploaded logs which are stored on 
Azure.

> PerJobMiniClusterFactoryTest.testJobClient()
> 
>
> Key: FLINK-19516
> URL: https://issues.apache.org/jira/browse/FLINK-19516
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Coordination
>Affects Versions: 1.12.0
>Reporter: Stephan Ewen
>Assignee: Till Rohrmann
>Priority: Critical
>  Labels: pull-request-available, test-stability
>
> *Log:*
> https://dev.azure.com/sewen0794/19b23adf-d190-4fb4-ae6e-2e92b08923a3/_apis/build/builds/151/logs/137
> *Exception:*
> {code}
> [ERROR] 
> testJobClient(org.apache.flink.client.program.PerJobMiniClusterFactoryTest)  
> Time elapsed: 0.392 s  <<< FAILURE!
> java.lang.AssertionError: 
> Expected: is 
>  but: was 
>   at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
>   at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
>   at 
> org.apache.flink.client.program.PerJobMiniClusterFactoryTest.assertThatMiniClusterIsShutdown(PerJobMiniClusterFactoryTest.java:161)
>   at 
> org.apache.flink.client.program.PerJobMiniClusterFactoryTest.testJobClient(PerJobMiniClusterFactoryTest.java:93)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-19516) PerJobMiniClusterFactoryTest.testJobClient()

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-19516:
---
Labels: pull-request-available test-stability  (was: test-stability)

> PerJobMiniClusterFactoryTest.testJobClient()
> 
>
> Key: FLINK-19516
> URL: https://issues.apache.org/jira/browse/FLINK-19516
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Coordination
>Affects Versions: 1.12.0
>Reporter: Stephan Ewen
>Assignee: Till Rohrmann
>Priority: Critical
>  Labels: pull-request-available, test-stability
>
> *Log:*
> https://dev.azure.com/sewen0794/19b23adf-d190-4fb4-ae6e-2e92b08923a3/_apis/build/builds/151/logs/137
> *Exception:*
> {code}
> [ERROR] 
> testJobClient(org.apache.flink.client.program.PerJobMiniClusterFactoryTest)  
> Time elapsed: 0.392 s  <<< FAILURE!
> java.lang.AssertionError: 
> Expected: is 
>  but: was 
>   at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
>   at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
>   at 
> org.apache.flink.client.program.PerJobMiniClusterFactoryTest.assertThatMiniClusterIsShutdown(PerJobMiniClusterFactoryTest.java:161)
>   at 
> org.apache.flink.client.program.PerJobMiniClusterFactoryTest.testJobClient(PerJobMiniClusterFactoryTest.java:93)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] tillrohrmann opened a new pull request #13557: [FLINK-19516] Let MiniClusterJobClient.jobResultFuture complete after MiniCluster shut down

[GitBox]

tillrohrmann opened a new pull request #13557:
URL: https://github.com/apache/flink/pull/13557


   This commit changes the MiniClusterJobClient.jobResultFuture such that it 
completes after
   the MiniCluster's shut down has been triggered.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13556: [FLINK-19485] Decouple runtime operator implementation from DataStream operations

[GitBox]

flinkbot edited a comment on pull request #13556:
URL: https://github.com/apache/flink/pull/13556#issuecomment-704987982


   
   ## CI report:
   
   * 40f3f7d289665256ea5a8a938cbbf4ded64396f1 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7270)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

flinkbot edited a comment on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-704987370


   
   ## CI report:
   
   * 7d7d685a6d87ddc77424bf0e8e07ce491e717f0e Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7269)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13551: [FLINK-19520][configuration] Add randomization of checkpoint config.

[GitBox]

flinkbot edited a comment on pull request #13551:
URL: https://github.com/apache/flink/pull/13551#issuecomment-704858051


   
   ## CI report:
   
   * 5b037bbc8fd11fce426527e4caefccd075ddc4b6 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7262)
 
   * fe4a6228d31108e71a6e9faab97598ffa9a09b8b Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7268)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] dawidwys commented on a change in pull request #13529: [FLINK-19473] Implement multi inputs sorting DataInput

[GitBox]

dawidwys commented on a change in pull request #13529:
URL: https://github.com/apache/flink/pull/13529#discussion_r501079844



##
File path: 
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/io/StreamTwoInputProcessorFactory.java
##
@@ -40,38 +48,78 @@
  */
 public class StreamTwoInputProcessorFactory {
public static  StreamTwoInputProcessor create(
+   AbstractInvokable ownerTask,
CheckpointedInputGate[] checkpointedInputGates,
-   TypeSerializer inputSerializer1,
-   TypeSerializer inputSerializer2,
IOManager ioManager,
+   MemoryManager memoryManager,
TaskIOMetricGroup taskIOMetricGroup,
StreamStatusMaintainer streamStatusMaintainer,
TwoInputStreamOperator streamOperator,
TwoInputSelectionHandler inputSelectionHandler,
WatermarkGauge input1WatermarkGauge,
WatermarkGauge input2WatermarkGauge,
OperatorChain operatorChain,
+   StreamConfig streamConfig,
+   Configuration taskManagerConfig,
+   Configuration jobConfig,
+   ExecutionConfig executionConfig,
+   ClassLoader userClassloader,
Counter numRecordsIn) {
 
checkNotNull(operatorChain);
checkNotNull(inputSelectionHandler);
+
StreamStatusTracker statusTracker = new StreamStatusTracker();
taskIOMetricGroup.reuseRecordsInputCounter(numRecordsIn);
+   TypeSerializer typeSerializer1 = 
streamConfig.getTypeSerializerIn(0, userClassloader);
+   StreamTaskInput input1 = new StreamTaskNetworkInput<>(
+   checkpointedInputGates[0],
+   typeSerializer1,
+   ioManager,
+   new 
StatusWatermarkValve(checkpointedInputGates[0].getNumberOfInputChannels()),
+   0);
+   TypeSerializer typeSerializer2 = 
streamConfig.getTypeSerializerIn(1, userClassloader);
+   StreamTaskInput input2 = new StreamTaskNetworkInput<>(
+   checkpointedInputGates[1],
+   typeSerializer2,
+   ioManager,
+   new 
StatusWatermarkValve(checkpointedInputGates[1].getNumberOfInputChannels()),
+   1);
+
+   if (streamConfig.shouldSortInputs()) {
+   @SuppressWarnings("unchecked")
+   MultiInputSortingDataInputs multiInputs = new 
MultiInputSortingDataInputs(

Review comment:
   Yes, I must admit I was a bit harsh on this one :sweat_smile: 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] dawidwys commented on a change in pull request #13529: [FLINK-19473] Implement multi inputs sorting DataInput

[GitBox]

dawidwys commented on a change in pull request #13529:
URL: https://github.com/apache/flink/pull/13529#discussion_r501078332



##
File path: 
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/sort/MultiInputSortingDataInputs.java
##
@@ -0,0 +1,341 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.api.operators.sort;
+
+import org.apache.flink.api.common.typeutils.TypeComparator;
+import org.apache.flink.api.common.typeutils.TypeSerializer;
+import org.apache.flink.api.java.functions.KeySelector;
+import org.apache.flink.api.java.tuple.Tuple2;
+import org.apache.flink.configuration.AlgorithmOptions;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.core.io.InputStatus;
+import org.apache.flink.core.memory.DataOutputSerializer;
+import org.apache.flink.runtime.checkpoint.channel.ChannelStateWriter;
+import org.apache.flink.runtime.io.disk.iomanager.IOManager;
+import org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable;
+import org.apache.flink.runtime.memory.MemoryAllocationException;
+import org.apache.flink.runtime.memory.MemoryManager;
+import org.apache.flink.runtime.operators.sort.ExternalSorter;
+import org.apache.flink.runtime.operators.sort.PushSorter;
+import org.apache.flink.streaming.api.watermark.Watermark;
+import org.apache.flink.streaming.runtime.io.PushingAsyncDataInput;
+import org.apache.flink.streaming.runtime.io.StreamTaskInput;
+import org.apache.flink.streaming.runtime.streamrecord.LatencyMarker;
+import org.apache.flink.streaming.runtime.streamrecord.StreamRecord;
+import org.apache.flink.streaming.runtime.streamstatus.StreamStatus;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.MutableObjectIterator;
+
+import javax.annotation.Nonnull;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.PriorityQueue;
+import java.util.concurrent.CompletableFuture;
+
+/**
+ * An entry class for creating coupled, sorting inputs. The inputs are sorted 
independently and afterwards
+ * the inputs are being merged in order. It is done, by reporting availability 
only for the input which is
+ * current head for the sorted inputs.
+ */
+public class MultiInputSortingDataInputs {

Review comment:
   It is. It is used in the 
`StreamTwoInputProcessorFactory/StreamMultipleInputProcessorFactory` to wrap 
inputs into sorted inputs.
   
   Not in this commit though, but in subsequent ones.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Assigned] (FLINK-19516) PerJobMiniClusterFactoryTest.testJobClient()

[Till Rohrmann (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Till Rohrmann reassigned FLINK-19516:
-

Assignee: Till Rohrmann

> PerJobMiniClusterFactoryTest.testJobClient()
> 
>
> Key: FLINK-19516
> URL: https://issues.apache.org/jira/browse/FLINK-19516
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Coordination
>Affects Versions: 1.12.0
>Reporter: Stephan Ewen
>Assignee: Till Rohrmann
>Priority: Critical
>  Labels: test-stability
>
> *Log:*
> https://dev.azure.com/sewen0794/19b23adf-d190-4fb4-ae6e-2e92b08923a3/_apis/build/builds/151/logs/137
> *Exception:*
> {code}
> [ERROR] 
> testJobClient(org.apache.flink.client.program.PerJobMiniClusterFactoryTest)  
> Time elapsed: 0.392 s  <<< FAILURE!
> java.lang.AssertionError: 
> Expected: is 
>  but: was 
>   at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
>   at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
>   at 
> org.apache.flink.client.program.PerJobMiniClusterFactoryTest.assertThatMiniClusterIsShutdown(PerJobMiniClusterFactoryTest.java:161)
>   at 
> org.apache.flink.client.program.PerJobMiniClusterFactoryTest.testJobClient(PerJobMiniClusterFactoryTest.java:93)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] dawidwys commented on a change in pull request #13529: [FLINK-19473] Implement multi inputs sorting DataInput

[GitBox]

dawidwys commented on a change in pull request #13529:
URL: https://github.com/apache/flink/pull/13529#discussion_r501078332



##
File path: 
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/sort/MultiInputSortingDataInputs.java
##
@@ -0,0 +1,341 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.api.operators.sort;
+
+import org.apache.flink.api.common.typeutils.TypeComparator;
+import org.apache.flink.api.common.typeutils.TypeSerializer;
+import org.apache.flink.api.java.functions.KeySelector;
+import org.apache.flink.api.java.tuple.Tuple2;
+import org.apache.flink.configuration.AlgorithmOptions;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.core.io.InputStatus;
+import org.apache.flink.core.memory.DataOutputSerializer;
+import org.apache.flink.runtime.checkpoint.channel.ChannelStateWriter;
+import org.apache.flink.runtime.io.disk.iomanager.IOManager;
+import org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable;
+import org.apache.flink.runtime.memory.MemoryAllocationException;
+import org.apache.flink.runtime.memory.MemoryManager;
+import org.apache.flink.runtime.operators.sort.ExternalSorter;
+import org.apache.flink.runtime.operators.sort.PushSorter;
+import org.apache.flink.streaming.api.watermark.Watermark;
+import org.apache.flink.streaming.runtime.io.PushingAsyncDataInput;
+import org.apache.flink.streaming.runtime.io.StreamTaskInput;
+import org.apache.flink.streaming.runtime.streamrecord.LatencyMarker;
+import org.apache.flink.streaming.runtime.streamrecord.StreamRecord;
+import org.apache.flink.streaming.runtime.streamstatus.StreamStatus;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.MutableObjectIterator;
+
+import javax.annotation.Nonnull;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.PriorityQueue;
+import java.util.concurrent.CompletableFuture;
+
+/**
+ * An entry class for creating coupled, sorting inputs. The inputs are sorted 
independently and afterwards
+ * the inputs are being merged in order. It is done, by reporting availability 
only for the input which is
+ * current head for the sorted inputs.
+ */
+public class MultiInputSortingDataInputs {

Review comment:
   It is. It is used in the 
`StreamTwoInputProcessorFactory/StreamMultipleInputProcessorFactory` to wrap 
inputs into sorted inputs.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (FLINK-19516) PerJobMiniClusterFactoryTest.testJobClient()

[Till Rohrmann (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209599#comment-17209599
 ] 

Till Rohrmann commented on FLINK-19516:
---

I believe that this problem has been introduced with FLINK-19123 because we no 
longer chain the result future after the callback which triggers the shutdown 
of the {{MiniCluster}}.

> PerJobMiniClusterFactoryTest.testJobClient()
> 
>
> Key: FLINK-19516
> URL: https://issues.apache.org/jira/browse/FLINK-19516
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Coordination
>Affects Versions: 1.12.0
>Reporter: Stephan Ewen
>Priority: Critical
>  Labels: test-stability
>
> *Log:*
> https://dev.azure.com/sewen0794/19b23adf-d190-4fb4-ae6e-2e92b08923a3/_apis/build/builds/151/logs/137
> *Exception:*
> {code}
> [ERROR] 
> testJobClient(org.apache.flink.client.program.PerJobMiniClusterFactoryTest)  
> Time elapsed: 0.392 s  <<< FAILURE!
> java.lang.AssertionError: 
> Expected: is 
>  but: was 
>   at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
>   at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
>   at 
> org.apache.flink.client.program.PerJobMiniClusterFactoryTest.assertThatMiniClusterIsShutdown(PerJobMiniClusterFactoryTest.java:161)
>   at 
> org.apache.flink.client.program.PerJobMiniClusterFactoryTest.testJobClient(PerJobMiniClusterFactoryTest.java:93)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] kl0u commented on pull request #13554: [FLINK-19493] In CliFrontend, make flow of Configuration more obvious

[GitBox]

kl0u commented on pull request #13554:
URL: https://github.com/apache/flink/pull/13554#issuecomment-704990097


   Will have a look in a bit.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] dawidwys commented on a change in pull request #13529: [FLINK-19473] Implement multi inputs sorting DataInput

[GitBox]

dawidwys commented on a change in pull request #13529:
URL: https://github.com/apache/flink/pull/13529#discussion_r501077259



##
File path: 
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/EndOfInputAware.java
##
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.api.operators;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.streaming.runtime.io.StreamOneInputProcessor;
+import org.apache.flink.streaming.runtime.tasks.OperatorChain;
+
+/**
+ * An interface for {@link OperatorChain} to extract the feature of ending the 
input.
+ *
+ * It's purpose is mainly to make it easier to instantiate {@link 
StreamOneInputProcessor} which needs to
+ * notify the chain that an input has ended.
+ */
+@Internal
+public interface EndOfInputAware {
+
+   /**
+* Ends the main operator input specified by {@code inputId}).
+*
+* @param inputId the input ID starts from 1 which indicates the first 
input.
+*/
+   void endMainOperatorInput(int inputId) throws Exception;
+}

Review comment:
   Ok, I will reuse the `BoundedMultiInput`, even though I am only like 90% 
convinced, because I am not a fan of reusing a single interface for multiple 
purposes. In this case one can argue though `OperatorChain` is an `Operator` 
and thus e.g. javadoc still makes sense.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #13556: [FLINK-19485] Decouple runtime operator implementation from DataStream operations

[GitBox]

flinkbot commented on pull request #13556:
URL: https://github.com/apache/flink/pull/13556#issuecomment-704987982


   
   ## CI report:
   
   * 40f3f7d289665256ea5a8a938cbbf4ded64396f1 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

flinkbot commented on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-704987370


   
   ## CI report:
   
   * 7d7d685a6d87ddc77424bf0e8e07ce491e717f0e UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] scottxusayhi commented on pull request #13555: [FLINK-17613][e2e][k8s] Test against multiple k8s versions

[GitBox]

scottxusayhi commented on pull request #13555:
URL: https://github.com/apache/flink/pull/13555#issuecomment-704987085


   @wangyang0918 could you help to review?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #13551: [FLINK-19520][configuration] Add randomization of checkpoint config.

[GitBox]

flinkbot edited a comment on pull request #13551:
URL: https://github.com/apache/flink/pull/13551#issuecomment-704858051


   
   ## CI report:
   
   * 5b037bbc8fd11fce426527e4caefccd075ddc4b6 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7262)
 
   * fe4a6228d31108e71a6e9faab97598ffa9a09b8b UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (FLINK-19486) expected: class org.apache.flink.runtime.state.KeyGroupsStateHandle, but found: class org.apache.flink.runtime.state.IncrementalRemoteKeyedStateHandle

[Aljoscha Krettek (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209593#comment-17209593
 ] 

Aljoscha Krettek commented on FLINK-19486:
--

Exception message improved:
master: ab230345322a09669d54333cc9fa07af99492f21

> expected: class org.apache.flink.runtime.state.KeyGroupsStateHandle, but 
> found: class org.apache.flink.runtime.state.IncrementalRemoteKeyedStateHandle
> --
>
> Key: FLINK-19486
> URL: https://issues.apache.org/jira/browse/FLINK-19486
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Checkpointing, Runtime / State Backends
>Affects Versions: 1.11.1
>Reporter: appleyuchi
>Priority: Major
>  Labels: pull-request-available
> Attachments: pom.xml
>
>
> *The reason why I reopen it is that:*
>  
> *I can {color:#de350b}not{color} {color:#de350b}resume from incremental 
> checkpoint manually{color} when I stop the following program by force(maybe  
> NOT supported officially?).*
> ||My Experience||StateBackEnd||result||
> |Normal Checkpoint|FsStateBackend|succeed|
> |Incremental Checkpoint|RocksDBStateBackend|failed|
> ||My Other Effort||Content||Result||
> |①|*I {color:#de350b}hava subscribed to mailing list{color}*|*David Anderson 
> has answered this several days ago. But he did not tell me how I can fix it.*
> *I guess he's very busy.*
> |②|{color:#de350b}stackoverflow account{color}|my account is forbidden to ask 
> questions.
> |③|alibaba {color:#de350b}Flink dingtalk group{color}|no one can answer this 
> question
> |④|I have read 
> [document|https://ci.apache.org/projects/flink/flink-docs-stable/ops/state/checkpoints.html]
>  carefully|I can NOT succeed in it|
> |⑤|apologise for post it here|how can I make sure this question is a support 
> instead of a bug or due to my own mistake?
> no successful example in Google/Baidu,am I right?|
> |⑥|about less than 1000  people visit my [failure 
> record|https://yuchi.blog.csdn.net/article/details/108896441] about this 
> experiment.|in vain|
>  *Please help,thanks~*
>  
> -original posting 
> content-
> *I want to do an experiment of"incremental checkpoint"*
> ||information||content||
> |pom.xml|[ is in the attachment]
> |code| [https://paste.ubuntu.com/p/DpTyQKq6Vk/]
> |error|[https://paste.ubuntu.com/p/49HRYXFzR2/]
>  
> *some of the above error is:*
> *Caused by: java.lang.IllegalStateException: Unexpected state handle type, 
> expected: class org.apache.flink.runtime.state.KeyGroupsStateHandle, but 
> found: class org.apache.flink.runtime.state.IncrementalRemoteKeyedStateHandle*
>  
>  
> The steps for incremental experiment are:
> ||step||content||
> |①|mvn clean scala:compile compile package|
> |②|nc -lk |
> |③|flink run -c wordcount_increstate  datastream_api-1.0-SNAPSHOT.jar
> Job has been submitted with JobID df6d62a43620f258155b8538ef0ddf1b|
> |④|input the following conents in nc -lk 
> before
>  error
>  error
>  error
>  error
> |⑤|flink run -s 
> hdfs://Desktop:9000/tmp/flinkck/df6d62a43620f258155b8538ef0ddf1b/chk-22 -c 
> StateWordCount datastream_api-1.0-SNAPSHOT.jar
> Then the above error happens.
> Please help,Thanks~!
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot edited a comment on pull request #13529: [FLINK-19473] Implement multi inputs sorting DataInput

[GitBox]

flinkbot edited a comment on pull request #13529:
URL: https://github.com/apache/flink/pull/13529#issuecomment-702193043


   
   ## CI report:
   
   * 0adec04c920791b2e1dc0cfef185eb909eb616f7 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7260)
 
   * 0678c7621f0f261144a624044d7055a4687b98d2 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7266)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org