GitHub user uang520 opened a pull request:
https://github.com/apache/flink/pull/6410
Release 1.6
*Thank you very much for contributing to Apache Flink - we are happy that
you want to help us improve Flink. To help the community review your
contribution in the best possible way, please go through the checklist below,
which will get the contribution into a shape in which it can be best reviewed.*
*Please understand that we do not do this to make contributions to Flink a
hassle. In order to uphold a high standard of quality for code contributions,
while at the same time managing a large number of contributions, we need
contributors to prepare the contributions well, and give reviewers enough
contextual information for the review. Please also understand that
contributions that do not follow this guide will take longer to review and thus
typically be picked up with lower priority by the community.*
## Contribution Checklist
- Make sure that the pull request corresponds to a [JIRA
issue](https://issues.apache.org/jira/projects/FLINK/issues). Exceptions are
made for typos in JavaDoc or documentation files, which need no JIRA issue.
- Name the pull request in the form "[FLINK-XXXX] [component] Title of
the pull request", where *FLINK-XXXX* should be replaced by the actual issue
number. Skip *component* if you are unsure about which is the best component.
Typo fixes that have no associated JIRA issue should be named following
this pattern: `[hotfix] [docs] Fix typo in event time introduction` or
`[hotfix] [javadocs] Expand JavaDoc for PuncuatedWatermarkGenerator`.
- Fill out the template below to describe the changes contributed by the
pull request. That will give reviewers the context they need to do the review.
- Make sure that the change passes the automated tests, i.e., `mvn clean
verify` passes. You can set up Travis CI to do that following [this
guide](http://flink.apache.org/contribute-code.html#best-practices).
- Each pull request should address only one issue, not mix up code from
multiple issues.
- Each commit in the pull request has a meaningful commit message
(including the JIRA id)
- Once all items of the checklist are addressed, remove the above text
and this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
## What is the purpose of the change
*(For example: This pull request makes task deployment go through the blob
server, rather than through RPC. That way we avoid re-transferring them on each
deployment (during recovery).)*
## Brief change log
*(for example:)*
- *The TaskInfo is stored in the blob store on job creation time as a
persistent artifact*
- *Deployments RPC transmits only the blob storage reference*
- *TaskManagers retrieve the TaskInfo from the blob cache*
## Verifying this change
*(Please pick either of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe
tests)*.
*(or)*
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end deployment with large payloads
(100MB)*
- *Extended integration test for recovery after master (JobManager)
failure*
- *Added test that validates that TaskInfo is transferred only once
across recoveries*
- *Manually verified the change by running a 4 node cluser with 2
JobManagers and 4 TaskManagers, a stateful streaming program, and killing one
JobManager and two TaskManagers during the execution, verifying that recovery
happens correctly.*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (yes / no)
- The serializers: (yes / no / don't know)
- The runtime per-record code paths (performance sensitive): (yes / no /
don't know)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know)
- The S3 file system connector: (yes / no / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs /
JavaDocs / not documented)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/flink release-1.6
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6410.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6410
----
commit 8005a2ebac1afbec6fcf43a4442f51f442f33590
Author: Lakshmi Gururaja Rao <glaksh100@...>
Date: 2018-07-10T18:40:02Z
[FLINK-9692] [kinesis] Adaptive reads from Kinesis
commit c3d3ff37de47e9302ba2a7397bef7933fab82f81
Author: Stephan Ewen <sewen@...>
Date: 2018-07-17T08:16:50Z
[FLINK-9692] [kinesis] Harmonize style of config variable names
commit ffeb978260c0ce770911183ff042e502724a3fd3
Author: Aljoscha Krettek <aljoscha.krettek@...>
Date: 2018-07-16T13:58:07Z
[FLINK-9857] Delay firing of processing-time timers by 1 ms
commit 0785edecb3a1c6f32a71fe4c49923381687db66f
Author: kkloudas <kkloudas@...>
Date: 2018-07-17T21:03:57Z
[FLINK-9880] Fix order in BucketerContext#update.
commit 343614cd8e73f2d10ba5cb99759b022dd829d03f
Author: yanghua <yanghua1127@...>
Date: 2018-07-10T15:16:12Z
[FLINK-9777] YARN: JM and TM Memory must be specified with Units
This closes #6297
commit 4022063ae26cafb5b5d2edf03a74d26f6daff3b1
Author: zentol <chesnay@...>
Date: 2018-07-11T16:37:15Z
[hotfix][rest] Simplify dispatcher host retrieval
Adjusted to work like the JobSubmitHandler.
commit 1386f91b0ad50ee4900e221581f19608d7217994
Author: zentol <chesnay@...>
Date: 2018-07-11T16:41:06Z
[FLINK-9811][test] Add test for jar handler interactions
This closes #6311.
commit 7e2ac0ac11ba69d9cac6dbdad6910bceb42af77d
Author: Till Rohrmann <trohrmann@...>
Date: 2018-07-18T07:48:49Z
[hotfix] Harden JarUploadHandlerTest#testUploadJar
Use lastIndexOf("_") instead of indexOf("_") in order to find the file name
part
commit ff2e7d59455ba0d3e01a2b48c186e0216bf8fdfc
Author: Till Rohrmann <trohrmann@...>
Date: 2018-07-18T09:11:25Z
[hotfix] Properly stop Flink Yarn application in
YARNSessionFIFOITCase#testJavaAPI
Calls ClusterClient#shutDownCluster in order to stop the Flink Yarn
application
commit 8a247605bf41738900a4c8f6fa7a5949b7185e6d
Author: linjun <1428117789@...>
Date: 2018-07-10T14:55:11Z
[FLINK-9793][yarn] Fix flink-dist detection to prevent duplicate upload
This closes #6296.
commit 969ab84777437683b2b244db96d8e1436275427c
Author: Dawid Wysakowicz <dwysakowicz@...>
Date: 2018-07-16T15:30:17Z
[FLINK-9866] Allow passing command line arguments to standalone job
This closes #6344
commit f576381e5fb45bf3cb4aaa54ba13ae11b4df3ca1
Author: Dawid Wysakowicz <dwysakowicz@...>
Date: 2018-07-11T07:52:48Z
[FLINK-9792] Added custom Description class for ConfigOptions to enable
rich formatting.
This closes #6312
commit 579d155a7b2dc6bdd0fc1f2f95529d4c875c970a
Author: Dawid Wysakowicz <dwysakowicz@...>
Date: 2018-07-18T13:17:32Z
[FLINK-9866][hotfix] Pass program arguments last to standalone-job.sh
commit b9fe077d221bdb013ed57f2555405c9fe4a96aa1
Author: Wosin <blind.van@...>
Date: 2018-07-04T08:27:54Z
[FLINK-9575] Remove job-related BLOBS only if the job was removed
sucessfully
This closes #6322.
commit e2b4ffc016da822dda544b31fb3caf679f80a9d9
Author: Till Rohrmann <trohrmann@...>
Date: 2018-07-18T11:49:09Z
[FLINK-9575][tests] Simplify
DispatcherTest#testBlobsAreRemovedOnlyIfJobIsRemovedProperly
Move DispatcherTest#testBlobsAreRemovedOnlyIfJobIsRemovedProperly into
DispatcherResourceCleanupTest
and split it up into a success and failure case.
Moreover, this commit changes the logic of blob cleanup to also cleanup
locally in case of a removal
failure of a job from a SubmittedJobGraphStore.
commit e87532e01ce180ab8f6666c91edc57814bd25b92
Author: zentol <chesnay@...>
Date: 2018-07-17T09:34:29Z
[FLINK-9872][tests] Properly cancel test job
This closes #6349.
commit 9ae2c3bacc4c25b813a897bd419226c5aaa21bd3
Author: zentol <chesnay@...>
Date: 2018-07-17T11:29:16Z
[FLINK-9815][yarn][tests] Harden tests against slow job shutdowns
This closes #6352.
commit 4cd682ded2a24a5a658eaab7b4bf9100b76a99f1
Author: zentol <chesnay@...>
Date: 2018-07-13T12:00:10Z
[FLINK-9499][rest] Support JSON request in JarHandlers
This closes #6330.
commit 71953b8160221ce22c87f030cfb54d72fba870a3
Author: xueyu <278006819@...>
Date: 2018-07-18T13:39:57Z
[hotfix] [sql-client] Fix typo in SqlExecutionException
This closes #6364.
commit 1e4cfc97bb1c02cd2ea5bd451d40cfb05b949166
Author: Timo Walther <twalthr@...>
Date: 2018-07-19T06:50:09Z
[hotfix] [docs] Update docs about streaming joins
commit 4b7eff7712ce009ef6641f94cf06848d83a2f2ec
Author: Ashwin Sinha <ashwin.sinha@...>
Date: 2018-07-17T20:49:20Z
[FLINK-9881] [table] Fix a typo in table.scala
This closes #6354.
commit 134bdd336334337e6b3e33cd60a5b080ba811777
Author: zentol <chesnay@...>
Date: 2018-07-16T13:16:19Z
[FLINK-9748][release] Use dedicated directory for release artifacts
This closes #6342.
commit 877cd7ef6e8a876a2a3579d0761bc2d160a4daf4
Author: Oleksandr Nitavskyi <o.nitavskyi@...>
Date: 2018-07-02T07:42:17Z
[FLINK-9762][yarn] Use local default tmp directories on Yarn and Mesos
This closes #6284.
commit fcd266fb439aa9288d6114d9bfb1e22011588d74
Author: Till Rohrmann <trohrmann@...>
Date: 2018-07-18T13:45:15Z
[FLINK-9762] Consolidate configuration cloning in BootstrapTools
commit 1d03720cfd1d4e28ce5ec1afebe38ff81ca82f1d
Author: Till Rohrmann <trohrmann@...>
Date: 2018-07-18T13:49:55Z
[hotfix] Fix checkstyle violations in BootstrapTool
commit 49c6f385df97958cda77933edfce93cd71d46600
Author: Nico Kruber <nico@...>
Date: 2018-07-17T22:51:38Z
[FLINK-9860][REST] fix buffer leak in FileUploadHandler
Change the parent's instance of SimpleChannelInboundHandler to auto-release
objects and make sure we retain them only when needed.
-> This should be the safer approach to not leaking.
Enable Netty resource leak detection for FileUploadHandlerTest and have the
test fail on detection.
This closes #6363.
commit 41d4d8d00ee0a9a73a7674a3b3143a5452cd436d
Author: Dawid Wysakowicz <dwysakowicz@...>
Date: 2018-07-17T09:25:42Z
[FLINK-9871] Use Description class for ConfigOptions with rich formatting
This closes #6371
commit 0fec75c03bba0fa85a14e3f73baeb01998c83be0
Author: Nico Kruber <nico@...>
Date: 2018-07-04T22:48:33Z
[FLINK-9755][network] forward exceptions in
RemoteInputChannel#notifyBufferAvailable() to the responsible thread
This mainly involves state checks but previously these have only been
swallowed
without re-registration or any other logging/handling. This may have lead to
some thread stalling while waiting for the notification that never came.
This closes #6272.
commit 402745ebad3eaf01622ea85524f7ff029fa8df8b
Author: Nico Kruber <nico@...>
Date: 2018-05-24T22:09:37Z
[FLINK-9435][java] optimise ComparableKeySelector and ArrayKeySelector for
more efficient Tuple creation
Benchmark results (2 runs) by running the benchmarks from
https://github.com/dataArtisans/flink-benchmarks/pull/5:
Benchmark Mode Cnt Score Error Units
================= old =================
KeyByBenchmarks.arrayKeyBy thrpt 30 1151.305 ± 21.096 ops/ms
KeyByBenchmarks.arrayKeyBy thrpt 30 1117.486 ± 43.508 ops/ms
KeyByBenchmarks.tupleKeyBy thrpt 30 1659.634 ± 28.627 ops/ms
KeyByBenchmarks.tupleKeyBy thrpt 30 1554.265 ± 82.604 ops/ms
================= new =================
KeyByBenchmarks.arrayKeyBy thrpt 30 1150.552 ± 51.185 ops/ms
KeyByBenchmarks.arrayKeyBy thrpt 30 1195.777 ± 10.621 ops/ms
KeyByBenchmarks.tupleKeyBy thrpt 30 1743.633 ± 27.109 ops/ms
KeyByBenchmarks.tupleKeyBy thrpt 30 1697.885 ± 22.101 ops/ms
This closes #6115.
commit 4862101ddcc5ede17fcd0a304583e91e2f266044
Author: Timo Walther <twalthr@...>
Date: 2018-06-04T10:49:43Z
[FLINK-7251] [types] Remove the flink-java8 module and improve lambda type
extraction
This commit removes the flink-java8 module and merges some tests into
flink-core/flink-runtime. It ensures to have the possibility for passing
explicit type information in DataStream API as a fallback. Since the tycho
compiler approach was very hacky and seems not to work anymore, this commit
also removes all references in the docs and quickstarts.
This closes #6120.
----
---
