GitHub user walterddr opened a pull request: https://github.com/apache/flink/pull/5796
[Flink 9104][doc]Re-generate REST API documentation for FLIP-6 ## What is the purpose of the change Fix REST-API doc generator and regenerate rest_dispatcher.html ## Brief change log - Changes according to FLINK-8843 - Escape HTML characters ## Verifying this change N/A ## Does this pull request potentially affect one of the following parts: no ## Documentation docs updated You can merge this pull request into a Git repository by running: $ git pull https://github.com/walterddr/flink FLINK-9104 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/5796.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5796 ---- commit 8eb6a30798c09d171e3eb8019b53e677252bd5ba Author: Piotr Nowojski <piotr.nowojski@...> Date: 2018-02-23T10:28:20Z [FLINK-8694][runtime] Fix notifyDataAvailable race condition Before there was a race condition that might resulted in igonoring some notifyDataAvailable calls. This fixes the problem by moving buffersAvailable handling to Supartitions and adds stress test for flushAlways (without this fix this test is dead locking). (cherry picked from commit ebd39f3) commit 61a34a691e7d5233f18ac72a1ab8fb09b53c4753 Author: Piotr Nowojski <piotr.nowojski@...> Date: 2018-02-26T15:13:06Z [FLINK-8805][runtime] Optimize EvenSerializer.isEvent method For example, previously if the method was used to check for EndOfPartitionEvent and the Buffer contained huge custom event, the even had to be deserialized before performing the actual check. Now we are quickly entering the correct if/else branch and doing full costly deserialization only if we have to. Other calls to isEvent() then checking against EndOfPartitionEvent were not used. (cherry picked from commit 767027f) commit d5338c4154e5de029b3b30e3ef0a0732bf7f68e7 Author: Piotr Nowojski <piotr.nowojski@...> Date: 2018-02-27T09:39:00Z [FLINK-8750][runtime] Improve detection of no remaining data after EndOfPartitionEvent Because of race condition between: 1. releasing inputChannelsWithData lock in this method and reaching this place 2. empty data notification that re-enqueues a channel we can end up with moreAvailable flag set to true, while we expect no more data. This commit detects such situation, makes a correct assertion and turn off moreAvailable flag. (cherry picked from commit b9b7416) commit 32384ed9b00cf0e1961d355dc4080f25a2156e58 Author: Zhijiang <wangzhijiang999@...> Date: 2018-02-22T14:41:38Z [FLINK-8747][bugfix] The tag of waiting for floating buffers in RemoteInputChannel should be updated properly (cherry picked from commit 6e9e0dd) commit f1453276095c55264f7b4097d16e2987a44b3f33 Author: Zhijiang <wangzhijiang999@...> Date: 2018-02-23T01:55:57Z [hotfix] Fix package private and comments (cherry picked from commit 6165b3d) commit 18ff2ce15bdb1e7bd246e438e47527a24559c86d Author: Nico Kruber <nico@...> Date: 2018-02-26T16:50:10Z [hotfix][network] minor improvements in UnionInputGate (cherry picked from commit 4203557) commit 9265666517830350a4a7037029e347f33df1bea2 Author: Nico Kruber <nico@...> Date: 2018-02-26T16:52:37Z [FLINK-8737][network] disallow creating a union of UnionInputGate instances Recently, the pollNextBufferOrEvent() was added but not implemented but this is used in getNextBufferOrEvent() and thus any UnionInputGate containing a UnionInputGate would have failed already. There should be no use case for wiring up inputs this way. Therefore, fail early when trying to construct this. (cherry picked from commit e8de538) commit 26c8f6c2a3ff75ffb954c816a57908318a2d8099 Author: Stephan Ewen <sewen@...> Date: 2018-02-28T11:15:30Z [hotfix] [tests] Fix SelfConnectionITCase The test previously did not fail on failed execution, and thus evaluated incomplete results from a failed execution with th expected results. This cleans up serialization warnings and uses lambdas where possible, to make the code more readable. commit f60e46dafa8950d5e40cd8a3286c172ecaea6b73 Author: gyao <gary@...> Date: 2018-02-28T12:04:19Z [hotfix] Add missing space to log message in ZooKeeperLeaderElectionService commit b7247929d0745b3b83306d0c93d97faf4ece4107 Author: gyao <gary@...> Date: 2018-02-28T12:06:00Z [hotfix][Javadoc] Fix typo in YARN Utils: teh -> the commit adb3750226971f7c67a0d3069103b56e4fee1c27 Author: gyao <gary@...> Date: 2018-02-28T12:07:04Z [hotfix][Javadoc] Fix typo in YarnTestBase: teh -> the commit 94bbd564ce5214b3366cc6d299fcb99ae62a2bd8 Author: gyao <gary@...> Date: 2018-02-28T12:08:25Z [hotfix][tests] Fix wrong assertEquals in YARNSessionCapacitySchedulerITCase Test swapped actual and expected arguments. Remove catching Throwable in test; instead propagate all exceptions. commit e92eb391cada27b49d32e7b11453fb5f06e19880 Author: gyao <gary@...> Date: 2018-02-28T12:20:23Z [FLINK-7805][flip6] Recover YARN containers after AM restart. Recover previously running containers after a restart of the ApplicationMaster. This is a port of a feature that was already implemented prior to FLIP-6. Extract RegisterApplicationMasterResponseReflector class into separate file. This closes #5597. commit cc32ffe70b70b926168ea91a7e92f90187dbbe57 Author: Nico Kruber <nico@...> Date: 2018-03-01T08:43:56Z [FLINK-4387][QS] don't wait and process requests at Netty servers after shutdown request There is a race condition on an assertion in Netty's event loop that may cause tests to fail when finished early. This was fixed in 4.0.33.Final, see https://github.com/netty/netty/issues/4357. commit 4b8c2a43cb79e8a1b9ec4cae623cbeb19925704e Author: Timo Walther <twalthr@...> Date: 2018-03-01T09:58:14Z [hotfix] [formats] Make ObjectMapper final in JsonNodeDeserializationSchema commit 9c0e75f8ddbbfb247d4e538a00eeb5c6586e9d39 Author: zentol <chesnay@...> Date: 2018-03-01T10:52:28Z [hotfix][build] Add missing shade-plugin execution id's 2 metric modules weren't setting the execution id to "shade-flink" causing them to not pick up the default shade-plugin configuration. commit 06b05cd204bd9a12884ad12805a61005ef40fbe7 Author: Jelmer Kuperus <jkuperus@...> Date: 2018-02-28T20:34:08Z [FLINK-8814] [file system sinks] Control over the extension of part files created by BucketingSink. commit 122e2eb4606afd0cdf20ab3bf82bb524bb9c3673 Author: Aljoscha Krettek <aljoscha.krettek@...> Date: 2018-02-28T17:02:40Z [FLINK-8810] Move end-to-end test scripts to end-to-end module This also makes the tests executable by calling $ flink-end-to-end-tests/run-pre-commit-tests.sh commit 155dd0db6bd70615ac168b4f8ec13abe6aa2b66c Author: zjureel <zjureel@...> Date: 2017-12-21T09:49:11Z [FLINK-6352] [kafka] Support to set offset of Kafka with specific date commit 1fcd516a0c55f22d06b4ce3d9bc37fb9d03f0e33 Author: Tzu-Li (Gordon) Tai <tzulitai@...> Date: 2018-01-11T06:26:37Z [FLINK-6352] [kafka] Further improvements for timestamped-based startup mode 1) Eagerly deterrmin startup offsets when startup mode is TIMESTAMP 2) Remove usage of java Date in API to specify timestamp 3) Make tests more robust and flexible 4) Add documentation for the feature This closes #5282. commit 02cb9e39cc19ebbf82f06d87035aead20780eb2b Author: Till Rohrmann <trohrmann@...> Date: 2018-02-28T13:07:12Z [FLINK-8808] [flip6] Allow RestClusterClient to connect to local dispatcher The RestClusterClient resolves a dispatcher address without an explicit host to 'localhost'. That way we allow the RestClusterClient to talk to a Dispatcher which runs in a local ActorSystem. This closes #5599. commit f30ca2101f2151214b138f37f472f886fbdfd9f0 Author: Till Rohrmann <trohrmann@...> Date: 2018-02-28T16:36:39Z [FLINK-8811] [flip6] Add initial implementation of the MiniClusterClient The MiniClusterClient directly talks to the MiniCluster avoiding polling latencies of th RestClusterClient. This closes #5600. commit 19e4f68ba9cfbf5d0f54b325db4c5d196d262d09 Author: Till Rohrmann <trohrmann@...> Date: 2018-02-28T16:49:29Z [FLINK-8811] [flip6] Implement MiniClusterClient#getJobStatus commit 29b34e2255c41abc1c7c4af8ab268a39df57f0ff Author: Till Rohrmann <trohrmann@...> Date: 2018-02-28T16:54:06Z [FLINK-8811] [flip6] Implement MiniClusterClient#cancel commit 15847c4d9c3155aa9a46498d5cf3a6676e0fa08d Author: Till Rohrmann <trohrmann@...> Date: 2018-03-01T16:02:29Z [hotfix] Introduce null checks for SlotManager#suspend commit 40791147efa2f3ca1c8950ed2646a3ac3a844716 Author: Till Rohrmann <trohrmann@...> Date: 2018-03-01T16:05:44Z [hotfix] Let ClusterClient only shut down own HaServices commit 2b693502318dd3bfdae352b6ae4f4be6e4443bbc Author: Till Rohrmann <trohrmann@...> Date: 2018-03-01T16:06:27Z [hotfix] Unregister job from JobManagerRunner before completing the result future commit f0d4b3c98f71fa15b99e55634bc4d067b48149fc Author: Till Rohrmann <trohrmann@...> Date: 2018-03-01T16:13:44Z [hotfix] Close ResourceManager LeaderRetrievalService in TaskExecutor commit ea920efc64254b16322bb1d13b5ab1c8633cbda7 Author: Till Rohrmann <trohrmann@...> Date: 2018-03-01T17:21:19Z [hotfix] Correct shutdown order of RestClusterClient commit 8485912f807b8bc43c46fe785ec16a56d9253b5a Author: Till Rohrmann <trohrmann@...> Date: 2018-03-01T18:03:35Z [hotfix] Let AbstractEventTimeWindowCheckpointingITCase shutdown ZooKeeper after MiniCluster ---- ---