[GitHub] kafka pull request #4349: KAFKA-6366 [WIP]: Fix stack overflow in consumer d...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/4349 KAFKA-6366 [WIP]: Fix stack overflow in consumer due to fast offset commits during coordinator disconnect When the coordinator is marked unknown, we explicitly disconnect its connection and cancel pending requests. Currently the disconnect happens before the coordinator state is set to null, which means that callbacks which inspect the coordinator state will see it still as active. This can lead to further requests being sent. In pathological cases, the disconnect itself is not able to return because new requests are sent to the coordinator before the disconnect can complete, which leads to the stack overflow error. To fix the problem, I have reordered the disconnect to happen after the coordinator is set to null. I have added a basic test case to verify that callbacks for in-flight or unsent requests see the coordinator as unknown which prevents them from attempting to resend. We may need additional test cases after we determine whether this is in fact was it happening in the reported ticket. Note that I have also included some minor cleanups which I noticed along the way. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-6366 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/4349.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4349 commit 488de3dca5be6111fd447980c8e79477259dc99a Author: Jason Gustafson <jason@...> Date: 2017-12-18T18:53:38Z KAFKA-6366 [WIP]: Fix stack overflow in consumer due to fast offset commits during coordinator disconnect ---
[GitHub] kafka pull request #4287: KAFKA-6118: Fix transient failure testTwoConsumers...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/4287 KAFKA-6118: Fix transient failure testTwoConsumersWithDifferentSaslCredentials It's rare, but it can happen that the initial FindCoordinator request returns before the first Metadata request. Both authorization errors are fine for this test case. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-6118 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/4287.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4287 commit 334cd1d9150351f38c736efc0ae364bd9698e8c5 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-12-02T06:03:16Z KAFKA-6118: Fix transient failure testTwoConsumersWithDifferentSaslCredentials ---
[GitHub] kafka pull request #4285: KAFKA-6296: Increase jitter to fix transient failu...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/4285 KAFKA-6296: Increase jitter to fix transient failure in NetworkClientTest.testConnectionDelayDisconnected ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-6296 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/4285.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4285 commit 04dd99039670c4fbcf8f6030c41fa8f39297838c Author: Jason Gustafson <ja...@confluent.io> Date: 2017-12-01T19:09:27Z KAFKA-6296: Increase jitter to fix transient failure in NetworkClientTest.testConnectionDelayDisconnected ---
[GitHub] kafka pull request #4280: KAFKA-6289: NetworkClient should not expose failed...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/4280 KAFKA-6289: NetworkClient should not expose failed internal ApiVersions requests The NetworkClient internally ApiVersion requests to each broker following connection establishment. If this request happens to fail (perhaps due to an incompatible broker), the NetworkClient includes the response in the result of poll(). Applications will generally not be expecting this response which may lead to failed assertions (or in the case of AdminClient, an obscure log message). I've added test cases which await the ApiVersion request sent by NetworkClient to be in-flight, and then disconnect the connection and verify that the response is not included from poll(). ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-6289 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/4280.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4280 commit 3c7420b2ab51f8d918037681ed9862f0d2676ba7 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-12-01T05:03:54Z KAFKA-6289: NetworkClient should not expose failed internal ApiVersion requests ---
[GitHub] kafka pull request #4276: KAFKA-6260: Ensure selection keys are removed from...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/4276 KAFKA-6260: Ensure selection keys are removed from all collections on socket close When a socket is closed, we must remove corresponding selection keys from internal collections. This fixes an NPE which is caused by attempting to access the selection key's attached channel after it had been cleared after disconnecting. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-6260 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/4276.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4276 commit e715e673b7bca14e2a26a998348528d27ac8a9c8 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-11-29T19:10:39Z KAFKA-6260: Ensure selection keys are removed from all collections on socket close ---
[GitHub] kafka pull request #4256: KAFKA-6328: Fix 1.0.0 upgrade instructions relatin...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/4256 KAFKA-6328: Fix 1.0.0 upgrade instructions relating to the message format version The upgrade instructions concerning the message format versions did not account for upgrades from versions prior to 0.11.0.x. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-6328 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/4256.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4256 commit 9cb1a3885fe62d1883f0e3615cef126b99de0743 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-11-23T00:59:19Z KAFKA-6328: Fix 1.0.0 upgrade instructions relating to the message format version ---
[GitHub] kafka pull request #4147: MINOR: Fix inconsistency in StopReplica/LeaderAndI...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/4147 MINOR: Fix inconsistency in StopReplica/LeaderAndIsr error counts You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka fix-error-inconsistencies Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/4147.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4147 commit a3cc429885dc4b998ec4933f2e16ac1c14659d36 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-10-27T19:05:25Z MINOR: Fix inconsistency in StopReplica/LeaderAndIsr error counts ---
[GitHub] kafka pull request #4141: KAFKA-6134: Read partition reassignment lazily on ...
Github user hachikuji closed the pull request at: https://github.com/apache/kafka/pull/4141 ---
[GitHub] kafka pull request #4141: KAFKA-6134: Read partition reassignment lazily on ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/4141 KAFKA-6134: Read partition reassignment lazily on event handling This patch prevents an O(n^2) increase in memory utilization during partition reassignment. Instead of storing the reassigned partitions in the `PartitionReassignment` object (which is added after ever partition reassignment), we read the data fresh from ZK when processing the event. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-6134 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/4141.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4141 commit 5131bb19f6fe7fc1939035c48ead052a0ac967a4 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-10-27T02:01:05Z KAFKA-6134: Read partition reassignment lazily on event handling ---
[GitHub] kafka pull request #4076: MINOR: A few javadoc fixes
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/4076 MINOR: A few javadoc fixes You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka javadoc-fixes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/4076.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4076 commit 43817370f8673a309969477dc587545c528c06d5 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-10-16T22:42:39Z MINOR: A few javadoc fixes ---
[GitHub] kafka pull request #4047: MINOR: Factor out some common group/transactional ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/4047 MINOR: Factor out some common group/transactional fields in request objects You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka factor-out-some-common-fields Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/4047.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4047 commit b9c167ce588b282d8fb8548a8e0525199c21e2e7 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-10-09T19:08:41Z MINOR: Factor out some common group/transactional fields in request objects ---
[GitHub] kafka pull request #3976: MINOR: Follow-up cleanup of KAFKA-5960
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3976 MINOR: Follow-up cleanup of KAFKA-5960 You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka version-fix-followup Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3976.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3976 commit 1e7e98782841847f07d8d5a51dec9831785c607b Author: Jason Gustafson <ja...@confluent.io> Date: 2017-09-27T18:06:35Z MINOR: Follow-up cleanup of KAFKA-5960 ---
[GitHub] kafka pull request #3944: KAFKA-5960 [WIP]; Fix regression in produce versio...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3944 KAFKA-5960 [WIP]; Fix regression in produce version selection on old brokers You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5960 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3944.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3944 commit 83c613c83783e90a098d254b65713bab0013624a Author: Jason Gustafson <ja...@confluent.io> Date: 2017-09-22T07:11:31Z KAFKA-5960; Fix regression in produce version selection on old brokers ---
[GitHub] kafka pull request #3942: KAFKA-5957: Prevent second deallocate if response ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3942 KAFKA-5957: Prevent second deallocate if response for aborted batch returns You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5957 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3942.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3942 commit 4974842e629c098b0d36bc42e189bff211e7faac Author: Jason Gustafson <ja...@confluent.io> Date: 2017-09-22T00:09:30Z KAFKA-5957: Prevent second deallocate if produce response for aborted batch returns ---
[GitHub] kafka pull request #3863: MINOR: Use SecurityProtocol in AuthenticationConte...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3863 MINOR: Use SecurityProtocol in AuthenticationContext Since we removed the unused `TRACE` option from `SecurityProtocol`, it now seems safer to expose it from `AuthenticationContext`. Additionally this patch exposes javadocs under security.auth and relocates the `Login` and `AuthCallbackHandler` to a non-public package. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka use-security-protocol-in-auth-context Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3863.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3863 commit 7ea23a1e5f873ae9822ba4946c88ca3072c2674b Author: Jason Gustafson <ja...@confluent.io> Date: 2017-09-14T21:04:50Z MINOR: Use SecurityProtocol in AuthenticationContext ---
[GitHub] kafka pull request #3852: MINOR: Update TransactionManager to use LogContext
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3852 MINOR: Update TransactionManager to use LogContext You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-use-log-context-txn-manager Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3852.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3852 commit ed9809dad1cb5647a1a5d4d22ead333d098571bd Author: Jason Gustafson <ja...@confluent.io> Date: 2017-09-13T21:01:27Z MINOR: Update TransactionManager to use LogContext ---
[GitHub] kafka pull request #3813: MINOR: Move request/response schemas to the corres...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3813 MINOR: Move request/response schemas to the corresponding object representation This refactor achieves the following: 1. Breaks up the increasingly unmanageable `Protocol` class. 2. Removes the need for redundant field identifiers maintained separately in `Protocol` and the respective request/response objects. 3. Provides a better mechanism for sharing common fields between different schemas (e.g. topics, partitions, error codes, etc.). 4. Adds convenience helpers to `Struct` for common patterns (such as setting a field only if it exists). You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka protocol-schema-refactor Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3813.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3813 commit 9d008c806e43f7a13221fd5761aed79fb9b4f2cf Author: Jason Gustafson <ja...@confluent.io> Date: 2017-09-07T22:34:27Z MINOR: Move request/response schemas to the corresponding object representation ---
[GitHub] kafka pull request #3795: KAFKA-5783: Add KafkaPrincipalBuilder with support...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3795 KAFKA-5783: Add KafkaPrincipalBuilder with support for SASL (KIP-189) You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5783 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3795.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3795 commit 3c2f106802508bd3cf305d08b8031442cc868f8f Author: Jason Gustafson <ja...@confluent.io> Date: 2017-08-24T22:47:31Z KAFKA-5783: Add KafkaPrincipalBuilder with support for SASL (KIP-189) ---
[GitHub] kafka pull request #3778: KAFKA-5822: Consistent log formatting of topic par...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3778 KAFKA-5822: Consistent log formatting of topic partitions You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5822 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3778.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3778 commit a40373c229510fcadd6156253a59f6d7badf5d47 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-09-01T23:35:05Z KAFKA-5822: Consistent log formatting of topic partitions --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka-site pull request #72: Jason and Becket are PMC members
Github user hachikuji closed the pull request at: https://github.com/apache/kafka-site/pull/72 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka-site pull request #72: Jason and Becket are PMC members
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka-site/pull/72 Jason and Becket are PMC members You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka-site update-pmc-members Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka-site/pull/72.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #72 commit 6625934c1250c22cf9ba222f871aff78951b6071 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-08-30T01:05:52Z Jason and Becket are PMC members --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3716: KAFKA-5342: Clarify fatal/abortable exceptions use...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3716 KAFKA-5342: Clarify fatal/abortable exceptions used in producer You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5342 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3716.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3716 commit cdbe4fcfb5ae86a3a93f0954b0889f1d84e2ae93 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-08-22T21:55:57Z KAFKA-5342: Clarify fatal/abortable exceptions used in producer --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3688: KAFKA-5747: Producer snapshot loading should cover...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3688 KAFKA-5747: Producer snapshot loading should cover schema and other errors You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5747 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3688.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3688 commit 21ea5414239206fdaacebb7a1304fd9fe598c68f Author: Jason Gustafson <ja...@confluent.io> Date: 2017-08-17T21:27:43Z KAFKA-5747: Producer snapshot loading should cover schema and other errors --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3676: MINOR: Ensure consumer logging has clientId/groupI...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3676 MINOR: Ensure consumer logging has clientId/groupId context You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka log-consumer-wakeups Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3676.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3676 commit 2103791ece6b19c81c031e7a40250b9921b60be9 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-08-16T18:39:13Z MINOR: Ensure consumer logging has clientId/groupId context --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3673: MINOR: Consolidate broker request/response handlin...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3673 MINOR: Consolidate broker request/response handling This patch contains a few small improvements to make request/response handling more consistent. Primarily it consolidates request/response serialization logic so that `SaslServerAuthenticator` and `KafkaApis` follow the same path. It also reduces the amount of custom logic needed to handle unsupported versions of the ApiVersions requests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka consolidate-response-handling Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3673.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3673 commit bbbeb17ed5aa4fcd5ca52bb98564dafdd854bb94 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-08-04T21:58:11Z MINOR: Consolidate broker request/response handling --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3666: KAFKA-5730: Consumer should invoke async commit ca...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3666 KAFKA-5730: Consumer should invoke async commit callback before sync commit returns You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5730 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3666.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3666 commit 913b11db4a1651010270e4615d91937b5b65cd4c Author: Jason Gustafson <ja...@confluent.io> Date: 2017-08-14T18:56:47Z KAFKA-5730: Consumer should invoke async commit callback before sync commit returns --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3665: HOTFIX: ConsoleConsumer using wrong old consumer c...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3665 HOTFIX: ConsoleConsumer using wrong old consumer config value for auto.offset.reset You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka hotfix-auto-reset-console-consumer Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3665.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3665 commit 37b49fb7857ee23a286d82d07c8169868f6eff9c Author: Jason Gustafson <ja...@confluent.io> Date: 2017-08-14T16:27:48Z HOTFIX: ConsoleConsumer using wrong old consumer config value for auto.offset.reset --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3651: MINOR: Add missing deprecations on old request obj...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3651 MINOR: Add missing deprecations on old request objects You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka add-missing-request-deprecations Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3651.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3651 commit a318ae47f6515c1138d56decbf274f5196bb5411 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-08-09T19:49:12Z MINOR: Add missing deprecations on old request objects --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3646: MINOR: Remove unneeded error handlers in deprecate...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3646 MINOR: Remove unneeded error handlers in deprecated request objects These handlers were previously used on the broker to handle uncaught exceptions, but now the broker users the new Java request objects exclusively. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka remove-old-request-error-handlers Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3646.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3646 commit 51546938116c8820a078f4ebed0b52a8d3916be0 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-08-09T00:10:00Z MINOR: Remove unneeded error handlers in deprecated request objects --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3627: MINOR: Fix error response handler for controlled s...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3627 MINOR: Fix error response handler for controlled shutdown v0 You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-fix-controlled-shutdown-error-response Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3627.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3627 commit b5ed4d9454c128e92d0ba9ee46cfa97c02ef81dc Author: Jason Gustafson <ja...@confluent.io> Date: 2017-08-04T18:38:40Z MINOR: Fix error response handler for controlled shutdown v0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3612: MINOR: Remove legacy ControlledShutdown request/re...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3612 MINOR: Remove legacy ControlledShutdown request/response objects You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka remove-old-controlled-shutdown-objects Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3612.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3612 commit 35185a2eae66e9220c0357f0022b43219fd81d27 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-08-02T23:59:56Z MINOR: Remove legacy ControlledShutdown request/response objects --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3575: KAFKA-5634; Do not allow segment deletion beyond h...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3575 KAFKA-5634; Do not allow segment deletion beyond high watermark You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5634 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3575.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3575 commit 28ba15871d954fa7036b26ebebc5268fbaf1b818 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-07-26T00:13:43Z KAFKA-5634; Do not allow segment deletion beyond high watermark --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3571: KAFKA-5611; AbstractCoordinator should handle wake...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3571 KAFKA-5611; AbstractCoordinator should handle wakeup raised from onJoinComplete You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5611 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3571.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3571 commit e0b4f65031dbb8135d872811c68dec94f7a45efd Author: Jason Gustafson <ja...@confluent.io> Date: 2017-07-25T05:14:42Z KAFKA-5611; AbstractCoordinator should handle wakeup raised from onJoinComplete --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3558: MINOR: Safer handling of requests prior to SASL au...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3558 MINOR: Safer handling of requests prior to SASL authentication This implements two improvements for request handling prior to SASL authentication: 1. Only parse request types that are allowed prior to authentication. 2. Limit the maximum request size (the default is 100Mb). You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-restrict-presasl-request-parsing Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3558.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3558 commit 02c7e50c91aad59ccd88700977363c1bb3f9340b Author: Jason Gustafson <ja...@confluent.io> Date: 2017-07-14T16:53:10Z MINOR: Safer handling of requests prior to SASL authentication --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3457: MINOR: Rename baseTimestamp to firstTimestamp to c...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3457 MINOR: Rename baseTimestamp to firstTimestamp to clarify usage You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka rename-base-timestamp Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3457.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3457 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3456: KAFKA-5522: ListOffsets should bound timestamp sea...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3456 KAFKA-5522: ListOffsets should bound timestamp search by LSO in read_committed You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5522 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3456.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3456 commit 99bb1b1ee84815f3a2553aad297d2019b12d1c3d Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-28T20:26:08Z KAFKA-5522: ListOffsets should bound timestamp search by LSO in read_committed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3426: MINOR: Fix race condition in KafkaConsumer close
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3426 MINOR: Fix race condition in KafkaConsumer close We intended to make `KafkaConsumer.close()` idempotent, but due to the fact that the `closed` variable is checked without a lock prior to beginning close logic, it is possible for two or more threads to see `closed=false` and attempt to close. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-fix-consumer-idempotent-close Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3426.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3426 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3406: KAFKA-5490: Retain empty batch for last sequence o...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3406 KAFKA-5490: Retain empty batch for last sequence of each producer You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5490 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3406.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3406 commit cf27cc1d69de90c513d96895ec2f557a49b2b3b6 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-21T23:55:36Z KAFKA-5490: Retain empty batch for last sequence of each producer --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3388: KAFKA-5021: Update delivery semantics documentatio...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3388 KAFKA-5021: Update delivery semantics documentation for EoS (KIP-98) You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5021 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3388.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3388 commit 4433be408249594f0771fcd57f9a9cbaddd70648 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-20T21:34:53Z KAFKA-5021: Update delivery semantics documentation for EoS (KIP-98) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Github user hachikuji closed the pull request at: https://github.com/apache/kafka-site/pull/60 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka-site issue #60: Update delivery semantics section for KIP-98
Github user hachikuji commented on the issue: https://github.com/apache/kafka-site/pull/60 I'm going to reopen this as a PR against kafka since that is where design.html is maintained. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3376: MINOR: MemoryRecordsBuilder.sizeInBytes should con...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3376 MINOR: MemoryRecordsBuilder.sizeInBytes should consider initial position You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka records-builder-improvements Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3376.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3376 commit c3b551569f327bae238b609ea9936755cb210218 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-19T22:09:48Z MINOR: MemoryRecordsBuilder.sizeInBytes should consider initial position --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Github user hachikuji commented on a diff in the pull request: https://github.com/apache/kafka-site/pull/60#discussion_r122826323 --- Diff: 0110/design.html --- @@ -264,21 +264,22 @@ messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself). -So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with -what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be -handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not -support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated -or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication. - -A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above. -Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted, -the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default -"read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed -(and any messages which were not part of any transaction). - -So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of -messages. Exactly-once delivery is supported when processing messages between Kafka topics, such as in Kafka Streams applications. Exactly-once delivery for other destination storage system generally requires -cooperation with that system, but Kafka provides the offset which makes implementing this straight-forward. +So what about exactly once semantics (i.e. the thing you actually want)? When consuming from a Kafka topic and producing to another topic (as in a https://kafka.apache.org/documentation/streams;>Kafka Streams +application), we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above. The consumer's position is stored as a message in a topic, so we can write the offset to Kafka in the +same transaction as the output topics receiving the processed data. If the transaction is aborted, the consumer's position will revert to its old value and none of the output data will be visible to consumers. Consumers support an "isolation level" configuration +to achieve this. In the default "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from +transactions which were committed (and any messages which were not part of any transaction). + +When writing to an external system, the limitation is in the need to coordinate the consumer's position with what is actually stored as output. The classic way of achieving this would be to introduce a two-phase +commit between the storage for the consumer position and the storage of the consumers output. But this can be handled more simply and generally by simply letting the consumer store its offset in the same place as --- End diff -- I'm not sure I understand the second question. Can you elaborate? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Github user hachikuji commented on a diff in the pull request: https://github.com/apache/kafka-site/pull/60#discussion_r122825810 --- Diff: 0110/design.html --- @@ -264,21 +264,22 @@ messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself). -So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with -what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be -handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not -support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated -or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication. - -A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above. -Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted, -the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default -"read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed -(and any messages which were not part of any transaction). - -So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of -messages. Exactly-once delivery is supported when processing messages between Kafka topics, such as in Kafka Streams applications. Exactly-once delivery for other destination storage system generally requires -cooperation with that system, but Kafka provides the offset which makes implementing this straight-forward. +So what about exactly once semantics (i.e. the thing you actually want)? When consuming from a Kafka topic and producing to another topic (as in a https://kafka.apache.org/documentation/streams;>Kafka Streams +application), we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above. The consumer's position is stored as a message in a topic, so we can write the offset to Kafka in the +same transaction as the output topics receiving the processed data. If the transaction is aborted, the consumer's position will revert to its old value and none of the output data will be visible to consumers. Consumers support an "isolation level" configuration +to achieve this. In the default "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from +transactions which were committed (and any messages which were not part of any transaction). + +When writing to an external system, the limitation is in the need to coordinate the consumer's position with what is actually stored as output. The classic way of achieving this would be to introduce a two-phase +commit between the storage for the consumer position and the storage of the consumers output. But this can be handled more simply and generally by simply letting the consumer store its offset in the same place as +its output. This is better because many of the output systems a consumer might want to write to will not support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its +offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated or neither is. We follow similar patterns for many other data systems which requi
[GitHub] kafka pull request #3373: MINOR: Detail message/batch size implications for ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3373 MINOR: Detail message/batch size implications for conversion between old and new formats You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka fetch-size-upgrade-notes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3373.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3373 commit 6676220d61d8a07e768923436169b629898b64da Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-19T20:52:18Z MINOR: Detail message/batch size implications for conversion between old and new formats --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3363: MINOR: Some small cleanups/improvements to KAFKA-5...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3363 MINOR: Some small cleanups/improvements to KAFKA-5031 You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5031-FOLLOWUP Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3363.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3363 commit 5bbfff43d5ab908e88811c61c9b1f86221b8cf6c Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-17T04:37:56Z MINOR: Some small cleanups/improvements to KAFKA-5031 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3361: KAFKA-5435: Improve producer state loading after f...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3361 KAFKA-5435: Improve producer state loading after failure You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5435-ALT Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3361.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3361 commit 5bf7d189589e4986e9552c955d101fe8e352d49c Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-17T00:25:12Z KAFKA-5435: Improve producer state loading after failure --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3306: KAFKA-5435: Ensure producer snapshot retained afte...
Github user hachikuji closed the pull request at: https://github.com/apache/kafka/pull/3306 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3356: KAFKA-5456: Ensure producer handles old format lar...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3356 KAFKA-5456: Ensure producer handles old format large compressed messages You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5456 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3356.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3356 commit a0805d21a0fdef0dab527464f39c3b0e07a59310 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-16T06:19:48Z KAFKA-5456: Ensure producer handles old format large compressed messages --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3348: MINOR: Fix unsafe access to incomplete collection ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3348 MINOR: Fix unsafe access to incomplete collection in RecordAccumulator You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka fix-has-unflushed-synchronization Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3348.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3348 commit adfca6ff34aafae082b6ccf8afe51dd0b5a63533 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-15T03:10:51Z MINOR: Fix unsafe access to incomplete collection in RecordAccumulator --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka-site issue #60: Update delivery semantics section for KIP-98
Github user hachikuji commented on the issue: https://github.com/apache/kafka-site/pull/60 @guozhangwang @ijuma @apurvam Pushed a couple changes based on the review. Please take another look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3340: MINOR: Add random aborts to system test transactio...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3340 MINOR: Add random aborts to system test transactional copier service You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka add-random-aborts-to-system-test Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3340.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3340 commit b55ff7dc9f6e347d258a1647ea7d36138f4c6108 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-10T17:23:27Z MINOR: Add random aborts to system test transactional copier service --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3331: KAFKA-5443: Consumer should use last offset from b...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3331 KAFKA-5443: Consumer should use last offset from batch to set next fetch offset You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5443 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3331.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3331 commit b991c8185c5bce1cedb39dc5158afaed1b8db2f4 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-14T04:17:33Z KAFKA-5443: Consumer should use last offset from batch to set next fetch offset --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3318: HOTFIX: Fix invalid long format conversion in requ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3318 HOTFIX: Fix invalid long format conversion in request logger message You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka hotfix-request-metric-udpate Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3318.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3318 commit 659930d1dbda09690f9b046e096dee77a0318c83 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-13T17:22:57Z HOTFIX: Fix invalid long format conversion in request logger message --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3306: KAFKA-5435: Ensure producer snapshot retained afte...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3306 KAFKA-5435: Ensure producer snapshot retained after truncation You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5435 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3306.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3306 commit 2e0a797cb5a0baa12032697500fb4a8c5d87b25a Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-12T17:38:33Z KAFKA-5435: Ensure producer snapshot retained after truncation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3300: KAFKA-5429: Ignore produce response if batch was p...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3300 KAFKA-5429: Ignore produce response if batch was previously aborted You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5429 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3300.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3300 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3298: KAFKA-5428: Transactional producer should only abo...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3298 KAFKA-5428: Transactional producer should only abort batches in fatal error state You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5428 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3298.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3298 commit 5bd97c0e10ba73a79299d40d3223718acfe7e97d Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-11T07:18:05Z KAFKA-5428: Transactional producer should only abort batches in fatal error state --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3297: KAFKA-5427: Transactional producer should allow Fi...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3297 KAFKA-5427: Transactional producer should allow FindCoordinator in error state You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5427 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3297.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3297 commit 76ffa96261084474cde0db350b96d8e4df9cbc55 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-10T23:31:32Z KAFKA-5427: Transactional producer should allow FindCoordinator in error state --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Github user hachikuji commented on a diff in the pull request: https://github.com/apache/kafka-site/pull/60#discussion_r121249550 --- Diff: 0110/design.html --- @@ -261,15 +262,23 @@ It can read the messages, process the messages, and finally save its position. In this case there is a possibility that the consumer process crashes after processing messages but before saving its position. In this case when the new process takes over the first few messages it receives will already have been processed. This corresponds to the "at-least-once" semantics in the case of consumer failure. In many cases messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself). -So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to co-ordinate the consumer's position with + + +So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication. - -So effectively Kafka guarantees at-least-once delivery by default and allows the user to implement at most once delivery by disabling retries on the producer and committing its offset prior to processing a batch of -messages. Exactly-once delivery requires co-operation with the destination storage system but Kafka provides the offset which makes implementing this straight-forward. +A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above. +Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted, +the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default --- End diff -- Thanks. I will try to clarify these lines. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Github user hachikuji commented on a diff in the pull request: https://github.com/apache/kafka-site/pull/60#discussion_r121249508 --- Diff: 0110/design.html --- @@ -261,15 +262,23 @@ It can read the messages, process the messages, and finally save its position. In this case there is a possibility that the consumer process crashes after processing messages but before saving its position. In this case when the new process takes over the first few messages it receives will already have been processed. This corresponds to the "at-least-once" semantics in the case of consumer failure. In many cases messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself). -So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to co-ordinate the consumer's position with + + +So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication. - -So effectively Kafka guarantees at-least-once delivery by default and allows the user to implement at most once delivery by disabling retries on the producer and committing its offset prior to processing a batch of -messages. Exactly-once delivery requires co-operation with the destination storage system but Kafka provides the offset which makes implementing this straight-forward. +A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above. +Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted, +the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default +"read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed +(and any messages which were not part of any transaction). + +So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of --- End diff -- Good point. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3284: MINOR: A few cleanups of usage of Either in TC
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3284 MINOR: A few cleanups of usage of Either in TC You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-either-usage-cleanup Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3284.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3284 commit cc8f8e75afed382e0eb77b3238aa3dff30a2c100 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-09T18:30:36Z MINOR: A few cleanups of usage of Either in TC --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3277: KAFKA-5414: Revert "KAFKA-5327: ConsoleConsumer sh...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3277 KAFKA-5414: Revert "KAFKA-5327: ConsoleConsumer should manually commi… …t offsets for records that are returned in receive()" This reverts commit d7d1196a0b542adb46d22eeb5b6d12af950b64c9. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5414 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3277.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3277 commit 2676945c37f3526f73e0dbc0da66e14bb4f73d7e Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-09T00:26:01Z KAFKA-5414: Revert "KAFKA-5327: ConsoleConsumer should manually commit offsets for records that are returned in receive()" This reverts commit d7d1196a0b542adb46d22eeb5b6d12af950b64c9. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka-site issue #60: Update delivery semantics section for KIP-98
Github user hachikuji commented on the issue: https://github.com/apache/kafka-site/pull/60 cc @guozhangwang @apurva @ijuma --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka-site/pull/60 Update delivery semantics section for KIP-98 You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka-site update-delivery-semantics Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka-site/pull/60.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #60 commit 0759f3a22e377e12e857eb6da4977adb62261d30 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-08T21:28:13Z Update delivery semantics section for KIP-98 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3259: MINOR: A few logging improvements in group coordin...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3259 MINOR: A few logging improvements in group coordinator You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka group-coordinator-logging-improvements Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3259.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3259 commit c4dcd2ee0d65d5c78eb1c55142eaab1293b4435d Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-07T16:20:58Z MINOR: A few logging improvements in group coordinator --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3248: KAFKA-5378: Return LSO in FetchResponse plus some ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3248 KAFKA-5378: Return LSO in FetchResponse plus some metrics You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5378 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3248.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3248 commit 3395e4da000dc2cf75861e230a19aeebbe5c317c Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-06T07:32:00Z KAFKA-5378: Return LSO in FetchResponse commit 09fb074348fe0972b05ba33625470af018b4962a Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-06T17:15:30Z Use LSO to compute fetch lag in READ_COMMITTED commit 0c651d8af634c979c033d397480fae032df92578 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-06T17:39:38Z Add LSO lag metric on broker --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3239: KAFKA-5376: Ensure aborted transactions are propag...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3239 KAFKA-5376: Ensure aborted transactions are propagated in DelayedFetch You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5376 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3239.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3239 commit dd358917d3ee5233f477061af01be6a202138d83 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-05T18:59:29Z KAFKA-5376: Ensure aborted transactions are propagated in DelayedFetch --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3231: KAFKA-5364: ensurePartitionAdded does not handle p...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3231 KAFKA-5364: ensurePartitionAdded does not handle pending partitions in abortable error state You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5364 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3231.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3231 commit 7e65cf7e4642e8dca29a02321e29e4701a1712ba Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-04T01:26:51Z KAFKA-5364: ensurePartitionAdded does not handle pending partitions in abortable error state --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3230: KAFKA-5355: Test cases to ensure isolation level p...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3230 KAFKA-5355: Test cases to ensure isolation level propagated in delayed fetch You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5355-TESTS Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3230.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3230 commit ef598178633d1a2e7e87a3a1e5b4ee879d3e6f47 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-03T22:43:02Z KAFKA-5355: Test cases to ensure isolation level propagated in delayed fetch --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3221: KAFKA-5355: DelayedFetch should propagate isolatio...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3221 KAFKA-5355: DelayedFetch should propagate isolation level to log read You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5355 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3221.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3221 commit 82430eefb669718a35b93a34a2c8afec2230351b Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-03T02:26:39Z KAFKA-5355: DelayedFetch should propagate isolation level to log read --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3212: KAFKA-5019: Upgrades notes for idempotent/transact...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3212 KAFKA-5019: Upgrades notes for idempotent/transactional features and new message format You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5019 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3212.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3212 commit 84390c7a6c65e78d01e7eb164a65e27d1cd03adb Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-02T20:49:32Z KAFKA-5019: Upgrades notes for idempotent/transactional features and new message format --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3203: KAFKA-5365 [WIP]: Fix regression in compressed mes...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3203 KAFKA-5365 [WIP]: Fix regression in compressed message iteration affecting magic v0 and v1 You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5365 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3203.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3203 commit 0a67684c9d895be1dc7a883e753b5c42c662e819 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-06-02T00:52:52Z KAFKA-5365 [WIP]: Fix regression in compressed message iteration affecting magic v0 and v1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3165: KAFKA-5316: LogCleaner should account for larger r...
Github user hachikuji closed the pull request at: https://github.com/apache/kafka/pull/3165 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3185: MINOR: Logging/debugging improvements for transact...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3185 MINOR: Logging/debugging improvements for transactions You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-transaction-logging-improvements Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3185.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3185 commit b0222146ec420c0f0a7bec75a5b6141862cd102b Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-31T22:40:17Z MINOR: Logging/debugging improvements for transactions --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3183: KAFKA-5283: Handle producer epoch/sequence overflo...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3183 KAFKA-5283: Handle producer epoch/sequence overflow by wrapping around You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5283 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3183.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3183 commit 1b92d3958a50fff256262bcf6c3cf4b5747fb580 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-31T20:16:04Z KAFKA-5283: Handle producer epoch/sequence overflow by wrapping around --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3175: KAFKA-5349: Fix illegal state error in consumer's ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3175 KAFKA-5349: Fix illegal state error in consumer's ListOffset handler You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5349 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3175.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3175 commit 12a38b4506494c76bc315526044dad9ec217ec80 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-31T07:05:19Z KAFKA-5349: Fix illegal state error in consumer's ListOffset handler --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3171: MINOR: A few cleanups in KafkaApis and Transaction...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3171 MINOR: A few cleanups in KafkaApis and TransactionMarkerChannelManager You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-txn-channel-cleanups Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3171.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3171 commit 4b5717a935abc1c6e9ceccf51f7a0fdf91aee763 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-30T23:09:57Z MINOR: A few cleanups in KafkaApis and TransactionMarkerChannelManager --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3165: KAFKA-5316: LogCleaner should account for larger r...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3165 KAFKA-5316: LogCleaner should account for larger record sets after cleaning You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5316-0.10.2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3165.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3165 commit 98539df716ca866f06a708d7556ad39ca4e1caae Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-30T05:49:33Z KAFKA-5316: LogCleaner should account for larger record sets after cleaning --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3162: KAFKA-5340: Batch splitting should preserve magic ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3162 KAFKA-5340: Batch splitting should preserve magic and transactional flag You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5340 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3162.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3162 commit 8881e16f5d690e3cc107786d08d6ad39cb4d2b14 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-29T03:27:59Z KAFKA-5340: Batch splitting should preserve magic and transactional flag --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3161: KAFKA-5251: Producer should cancel unsent AddParti...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3161 KAFKA-5251: Producer should cancel unsent AddPartitions and Produce requests on abort You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5251 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3161.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3161 commit 501a761326128172546b5235c3cdd4d1862a2de8 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-26T16:36:18Z KAFKA-5251: Producer should cancel unsent Produce and AddPartitions requests on abort --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3160: KAFKA-5093: Avoid loading full batch data when pos...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3160 KAFKA-5093: Avoid loading full batch data when possible when iterating FileRecords You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5093 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3160.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3160 commit a453e37032a8e0db7dde69f2c76d22070cabe80a Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-27T07:56:55Z KAFKA-5093: Avoid loading full batch data when possible when iterating FileRecords --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3145: MINOR: Add test case for duplicate check after log...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3145 MINOR: Add test case for duplicate check after log cleaning You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-improve-base-sequence-docs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3145.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3145 commit b75e295f2743b367ca92525c7863fc05fca77438 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-25T18:48:02Z MINOR: Add test case for duplicate check after log cleaning --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3138: KAFKA-5017: Record batch first offset remains accu...
Github user hachikuji closed the pull request at: https://github.com/apache/kafka/pull/3138 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3142: KAFKA-5316: LogCleaner should account for larger r...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3142 KAFKA-5316: LogCleaner should account for larger record sets after cleaning You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5316 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3142.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3142 commit e8981baa183444870f8ab1d6634e4e3f634371c2 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-24T23:41:49Z KAFKA-5316: LogCleaner should account for larger record sets after cleaning --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3138: KAFKA-5017: Record batch first offset remains accu...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3138 KAFKA-5017: Record batch first offset remains accurate after compaction You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5017 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3138.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3138 commit c24ba61c27abb12d86705b9f0c7e322e24cdf692 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-24T18:37:26Z KAFKA-5017: Record batch first offset remains accurate after compaction --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3133: MINOR: GroupCoordinator can append with group lock
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3133 MINOR: GroupCoordinator can append with group lock You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-replica-manager-append-refactor Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3133.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3133 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3123: KAFKA-4935 [WIP]: Deprecate client checksum API an...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3123 KAFKA-4935 [WIP]: Deprecate client checksum API and compute lazy partial checksum for magic v2 You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-4935 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3123.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3123 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3118: MINOR: Broker should disallow downconversion of tr...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3118 MINOR: Broker should disallow downconversion of transactional/idempotent records You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka disallow-transactional-idempotent-downconversion Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3118.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3118 commit b1ea146b167668cdbbeadc41b88874167eb59feb Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-22T23:02:44Z MINOR: Broker should disallow downconversion of transactional/idempotent records --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3113: KAFKA-5186: Avoid expensive log scan to build prod...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3113 KAFKA-5186: Avoid expensive log scan to build producer state when upgrading You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5186 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3113.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3113 commit 90b4051e7984b5611fa8e2395fa3d5643b3c1f0b Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-22T00:51:04Z KAFKA-5186: Avoid expensive log scan to build producer state when upgrading --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3089: KAFKA-5268: Fix bounce test transient failure by c...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3089 KAFKA-5268: Fix bounce test transient failure by clearing partitions before writing Complete state to transaction log You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5268 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3089.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3089 commit b3ee89dc88f8a05173481fdff3f71a8656b4ff9e Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-18T16:50:04Z KAFKA-5268: Fix bounce test transient failure by clearing partitions before writing Complete state to transaction log --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3084: TRIVIAL: Remove redundant asMap utility in Consume...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3084 TRIVIAL: Remove redundant asMap utility in ConsumerProtocol You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka trivial-remove-redundant-utility Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3084.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3084 commit 5f43586cbba99f2df79af26cc31b0cb01741f9ee Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-18T06:03:12Z TRIVIAL: Remove redundant asMap utility in ConsumerProtocol --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3081: MINOR: Log transaction metadata state transitions ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3081 MINOR: Log transaction metadata state transitions plus a few cleanups You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-add-txn-transition-logging Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3081.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3081 commit a68f04245a4bb192a0f0ea258b803f7bfd544ad0 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-18T00:46:14Z MINOR: Log transaction metadata state transitions plus a few cleanups --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3075: KAFKA-5259 [WIP]: TransactionalId auth implies Pro...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3075 KAFKA-5259 [WIP]: TransactionalId auth implies ProducerId auth You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5259-FIXED Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3075.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3075 commit 18e9cd7140a586af78c934092fb55cb71b4428ef Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-16T20:20:04Z KAFKA-5259: TransactionalId auth implies ProducerId auth --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3064: KAFKA-5252: Fix transient failures LogCleanerTest ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3064 KAFKA-5252: Fix transient failures LogCleanerTest testCommitMarkerRemoval and testAbortMarkerRemoval You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5252 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3064.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3064 commit 2e9a259b5780aaf88794411dec60d7b953368ae3 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-16T02:15:59Z KAFKA-5252: Fix transient failures LogCleanerTest testCommitMarkerRemoval and testAbortMarkerRemoval --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3060: KAFKA-5249: Fix incorrect producer snapshot offset...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3060 KAFKA-5249: Fix incorrect producer snapshot offsets when recovering segments You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5249 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3060.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3060 commit 5018fcdca321c322850f8d7bddfbc503a7dda8a2 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-15T19:19:42Z KAFKA-5249: Fix incorrect producer snapshot offsets when recovering segments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3058: KAFKA-5248: Remove unused/unneeded retention time ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3058 KAFKA-5248: Remove unused/unneeded retention time in TxnOffsetCommitRequest You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5248 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3058.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3058 commit 9290f30b60d1264697494aaacda2d485ee6b237c Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-15T18:13:11Z KAFKA-5248: Remove unused/unneeded retention time in TxnOffsetCommitRequest --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3040: MINOR: Some minor improvements to TxnOffsetCommit ...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3040 MINOR: Some minor improvements to TxnOffsetCommit handling You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka txn-offset-commit-cleanups Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3040.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3040 commit 01a02cbfbbea991cca233e1951a5d7f8677da208 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-13T02:58:41Z MINOR: Some minor cleanups from TxnOffsetCommit patch --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #3008: KAFKA-5196: Make LogCleaner transaction-aware
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3008 KAFKA-5196: Make LogCleaner transaction-aware You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5196 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3008.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3008 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #2997: MINOR: Rename InitPidRequest/InitPidResponse renam...
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/2997 MINOR: Rename InitPidRequest/InitPidResponse renamed to InitProducerIdRequest/InitProducerIdResponse You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-rename-initpid Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/2997.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2997 commit d3f4aa6aaece6c46779749f3305a94a0c94e7be1 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-08T17:44:40Z MINOR: Rename InitPidRequest/InitPidResponse renamed to InitProducerIdRequest/InitProducerIdResponse --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] kafka pull request #2933: MINOR: Some cleanup in the transactional producer
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/2933 MINOR: Some cleanup in the transactional producer You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-transactional-client-cleanup Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/2933.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2933 commit 24d2538c585d79b2879f69113e1b09cf8dad7ed8 Author: Jason Gustafson <ja...@confluent.io> Date: 2017-04-28T06:09:27Z MINOR: Some cleanup in the transactional producer --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
