[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540793#comment-14540793
]
Jiangjie Qin commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch,
KAFKA-1660_2015-04-10_15:08:54.patch, KAFKA-1660_2015-04-16_11:35:37.patch,
KAFKA-1660_2015-04-20_17:38:22.patch, KAFKA-1660_2015-04-29_16:58:43.patch,
KAFKA-1660_2015-04-29_17:37:51.patch, KAFKA-1660_2015-05-12_14:28:53.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14526902#comment-14526902
]
Jiangjie Qin commented on KAFKA-1660:
-
[~jkreps] [~jjkoshy] ping for a review :)
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch,
KAFKA-1660_2015-04-10_15:08:54.patch, KAFKA-1660_2015-04-16_11:35:37.patch,
KAFKA-1660_2015-04-20_17:38:22.patch, KAFKA-1660_2015-04-29_16:58:43.patch,
KAFKA-1660_2015-04-29_17:37:51.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14526939#comment-14526939
]
Jiangjie Qin commented on KAFKA-1660:
-
Thanks for the quick response, [~jkreps].
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch,
KAFKA-1660_2015-04-10_15:08:54.patch, KAFKA-1660_2015-04-16_11:35:37.patch,
KAFKA-1660_2015-04-20_17:38:22.patch, KAFKA-1660_2015-04-29_16:58:43.patch,
KAFKA-1660_2015-04-29_17:37:51.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14520141#comment-14520141
]
Jiangjie Qin commented on KAFKA-1660:
-
Hi [~jkreps], I prefer the counter as well because it has less dependency on
the lock implementation (I think in the flush() call RB you mentioned that we
want to avoid unnecessary dependency in general). And it solves the nested lock
also. I'll submit a patch using counter.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch,
KAFKA-1660_2015-04-10_15:08:54.patch, KAFKA-1660_2015-04-16_11:35:37.patch,
KAFKA-1660_2015-04-20_17:38:22.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14520341#comment-14520341
]
Joel Koshy commented on KAFKA-1660:
---
(There also needs to be a {{batch.done()}} in the if branch. However, after
further discussion we see a number of subtle issues with the above so we can
abandon it and think it is better to just go with one of the other two
approaches.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch,
KAFKA-1660_2015-04-10_15:08:54.patch, KAFKA-1660_2015-04-16_11:35:37.patch,
KAFKA-1660_2015-04-20_17:38:22.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14520557#comment-14520557
]
Jiangjie Qin commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch,
KAFKA-1660_2015-04-10_15:08:54.patch, KAFKA-1660_2015-04-16_11:35:37.patch,
KAFKA-1660_2015-04-20_17:38:22.patch, KAFKA-1660_2015-04-29_16:58:43.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14520619#comment-14520619
]
Jiangjie Qin commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch,
KAFKA-1660_2015-04-10_15:08:54.patch, KAFKA-1660_2015-04-16_11:35:37.patch,
KAFKA-1660_2015-04-20_17:38:22.patch, KAFKA-1660_2015-04-29_16:58:43.patch,
KAFKA-1660_2015-04-29_17:37:51.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518314#comment-14518314
]
Jay Kreps commented on KAFKA-1660:
--
Here was the source of my paranoia about r/w locks compared to counters:
http://blog.takipi.com/java-8-stampedlocks-vs-readwritelocks-and-synchronized/
http://blog.takipi.com/java-8-longadders-the-fastest-way-to-add-numbers-concurrently/
Now they have added some new StampedLock that is meant to be a faster
replacement but is only in jdk 8.
This may not be applicable in the case where writes are really really rare
which may be why we don't see it in our perf testing.
My concern about RecordAccumulator complexity remains (basically nested locks
like that always seem to bite us). I'm not sure I have a better solution there
though. Let me think a bit and see if I can come up with anything.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch,
KAFKA-1660_2015-04-10_15:08:54.patch, KAFKA-1660_2015-04-16_11:35:37.patch,
KAFKA-1660_2015-04-20_17:38:22.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518484#comment-14518484
]
Jiangjie Qin commented on KAFKA-1660:
-
Thanks for the reference, Jay. Totally agreed. We did not see performance
degrade because the write lock is only grabbed once in the entire life cycle.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch,
KAFKA-1660_2015-04-10_15:08:54.patch, KAFKA-1660_2015-04-16_11:35:37.patch,
KAFKA-1660_2015-04-20_17:38:22.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504039#comment-14504039
]
Jiangjie Qin commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch,
KAFKA-1660_2015-04-10_15:08:54.patch, KAFKA-1660_2015-04-16_11:35:37.patch,
KAFKA-1660_2015-04-20_17:38:22.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498469#comment-14498469
]
Jiangjie Qin commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch,
KAFKA-1660_2015-04-10_15:08:54.patch, KAFKA-1660_2015-04-16_11:35:37.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14490445#comment-14490445
]
Jiangjie Qin commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch,
KAFKA-1660_2015-04-10_15:08:54.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486022#comment-14486022
]
Jiangjie Qin commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch, KAFKA-1660_2015-04-08_14:01:12.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484498#comment-14484498
]
Jiangjie Qin commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch,
KAFKA-1660_2015-04-07_18:18:40.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390958#comment-14390958
]
Jiangjie Qin commented on KAFKA-1660:
-
[~jkreps] Ping to see if you could help review the patch when you get a chance?
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384895#comment-14384895
]
Jiangjie Qin commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch, KAFKA-1660_2015-03-27_16:35:42.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380359#comment-14380359
]
Jiangjie Qin commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch,
KAFKA-1660_2015-03-25_10:55:42.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369878#comment-14369878
]
Steven Zhen Wu commented on KAFKA-1660:
---
when this patch is merged, would it be possible to have a patch release like
0.8.2.2? we hit a thread lock-up problem caused by this issue.
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0x0002994dae30 (a org.apache.kafka.common.utils.KafkaThread)
at java.lang.Thread.join(Thread.java:1281)
- locked 0x0002994dae30 (a org.apache.kafka.common.utils.KafkaThread)
at java.lang.Thread.join(Thread.java:1355)
at org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:422)
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353478#comment-14353478
]
Jiangjie Qin commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch, KAFKA-1660_2015-03-09_12:56:39.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352515#comment-14352515
]
Jiangjie Qin commented on KAFKA-1660:
-
Created reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352525#comment-14352525
]
Jiangjie Qin commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/31850/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jiangjie Qin
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660.patch,
KAFKA-1660_2015-02-17_16:41:19.patch, KAFKA-1660_2015-03-02_10:41:49.patch,
KAFKA-1660_2015-03-08_21:14:50.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351186#comment-14351186
]
Jiangjie Qin commented on KAFKA-1660:
-
Thanks for the quick response [~parth.brahmbhatt], I'll take this over as
mirror maker would need this close call for no data loss.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Parth Brahmbhatt
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660_2015-02-17_16:41:19.patch,
KAFKA-1660_2015-03-02_10:41:49.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351159#comment-14351159
]
Jiangjie Qin commented on KAFKA-1660:
-
[~parth.brahmbhatt] Have you got a chance to work on a KIP for this?
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Parth Brahmbhatt
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660_2015-02-17_16:41:19.patch,
KAFKA-1660_2015-03-02_10:41:49.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351179#comment-14351179
]
Parth Brahmbhatt commented on KAFKA-1660:
-
No and I wont be able to do it for probably next couple of weeks. If this is a
pressing issue and someone else wants to take over the jira, please feel free
to do so.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Parth Brahmbhatt
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660_2015-02-17_16:41:19.patch,
KAFKA-1660_2015-03-02_10:41:49.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345918#comment-14345918
]
Guozhang Wang commented on KAFKA-1660:
--
+1 on creating a new KIP. Especially, I think Jiangjie has some comments about
interacting with flush() that we may need some more discussion.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Parth Brahmbhatt
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660_2015-02-17_16:41:19.patch,
KAFKA-1660_2015-03-02_10:41:49.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344481#comment-14344481
]
Jay Kreps commented on KAFKA-1660:
--
[~parth.brahmbhatt] Yeah this was exactly what I was thinking. It would be good
to add some tests for it and kick off the KIP discussion.
[~guozhang] It looks to me like this should work if called from within a
Callback, but I think you guys would have to specifically try that case or add
a unit test for it. It would be good if you guys can do a pass on the code
review once there are some tests.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Parth Brahmbhatt
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660_2015-02-17_16:41:19.patch,
KAFKA-1660_2015-03-02_10:41:49.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14343552#comment-14343552
]
Parth Brahmbhatt commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/29467/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Parth Brahmbhatt
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660_2015-02-17_16:41:19.patch,
KAFKA-1660_2015-03-02_10:41:49.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14335546#comment-14335546
]
Guozhang Wang commented on KAFKA-1660:
--
Parth, I am wondering if this new API can be triggered by the ioThread as well,
i.e. if some fatal exception gets thrown in the callback can I use close(0) to
immediately abort the whole producer process inside the callback?
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Parth Brahmbhatt
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660_2015-02-17_16:41:19.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325210#comment-14325210
]
Parth Brahmbhatt commented on KAFKA-1660:
-
[~jkreps] I have updated the review with my understanding of your last
statement. If that is what you mean I can add unit tests.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Parth Brahmbhatt
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660_2015-02-17_16:41:19.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325207#comment-14325207
]
Parth Brahmbhatt commented on KAFKA-1660:
-
Updated reviewboard https://reviews.apache.org/r/29467/diff/
against branch origin/trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jun Rao
Fix For: 0.8.3
Attachments: KAFKA-1660.patch, KAFKA-1660_2015-02-17_16:41:19.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14317171#comment-14317171
]
Parth Brahmbhatt commented on KAFKA-1660:
-
I am not aware of any clean way to force shutdown a running java thread, how do
you propose to do that in the second approach?
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jun Rao
Fix For: 0.8.3
Attachments: KAFKA-1660.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14317335#comment-14317335
]
Jay Kreps commented on KAFKA-1660:
--
Yeah I'm not proposing calling thread.stop(), we would stop the thread by
sending it a message somehow to stop processing and then it exists without
waiting for all messages to be sent. Basically the same way we implement
close() without a timeout (which also doesn't call thread.stop).
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2.0
Reporter: Andrew Stein
Assignee: Jun Rao
Fix For: 0.8.3
Attachments: KAFKA-1660.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311699#comment-14311699
]
Jay Kreps commented on KAFKA-1660:
--
This is very similar to KAFKA-1659 and KAFKA-1934.
We should do the discussion on the new API as a KIP.
I think there are two possible APIs that could be added.
The first variant attempts to close the producer but will only wait for up to
the given period of time. It returns whether the close succeeded or not.
{code}
public boolean tryClose(long timeout, TimeUnit unit);
{code}
One challenge here is how to stop sends on other threads. The current
implementation will actually throw an exception if you call send after you call
close. But in this variant I guess the idea is that if tryClose fails you would
want to have a non-closed producer instance so we would have to rethink that.
I'm not sure actually how solvable that is.
The second variant always closes the producer and attempts to do this
gracefully, waiting for all sends to complete, but if graceful sending doesn't
work by the time the timeout expires it just shuts down the sender thread and
exits:
{code}
public void close(long timeout, TimeUnit unit);
{code}
I actually think the second case is more usable---I think in most cases what
you want is to try to get rid of the messages you have for a good period of
time but not block forever. I think users of the first api would generally not
have any good recourse when tryClose failed and would just end up leaking the
connections and i/o thread.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2
Reporter: Andrew Stein
Assignee: Jun Rao
Fix For: 0.8.3
Attachments: KAFKA-1660.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14260510#comment-14260510
]
Parth Brahmbhatt commented on KAFKA-1660:
-
Created reviewboard https://reviews.apache.org/r/29467/diff/
against branch trunk
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2
Reporter: Andrew Stein
Assignee: Jun Rao
Fix For: 0.8.3
Attachments: KAFKA-1660.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14260511#comment-14260511
]
Parth Brahmbhatt commented on KAFKA-1660:
-
https://reviews.apache.org/r/29467/ patch available.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2
Reporter: Andrew Stein
Assignee: Jun Rao
Fix For: 0.8.3
Attachments: KAFKA-1660.patch
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14154809#comment-14154809
]
Andrew Stein commented on KAFKA-1660:
-
Hi [~jkreps],
I don't think that this (the request for {{producer.close(timeout)}}) and
KAFKA-1659 (the request for {{producer.abort()}}) are the same.
If {{producer.close(timeout)}} were to return without actually closing the
producer in time, the user could then continue with {{producer.abort()}}.
However, this would not be the only case where {{producer.abort()}} would be
useful. It could be used if a {{producer.send()}} were to fail or if a
{{future.get()}} were to fail as well.
In addition if {{producer.close()}} were to return without actually closing the
producer in time, the user could continue with other logic besides
{{producer.abort()}}.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2
Reporter: Andrew Stein
Assignee: Jun Rao
Fix For: 0.8.2
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14154882#comment-14154882
]
Jay Kreps commented on KAFKA-1660:
--
Gotcha, so if I understand close(time) is effectively tryClose(time) and
attempts to close and then gives up if it can't, and abort is effectively
forceClose() and forcefully kills the I/O thread.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2
Reporter: Andrew Stein
Assignee: Jun Rao
Fix For: 0.8.2
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.
[
https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14155625#comment-14155625
]
Andrew Stein commented on KAFKA-1660:
-
Yes, {{close(timeout)}} is {{tryClose(timeout)}}.
On the other hand {{abort()}} does not even need to try to {{forceClose()}}. It
kills the I/O thread. hopefully in a clean manner, perhaps by posting an
abortFlag to the sender. The sender can check for this flag in the two loops
of its {{run()}} -- This is just a suggestion.
Ability to call close() with a timeout on the Java Kafka Producer.
---
Key: KAFKA-1660
URL: https://issues.apache.org/jira/browse/KAFKA-1660
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 0.8.2
Reporter: Andrew Stein
Assignee: Jun Rao
Fix For: 0.8.2
I would like the ability to call {{close}} with a timeout on the Java
Client's KafkaProducer.
h6. Workaround
Currently, it is possible to ensure that {{close}} will return quickly by
first doing a {{future.get(timeout)}} on the last future produced on each
partition, but this means that the user has to define the partitions up front
at the time of {{send}} and track the returned {{future}}'s
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
