[jira] [Created] (KAFKA-6161) Make configure() and close() empty by default in serde interfaces
Evgeny Veretennikov created KAFKA-6161:
--
Summary: Make configure() and close() empty by default in serde
interfaces
Key: KAFKA-6161
URL: https://issues.apache.org/jira/browse/KAFKA-6161
Project: Kafka
Issue Type: Improvement
Components: clients
Reporter: Evgeny Veretennikov
Assignee: Evgeny Veretennikov
Priority: Normal
{{Serializer}}, {{Deserializer}} and {{Serde}} interfaces have methods
{{configure()}} and {{close()}}. Pretty often one want to leave these methods
empty. For example, a lot of serializers inside
{{org.apache.kafka.common.serialization}} package have these methods empty:
{code}
@Override
public void configure(Map configs, boolean isKey) {
// nothing to do
}
@Override
public void close() {
// nothing to do
}
{code}
Since we switched to source level 1.8 for javac, we can make these methods
empty by default.
As another option, we may create new interfaces (like
{{UnconfiguredSerializer}}), in which we will define these methods empty, but
just making methods default seems more convenient.
As a side note, making methods default shouldn't be binary incompatible. So,
seems like it's safe to make this change.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KAFKA-3823) QuickStart documentation is still showing MirrorMakers supports more than one consumer.config
[
https://issues.apache.org/jira/browse/KAFKA-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Veretennikov resolved KAFKA-3823.
Resolution: Fixed
Closed issue, as [~tombentley] suggested.
> QuickStart documentation is still showing MirrorMakers supports more than one
> consumer.config
> -
>
> Key: KAFKA-3823
> URL: https://issues.apache.org/jira/browse/KAFKA-3823
> Project: Kafka
> Issue Type: Bug
> Components: website
>Affects Versions: 0.9.0.0
>Reporter: Chak Lee
>Priority: Minor
>
> On the official QuickStart documentation, the MirrorMaker section is still
> showing the following example:
> {code}
> bin/kafka-mirror-maker.sh
>--consumer.config consumer-1.properties --consumer.config
> consumer-2.properties
>--producer.config producer.properties --whitelist my-topic
> {code}
> However, the support for this is already dropped in KAFKA-1650. If you
> tried to run the above script, you will get the following error:
> {code}
> [2016-06-10 18:35:11,201] ERROR Exception when starting mirror maker.
> (kafka.tools.MirrorMaker$)
> joptsimple.MultipleArgumentsForOptionException: Found multiple arguments for
> option consumer.config, but you asked for only one
> {code}
> Please update the website's QuickStart section for MirrorMakers. Thanks.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KAFKA-5529) ConsoleProducer uses deprecated BaseProducer
Evgeny Veretennikov created KAFKA-5529: -- Summary: ConsoleProducer uses deprecated BaseProducer Key: KAFKA-5529 URL: https://issues.apache.org/jira/browse/KAFKA-5529 Project: Kafka Issue Type: Improvement Components: tools Reporter: Evgeny Veretennikov Assignee: Evgeny Veretennikov Priority: Minor BaseProducer is deprecated, should use org.apache.kafka.clients.producer.KafkaProducer instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KAFKA-5511) ConfigDef.define() overloads take too many parameters
Evgeny Veretennikov created KAFKA-5511:
--
Summary: ConfigDef.define() overloads take too many parameters
Key: KAFKA-5511
URL: https://issues.apache.org/jira/browse/KAFKA-5511
Project: Kafka
Issue Type: Improvement
Components: clients
Reporter: Evgeny Veretennikov
Priority: Minor
Builder pattern can be helpful to get rid of all these {{define()}} overloads.
I think, it's better to create some {{ConfigKeyBuilder}} class to construct
keys.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KAFKA-5507) kafka-run-class.sh doesn't check for empty $CLASSPATH
Evgeny Veretennikov created KAFKA-5507:
--
Summary: kafka-run-class.sh doesn't check for empty $CLASSPATH
Key: KAFKA-5507
URL: https://issues.apache.org/jira/browse/KAFKA-5507
Project: Kafka
Issue Type: Bug
Components: tools
Affects Versions: 0.10.2.1
Environment: Linux
Reporter: Evgeny Veretennikov
Assignee: Evgeny Veretennikov
Priority: Minor
Steps to reproduce problem:
1. Clone Kafka repo using {{git clone}}.
2. Run some Bash-script, which invokes {{kafka-run-class.sh}} (without building
project beforehand).
For example:
{{$ ./bin/zookeeper-server-start.sh config/zookeeper.properties
Error: Could not find or load main class config.zookeeper.properties}}
Error is misleading. This happens, because {{kafka-run-class.sh}} tries to run
this command:
{{java ...a lot of JVM options... -cp
org.apache.zookeeper.server.quorum.QuorumPeerMain config/zookeeper.properties}}
Script doesn't check for empty $CLASSPATH variable. It's empty, because we
didn't build project.
{{kafka-run-class.bat}} script for Windows checks for it:
{{IF ["%CLASSPATH%"] EQU [""] (
echo Classpath is empty. Please build the project first e.g. by running
'gradlew jarAll'
EXIT /B 2
)}}
I think, we should add similar check to Bash-script, and show similar error.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KAFKA-5245) KStream builder should capture serdes
[ https://issues.apache.org/jira/browse/KAFKA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049523#comment-16049523 ] Evgeny Veretennikov commented on KAFKA-5245: [~guozhang], thanks for explanation! I guess, we should remove `beginner` and `newbie` tags from this ticket then... > KStream builder should capture serdes > -- > > Key: KAFKA-5245 > URL: https://issues.apache.org/jira/browse/KAFKA-5245 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 0.10.2.0, 0.10.2.1 >Reporter: Yeva Byzek >Assignee: anugrah >Priority: Minor > Labels: beginner, newbie > > Even if one specifies a serdes in `builder.stream`, later a call to > `groupByKey` may require the serdes again if it differs from the configured > streams app serdes. The preferred behavior is that if no serdes is provided > to `groupByKey`, it should use whatever was provided in `builder.stream` and > not what was in the app. > From the current docs: > “When to set explicit serdes: Variants of groupByKey exist to override the > configured default serdes of your application, which you must do if the key > and/or value types of the resulting KGroupedStream do not match the > configured default serdes.” -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5245) KStream builder should capture serdes
[ https://issues.apache.org/jira/browse/KAFKA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048738#comment-16048738 ] Evgeny Veretennikov commented on KAFKA-5245: There are actually a lot of KStream methods, that require serdes again, not only groupByKey(). For example, there are print(), through(), to(), etc. We should pass serdes from builder.stream() to all such methods, am I right? Also, behaviour of methods like groupByKey() will change after solving this ticket. Clients could be broken, in case they really need to use default serdes. Do we need KIP for this ticket? > KStream builder should capture serdes > -- > > Key: KAFKA-5245 > URL: https://issues.apache.org/jira/browse/KAFKA-5245 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 0.10.2.0, 0.10.2.1 >Reporter: Yeva Byzek >Assignee: anugrah >Priority: Minor > Labels: beginner, newbie > > Even if one specifies a serdes in `builder.stream`, later a call to > `groupByKey` may require the serdes again if it differs from the configured > streams app serdes. The preferred behavior is that if no serdes is provided > to `groupByKey`, it should use whatever was provided in `builder.stream` and > not what was in the app. > From the current docs: > “When to set explicit serdes: Variants of groupByKey exist to override the > configured default serdes of your application, which you must do if the key > and/or value types of the resulting KGroupedStream do not match the > configured default serdes.” -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5245) KStream builder should capture serdes
[ https://issues.apache.org/jira/browse/KAFKA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047933#comment-16047933 ] Evgeny Veretennikov commented on KAFKA-5245: Matthias, thanks for explanation! How can I self-assign this ticket? Seems like current assigner [~anukin] isn't active here... > KStream builder should capture serdes > -- > > Key: KAFKA-5245 > URL: https://issues.apache.org/jira/browse/KAFKA-5245 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 0.10.2.0, 0.10.2.1 >Reporter: Yeva Byzek >Assignee: anugrah >Priority: Minor > Labels: beginner, newbie > > Even if one specifies a serdes in `builder.stream`, later a call to > `groupByKey` may require the serdes again if it differs from the configured > streams app serdes. The preferred behavior is that if no serdes is provided > to `groupByKey`, it should use whatever was provided in `builder.stream` and > not what was in the app. > From the current docs: > “When to set explicit serdes: Variants of groupByKey exist to override the > configured default serdes of your application, which you must do if the key > and/or value types of the resulting KGroupedStream do not match the > configured default serdes.” -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5245) KStream builder should capture serdes
[ https://issues.apache.org/jira/browse/KAFKA-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045936#comment-16045936 ] Evgeny Veretennikov commented on KAFKA-5245: There is Kstream.groupByKey() method, which doesn't require serdes. I guess, this ticket should be just closed. > KStream builder should capture serdes > -- > > Key: KAFKA-5245 > URL: https://issues.apache.org/jira/browse/KAFKA-5245 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 0.10.2.0, 0.10.2.1 >Reporter: Yeva Byzek >Assignee: anugrah >Priority: Minor > Labels: beginner, newbie > > Even if one specifies a serdes in `builder.stream`, later a call to > `groupByKey` may require the serdes again if it differs from the configured > streams app serdes. The preferred behavior is that if no serdes is provided > to `groupByKey`, it should use whatever was provided in `builder.stream` and > not what was in the app. > From the current docs: > “When to set explicit serdes: Variants of groupByKey exist to override the > configured default serdes of your application, which you must do if the key > and/or value types of the resulting KGroupedStream do not match the > configured default serdes.” -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KAFKA-5145) Remove task close() call from closeNonAssignedSuspendedTasks method
[ https://issues.apache.org/jira/browse/KAFKA-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045676#comment-16045676 ] Evgeny Veretennikov commented on KAFKA-5145: It seems like ProcessorNode.close() method isn't actually called from StreamThread.suspendTasksAndState(). > Remove task close() call from closeNonAssignedSuspendedTasks method > --- > > Key: KAFKA-5145 > URL: https://issues.apache.org/jira/browse/KAFKA-5145 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 0.10.2.0 >Reporter: Narendra Kumar > Labels: newbie > Attachments: BugTest.java, DebugTransformer.java, logs.txt > > > While rebalancing ProcessorNode.close() can be called twice, once from > StreamThread.suspendTasksAndState() and once from > StreamThread.closeNonAssignedSuspendedTasks(). If ProcessorNode.close() > throws some exception because of calling close() multiple times( i.e. > IllegalStateException from some KafkaConsumer instance being used by some > processor for some lookup), we fail to close the task's state manager ( i.e. > call to task.closeStateManager(true); fails). After rebalance, if the same > task id is launched on same application instance but in different thread then > the task get stuck because it fails to get lock to the task's state directory. > Since processor close() is already called from > StreamThread.suspendTasksAndState() we don't need to call again from > StreamThread.closeNonAssignedSuspendedTasks(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
