Tomohiro Hashidate created KAFKA-10892:
------------------------------------------
Summary: Add Topology#connectSourceStoreAndTopic as a public method
Key: KAFKA-10892
URL: https://issues.apache.org/jira/browse/KAFKA-10892
Project: Kafka
Issue Type: Improvement
Components: streams
Affects Versions: 2.7.0
Reporter: Tomohiro Hashidate
I want Topology#connectSourceStoreAndTopic.
Because I want to use a topic as a source topic directly without a redundant
changelog topic for not only KeyValueStore but also WindowStore.
This issue is similar to [KAFKA-6840], but is a suggestion for a more general
approach
{code:java}
public synchronized Topology connectSourceStoreAndTopic(final String
sourceStoreName,
final String topic)
{
internalTopologyBuilder.connectSourceStoreAndTopic(sourceStoreName,
topic);
return this;
}
{code}
h3. Background
I want to use a topic as a source topic for WindowStore because using
WindowStore is suitable for the feature that I'm implementing.
The records stored in the topic are aggregated with a time window by another
application. The size of the topic is over 10TB.
I want to use the topic as a source topic for WindowStore directly.
But, I cannot do so on the current interface.
I need a redundant topic only for storing the records into WindowStore.
If this API is public, I can use topics incoming from other applications (not
only Kafka Streams applications) as source topics for any StateStore
implementation without redundant changelog topics.
Of course, I need to implement a processor for storing incoming records in such
a case.
But I think it's not difficult.
Please consider this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
