Thanks a lot Bruno, much clearer now.
It's only my opinion but since the Topology is a concept of the API as
well as the repartitioning logic, for me also this mechanism should be a
bit more transparent, but it aslo maybe that I'm plain wrong here :)
Thanks !
On Thu, May 14, 2020 at 9:24 PM
Yeah, the current API doesn't make it very clear how to do it. You can
set an in-memory like this:
> builder.globalTable("topic",
> Materialized.as(Stores.inMemoryKeyValueStore("store-name")));
We are already working on an improved API via KIP-591:
Hi Raffaele,
Change is an internal class in Streams and also its SerDes are
internal. To consume the repartition topic you mention outside of
Streams you would need to use those internal classes (note: I've never
tried this). Those classes can change at any time. So consuming from
repartition
Hi Bruno,
Also when you mention:
The record structure key, (oldValue, newValue) is called Change in
Kafka Streams and it is used where updates are emitted downstream
Does it also mean the same happen when we convert a KTable to a KStream ?
Do you know any docs or article about this topics?
Hi Bruno,
Thanks,
One more thing, As I told you I was consuming the repartitioning topic
created by group by
and I just saw the old and new value, as you are telling me now they are
indeed marked as old and new,
is this mark visible somehow consuming the repartitioning topic ?
Raffaele
On Thu,
Hi Guys,
I am using the mode *bulk *and poll.interval.ms *10* in the Source
connector configuration.
But I don't need to load data another time.?
I need to load the data only once ??
How can I able to do this ?
You could use Mirror Maker 1, it's a basic "consume from topic A in cluster
X, produce to topic A in cluster Y" app.
On Fri, 15 May 2020, 12:13 am JP MB, wrote:
> Hi guys,
> My use case is simply copying data from one Kafka to another. When
> searching on google, the immediate answer seems to
I m trying to better understand KTable and I have encountered a behaviour I
cannot wrap my mind around it.
So* groupByKey()* can only be applied to KStream and not to KTable, that's
because of the nature of KTable that and its UPSERT logic.
What I don't understand correctly and therefore ask your
When clients connect the Kafka broker assigns each client a subset of the topic
partitions. When a client becomes unresponsive then the broker assigns those
partitions to another client.
The state of your connector task isn't going to be transferred but another
connector task will eventually
Why not use autoincrement? It'll only emit new records on subsequent polls
then.
On Thu, 14 May 2020, 11:15 pm vishnu murali,
wrote:
> Hi Guys,
>
> I am using the mode *bulk *and poll.interval.ms *10* in the Source
> connector configuration.
>
> But I don't need to load data another time.?
Hi Kafka community,
we are currently using kafka client version 1.1.0 in our production. I'm
working on upgrade it to kafka version 2.0.0.
I see there are some metrics which are present in version 1.1.1 are removed
in kafka version 2.0.0.
S# version 1.1.1 2.0.0
1
Thanks Liam
But I am asking like assume I am having 10.
Using JDBC source I need to push that once..
No more additional data will be added in future in that table.
In that case i need to push that only once not more than one...
For this scenario I am asking!!
On Thu, May 14, 2020, 19:20
Yes, I saw that it would fit my use case but that brings another question.
Is MM1 deprecated or it will be discontinued ?
Regards
Em qui., 14 de mai. de 2020 às 14:46, Liam Clarke-Hutchinson <
liam.cla...@adscale.co.nz> escreveu:
> You could use Mirror Maker 1, it's a basic "consume from topic
Hi guys,
My use case is simply copying data from one Kafka to another. When
searching on google, the immediate answer seems to be Mirror Maker, so we
jumped to the most recent version MM2.
The thing is I don't want active/active replication, the consumers from
cluster A will be different from the
If you just want it once then delete the connector once it's processed all
the data
--
Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff
On Thu, 14 May 2020 at 16:14, vishnu murali
wrote:
> Thanks Liam
>
> But I am asking like assume I am having 10.
>
> Using
I'm working through a kafka implementation. I'm having issues with the
ssl.principal.mapping.rules configuration.
I've successfully started up Kafka with the SSL encrypt/authenticate in place
and I've successfully set up the super user using the full principal name. The
issue arises when I
Hello,
Since we upgraded from Kafka 2.3 to 2.4 we have big amount logs telling:
[2020-05-14 19:34:27,623] ERROR [ReplicaManager broker=1] Error processing
append operation on partition ourtopic-1 (kafka.server.ReplicaManager)
org.apache.kafka.common.InvalidRecordException: Inner record
Hi Raffaele,
In your example, Kafka Streams would send the new and the old value
downstream. More specifically, the groupBy() would send (as you also
observed)
London, (old value: London, new value: null)
Berlin, (old value: null, new value: Berlin)
At the count() record London, (old value:
18 matches
Mail list logo
