Hi Adrien,
I set fetch.max.wait.ms to 1500 ms and ran it again. It still isn't crossing
1150 records per fetch. On the producer side (using kakfa-producer-perf-test),
its producing about 30,000 records/sec and a million records in total.
I tried different configurations amongst these three
Hi Vishnu,
do you have check your fetch.max.wait.ms value ?
it may not be long enough time to wait until you recover your 5000 records ...
maybe just enough time to recover only 1150 records.
fetch.max.wait.ms
By setting fetch.min.bytes, you tell Kafka to wait until it has enough data to
Thank you everyone for your explanations, that's been most enlightening.
On Wed, Jul 18, 2018 at 2:28 AM Matthias J. Sax wrote:
>
> I see -- sorry for miss-understanding initially.
>
> I agree that it would be possible to detect. Feel free to file a Jira
> for this improvement and maybe pick it
Hi,
I am currently working with a single Kafka broker and a single Kafka consumer.
I am trying to get the consumer to fetch more records, so I can increase the
batch size when I write the data to a DB.
Each record is about 1 KB and I am trying to fetch at least 5000 records each
time. So, I
If you connect both stores to both processor, there will be only one
thread for both processors. Thus, a concurrent access can never happen.
-Matthias
On 7/18/18 10:51 AM, Druhin Sagar Goel wrote:
> Hi Matthias,
>
>
> I was under the impression that state stores are not thread safe and so two
Hi all,
We had an outage recently that I think we could have prevented, and I'd
like to get some feedback on the idea.
tl;dr:
When a partition leader writes an updated ISR, it should also record
its current log-end-offset. On leader election, if there are no live
replicas in the ISR, then a
Hi Matthias,
I was under the impression that state stores are not thread safe and so two
processors writing to the same store at the same time would not work. I
understood this from your reply in this post:
https://groups.google.com/forum/#!topic/confluent-platform/JTKyDE231y8. Is that
not
Thanks Dong.
I am a little late, but +1, too.
- verified signatures
- build from sources
- run unit test suite
- run streams quickstart
Thanks for running the release!
-Matthias
On 7/18/18 10:24 AM, Dong Lin wrote:
> Thank you all for taking time to certify and vote for the release!
>
>
Thank you all for taking time to certify and vote for the release!
This vote has passed with 10 +1 votes (3 bindings) and no 0 or -1 votes.
+1 vote from PMC Members:
- Jason Gustafson
- Rajini Sivaram
- Ismael Juma
+1 vote from Committers:
- Sriharsha Chintalapani
- Dong Lin
+1 vote from
You can connect both stores to both processor for this.
-Matthias
On 7/17/18 11:12 PM, Druhin Sagar Goel wrote:
> Hi,
>
> I am new to the Kafka Streams framework. I have the following streams use
> case:
>
> State store A
> State store B
>
> Processor A
> Processor B
>
> State store A is
That is expected behavior. Typically there are multiple kafka brokers and so if
one is down the client retries to send to a newly elected leader.
A send should not be considered successful until an ACK is received in the
client from the kafka cluster.
By default the ACK is async for
While this does not answer your question, I believe during the first call, a
lot of things happen - e.g. get admin and metadata info about the cluster, etc.
That takes "some time" and hence the poll interval that is acceptable/norm for
regular processing may not be sufficient for initialization
+1. Verified the following:
- javadocs
- web docs
- maven staging repository
Besides what Ismael mentioned on upgrade guide, some of the latest doc
fixes in 2.0 seems not be reflected in
http://kafka.apache.org/20/documentation.html yet (this does not need a new
RC, we can just re-copy-and-paste
Thanks Rajini! A documentation issue that we must fix before the release
(but does not require another RC), 1.2 (which became 2.0) is mentioned in
the upgrade notes:
http://kafka.apache.org/20/documentation.html#upgrade
Ismael
On Sun, Jul 15, 2018 at 9:25 AM Rajini Sivaram
wrote:
> Hi Ismael,
+1 (binding)
Verified signature of source artifact, ran tests and verified quickstart on
source artifact with Java 8, verified quickstart on binary artifact (Scala
2.12) with Java 8, sanity checked release notes and Maven staging
repository.
Thanks for managing the release Dong!
Ismael
On Sun,
The asynchronous sending of a message returns no error even if the
Kafka server is not started.
For all the following tests, the local Kafka server is stopped. First,
consider this piece of code:
public static void main(String[] args) throws Exception {
Properties config = new Properties();
Hi,
I have an existing Kafka Cluster that is configured as PLAINTEXT. We want
to enable SASL (GSSAPI) as an additional listener.
Is there a way to force specific topics to only accept traffic (pub/con)
from a certain listener?
e.g. if i create a topic and set ACLS, how do i stop a client from
KafkaConsumer's Javadoc [1] says:
> There is no client-side buffering in read_committed mode.
And it also says:
> timeout - The time, in milliseconds, spent waiting in poll if data
> is not available in the buffer. If 0, returns immediately with any
> records that are available currently in the
I have a 3 node Kafka cluster. And the following properties are set in
server.properties:
offsets.topic.replication.factor=3
default.replication.factor=3
min.insync.replicas=2
In one terminal, I ran the following commands:
bin/kafka-topics --zookeeper localhost:2181 --create
If the timeout is short say 100, the first poll does not return
records for my case. Jay Kreps gave an explanation on [1]. I think
that this behaviour for poll is counterintuitive, it will make Kafka
user's life much easier if this behaviour is documented in [2].
[1]
Hi,
I am new to the Kafka Streams framework. I have the following streams use case:
State store A
State store B
Processor A
Processor B
State store A is only written to by Processor A but also needs to be read by
Processor B. State store B needs to be written to by both Processor A and
21 matches
Mail list logo
