Yes, that's how Kafka works - all partitions are read in parallel, but only one
consumer form the same consumer group reads a partition at one time (a consumer
may consume multiple partitions, but no two consumers from same group consume
the same partition)
Virgil.
On 6/28/18, 7:45 PM, "Malik, Shibha (GE Renewable Energy, consultant)"
<shibha.ma...@ge.com> wrote:
You mean we use multiple partitions for a topic say topic is event
"Customer Account Registration" , we can have multiple consumers read from
different partition at a time ( only one consumer per partition ) and can
perform registration of different customers in parallel ?
-----Original Message-----
From: Virgil Palanciuc [mailto:virg...@adobe.com.INVALID]
Sent: Thursday, June 28, 2018 1:05 AM
To: users@kafka.apache.org
Subject: EXT: Re: [External] Re: Is order of data is not maintained in
Kafka, Is Kafka not suitable to do manage State / Transactional Scenarios such
as Updating a Bank Account scenarios etc
Messages within the same partition are ordered. You don't need to use only
one partition (unless you need global ordering) - you just need to use keys.
E.g. if your key is the account number, then all operations done on the same
account are ordered; if your key is the customer ID, all operations done by the
same customer are ordered; etc. Just find a suitable key.
Virgil.
On 6/28/18, 4:12 AM, "Malik, Shibha (GE Renewable Energy, consultant)"
<shibha.ma...@ge.com> wrote:
But then restricting a consumer to use only partition seems to be
similar to traditional message queues ( except the persistence feature ) and
does not offer parallelism , isn’t it ?
-----Original Message-----
From: Yi, Gene [mailto:gene...@sc.com.INVALID]
Sent: Wednesday, June 27, 2018 5:43 PM
To: users@kafka.apache.org
Subject: EXT: RE: [External] Re: Is order of data is not maintained in
Kafka, Is Kafka not suitable to do manage State / Transactional Scenarios such
as Updating a Bank Account scenarios etc
I think you may use one partition only?
______________________________________________________________
Gene Huijian Yi
TS - FM - App
Standard Chartered Global Business Services Co., Ltd.
Phone: +862259806930
Address: Standard Chartered Center, 3/F No.35 Xinhuanbei Road,
TEDA, Tianjin - 300457
E-mail: gene...@sc.com
______________________________________________________________
Please consider the environment before printing this email
-----Original Message-----
From: Tushar Madhukar [mailto:tushar.madhu...@gmail.com]
Sent: Thursday, June 28, 2018 7:45 AM
To: users@kafka.apache.org
Subject: [External] Re: Is order of data is not maintained in Kafka, Is
Kafka not suitable to do manage State / Transactional Scenarios such as
Updating a Bank Account scenarios etc
Hi,
Kafka guarantees ordering only within a partition (not across
partitions in a topic) ie consumers reading off a partition will always get
records in the order they were *received into the partition*.
Note that in some cases of retries by a non- idempotent producer, the
order in which a producer sends data can be different from the order in which
they are received into the partitions. An idempotent producer can be used to
avoid this scenario.
On Thu., 28 Jun. 2018, 8:33 am Malik, Shibha (GE Renewable Energy,
consultant), <shibha.ma...@ge.com> wrote:
> Is order of data is not maintained in Kafka, Is Kafka not suitable to
> do manage State / Transactional Scenarios such as Updating a Bank
> Account scenarios etc
>
This email and any attachments are confidential and may also be
privileged. If you are not the intended recipient, please delete all copies and
notify the sender immediately. You may wish to refer to the incorporation
details of Standard Chartered PLC, Standard Chartered Bank and their
subsidiaries at
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.sc.com%2Fen%2Four-locations&data=02%7C01%7Cvirgilp%40adobe.com%7C0166adb561584ccb057f08d5dc944573%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C1%7C636657451735238847&sdata=cEcwatemcNjhYxixWObUK31MkSaNyes1LOpjrZMPrJk%3D&reserved=0.
Please refer to
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.sc.com%2Fen%2Fprivacy-policy.html&data=02%7C01%7Cvirgilp%40adobe.com%7C0166adb561584ccb057f08d5dc944573%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C1%7C636657451735238847&sdata=vvInBt3TVOWqe1mMImHCSK3pVSRehR0xl3GZTe5FpGk%3D&reserved=0
for Standard Chartered Bank’s Privacy Policy.
