Re: Kafka disk recommendation for production cluster
I meant because there is no actual Raid Controller for running Raid on cloud, it is basically Software Based Raid [1]. [1] https://en.wikipedia.org/wiki/RAID#Software-based On Thu, Jul 12, 2018 at 6:29 PM M. Manna wrote: > From all cases I know so far (including RDBMS storage solution), RAID 10 > has been the choice. Also, the replication you are mentioning is the s/w > replication nothing to do with RAID 0 setup. > > > > On 11 July 2018 at 23:59, Ali Nazemian wrote: > > > Thanks. As this proposal is not available for the version of Kafka that > we > > are using, we should go with Raid option, but the question is how is it > > acceptable to use Software Raid0 on a cloud infrastructure? > > > > On Thu, 12 Jul. 2018, 08:02 M. Manna, wrote: > > > > > Dong Lin's KIP - > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 112%3A+Handle+disk+failure+for+JBOD > > > > > > Should give you some ideas. > > > > > > On 11 July 2018 at 14:31, Ali Nazemian wrote: > > > > > > > Hi All, > > > > > > > > I was wondering what the disk recommendation is for Kafka cluster? Is > > it > > > > acceptable to use RAID0 in the case that replication is 3? We are > > running > > > > on a cloud infrastructure and disk failure is addressed at another > > level, > > > > so the chance of single disk failure would be very low. Besides, our > > > > version of Kafka does not fully support JBOD as even a single disk > > > failure > > > > can stop a Kafka broker, so we will not get the actual benefit of > using > > > > JBOD anyway. However, I am not quite sure how software raid acts in > > this > > > > situation as there is no option to use HW Raid on cloud. > > > > > > > > Regards, > > > > Ali > > > > > > > > > > -- A.Nazemian
Re: Kafka disk recommendation for production cluster
>From all cases I know so far (including RDBMS storage solution), RAID 10 has been the choice. Also, the replication you are mentioning is the s/w replication nothing to do with RAID 0 setup. On 11 July 2018 at 23:59, Ali Nazemian wrote: > Thanks. As this proposal is not available for the version of Kafka that we > are using, we should go with Raid option, but the question is how is it > acceptable to use Software Raid0 on a cloud infrastructure? > > On Thu, 12 Jul. 2018, 08:02 M. Manna, wrote: > > > Dong Lin's KIP - > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 112%3A+Handle+disk+failure+for+JBOD > > > > Should give you some ideas. > > > > On 11 July 2018 at 14:31, Ali Nazemian wrote: > > > > > Hi All, > > > > > > I was wondering what the disk recommendation is for Kafka cluster? Is > it > > > acceptable to use RAID0 in the case that replication is 3? We are > running > > > on a cloud infrastructure and disk failure is addressed at another > level, > > > so the chance of single disk failure would be very low. Besides, our > > > version of Kafka does not fully support JBOD as even a single disk > > failure > > > can stop a Kafka broker, so we will not get the actual benefit of using > > > JBOD anyway. However, I am not quite sure how software raid acts in > this > > > situation as there is no option to use HW Raid on cloud. > > > > > > Regards, > > > Ali > > > > > >
Re: Kafka disk recommendation for production cluster
Dong Lin's KIP - https://cwiki.apache.org/confluence/display/KAFKA/KIP-112%3A+Handle+disk+failure+for+JBOD Should give you some ideas. On 11 July 2018 at 14:31, Ali Nazemian wrote: > Hi All, > > I was wondering what the disk recommendation is for Kafka cluster? Is it > acceptable to use RAID0 in the case that replication is 3? We are running > on a cloud infrastructure and disk failure is addressed at another level, > so the chance of single disk failure would be very low. Besides, our > version of Kafka does not fully support JBOD as even a single disk failure > can stop a Kafka broker, so we will not get the actual benefit of using > JBOD anyway. However, I am not quite sure how software raid acts in this > situation as there is no option to use HW Raid on cloud. > > Regards, > Ali >
