Thanks a lot for the explanation Peter, sounds like what I thought. I am just not sure I got the last part. So if a disk on such a broker fails, and we have kafka version <1, the whole broker dies ?
What happens when the disk is replaced then ? > On May 10, 2018, at 20:42, Peter Bukowinski <pmb...@gmail.com> wrote: > > Oops, sorry about the name misspelling, Andrian. (spell-check just tried to > correct it again). > >> On Thu, May 10, 2018 at 11:41 AM, Peter Bukowinski <pmb...@gmail.com> wrote: >> >> Adrian, >> >> Replicas are *always* assigned to different brokers. You cannot, for >> example, deploy a single broker with a replication factor of 2 or 3 (with >> min.insync.replicas of 2 or 3, respectively), even with multiple data >> directories. >> >> At the cluster level, kafka is not aware of an individual broker's storage >> topology (single or multiple storage locations). Topic partitions on a >> single, multi-data directory broker are distributed among storage locations >> in a round-robin manner. >> >> In a disk failure scenario, you will only lose one replica of all the >> topic partitions that existed on that disk, assuming you're running 1.0+. >> If you're not running 1.0+, then a single disk failure on a broker >> configured with JBOD will bring down the broker. >> >> Hope this helps, >> >> Peter Bukowinski >> >> On Thu, May 10, 2018 at 1:49 AM, Andrian Jardan <andrianjar...@gmail.com> >> wrote: >> >>> Hello everyone, >>> >>> I was wondering how data is spread across disks when more than 1 data >>> folder is specified on a broker ? >>> >>> I am specifically interested to understand if failure of 3 disks may lead >>> to data loss (with replication factor at 3)? >>> >>> Or is the data replicated so it resides on 3 brokers, and not 3 different >>> data folders ? >>> >>> Thanks ! >>> >>> — >>> Andrian Jardan >>> Infrastructure and DevOps expert >>> cell: +49 174 2815994 >>> Skype: macrosdnb >>> >>> >>
