I assume that you are considering the data loss to be the difference in size between the two directories? This is generally not a good guideline, as the batching and compression will be different between the two replicas.
-Todd On Mon, Jun 22, 2015 at 7:26 AM, Nirmal ram <nirmal106110...@gmail.com> wrote: > Hi, > > I noticed a data loss while storing in kafka logs. > Generally, leader hands the request to followers, is there a data loss in > that process? > > topic 'jun8' with 2 replicas and 8 partitions > > *Broker 1*[user@ jun8-6]$ ls -ltr > total 7337500 > -rw-rw-r-- 1 user user 1073741311 Jun 22 12:45 00000000000015195331.log > -rw-rw-r-- 1 user user 1127512 Jun 22 12:45 00000000000015195331.index > -rw-rw-r-- 1 user user 1073741396 Jun 22 12:48 00000000000016509739.log > -rw-rw-r-- 1 user user 1108544 Jun 22 12:48 00000000000016509739.index > -rw-rw-r-- 1 user user 1073740645 Jun 22 12:52 00000000000017823869.log > -rw-rw-r-- 1 user user 1129064 Jun 22 12:52 00000000000017823869.index > -rw-rw-r-- 1 user user 1073741800 Jun 22 13:17 00000000000019136798.log > -rw-rw-r-- 1 user user 1161152 Jun 22 13:17 00000000000019136798.index > -rw-rw-r-- 1 user user 1073741509 Jun 22 13:21 00000000000020451309.log > -rw-rw-r-- 1 user user 1152448 Jun 22 13:21 00000000000020451309.index > *-rw-rw-r-- 1 user user 1073740588 Jun 22 13:39 00000000000021764229.log* > -rw-rw-r-- 1 user user 1241168 Jun 22 13:39 00000000000021764229.index > -rw-rw-r-- 1 user user 1062343875 Jun 22 13:42 00000000000023077448.log > -rw-rw-r-- 1 user user 10485760 Jun 22 13:42 00000000000023077448.index > [user@ jun8-6]$ > > > > *Broker 2*[user@ jun8-6]$ ls -ltr > total 7340468 > -rw-rw-r-- 1 user user 1073741311 Jun 22 12:45 00000000000015195331.log > -rw-rw-r-- 1 user user 1857144 Jun 22 12:45 00000000000015195331.index > -rw-rw-r-- 1 user user 1073741396 Jun 22 12:48 00000000000016509739.log > -rw-rw-r-- 1 user user 1857168 Jun 22 12:48 00000000000016509739.index > -rw-rw-r-- 1 user user 1073740645 Jun 22 12:52 00000000000017823869.log > -rw-rw-r-- 1 user user 1857752 Jun 22 12:52 00000000000017823869.index > -rw-rw-r-- 1 user user 1073741800 Jun 22 13:17 00000000000019136798.log > -rw-rw-r-- 1 user user 1857440 Jun 22 13:17 00000000000019136798.index > -rw-rw-r-- 1 user user 1073741509 Jun 22 13:21 00000000000020451309.log > -rw-rw-r-- 1 user user 1856968 Jun 22 13:21 00000000000020451309.index > *-rw-rw-r-- 1 user user 1073722781 Jun 22 13:39 00000000000021764229.log* > -rw-rw-r-- 1 user user 1762288 Jun 22 13:39 00000000000021764229.index > -rw-rw-r-- 1 user user 10485760 Jun 22 13:42 00000000000023077448.index > -rw-rw-r-- 1 user user 1062343875 Jun 22 13:42 00000000000023077448.log > [user@ jun8-6]$ >
