On 10/27/15 12:34, Elias Levy wrote: > Anyone using Kafka with Brtfs successfully? Any recommendations against > taking that path? >
I would suggest avoiding BTRFS for any production systems, especially if you have high data integrity requirements. I know Linux distro's are starting to adopt it, but IMHO (and personal experience) I would not deem it production ready. If you are looking for a filesystem that supports many of the features (and more) of BTRFS I would take a close look at ZFS. It is a much more mature (and stable) codebase, and offers some features that could be beneficial for kafka (transparent filesystem compression, ability to cache read and write datasets on SSD storage, etc.). I ran some tests locally of a kafka broker using CentOS-6.x and XFS (which is what I run in production and consider to be a very stable filesystem) against FreeBSD-10.1-RELEASE and ZFS. Our ZFS system tended to outperform the XFS system in my local kafka consumer/producer tests - and performed better in synthetic filesystem benchmarks. Both systems were using OpenJDK-8.x, and were on the same hardware. If you do not want to live way outside of the sandbox there is ZFS support for Linux here: http://zfsonlinux.org/ http://open-zfs.org/wiki/Main_Page Hope this helps - and doesn't start a huge bikeshed discussion :) Cheers, -pete -- Pete Wright Lead Systems Architect Rubicon Project pwri...@rubiconproject.com 310.309.9298
