@clebertsuconic But even if is a personal request, IMHO would be really great to have a separate (open source of course!) project that aims to provide a journal as a first class citizen, maybe with pluggable features like different binary encoders/decoders (SBE, ProtoBuf, FlatBuffers..), replication/compaction/paging and using different stores (redis, SQL, cassandra, elasticsearch, Kafka)...the possibilities could be infinite, wdyt?
> when you do sync=false, is that actually writing to the > disk, or it's just putting it in memory until you call force or close > the disk? AFAIK it is really writing to the disk, but asynchronously and using the OS configuration for it (on linux: numa balancing, swappiness, dirty_expire_centisecs, dirty_ratio, dirty_writeback_centisecs and many more). > I'm not arguing how safe is to use either, but on my crash tests the > messages weren't lost if I waited some time before killing the server. Cool! Effectively It's guaranteed to be safe with process failure, indeed AFAIK pretty almost any OS that provides mmap ensures that every writes performed on mapped memory will be persisted on physical storage even if the owner process will crash! The only way to force a mmap file to lose data is with power failures/OS hard-crashes (that sadly Murphy's law said that will happen for sure eheh). > IAIO with no-sync probably has a higher level of guarantees of being >> actually on the disk in case of crashes.. but if the user don't need >> syncs, that's a rock & roll possibility. Yes, i think the same, it has a good balance between reliability (on process crashes, but on the cloud it could be enough :P) and speed but AIO is unbeatable from this POV: if the write is reported as performed it is no way on the disk, even on OS failures! Another 2 points of difference will be memory and cpu usage: the writes you're seeing are flushed by the OS on the disk in background, using the virtual memory directly as a buffer (but on lAIO there is no need to use it!); indeed the CPU usage will be, for high writes rate, a little higher than the lAIO version, due to this async flushes, right? > Take a look on my branch, mapped at my fork Thanks!For sure!!! -- View this message in context: http://activemq.2283324.n4.nabble.com/DISCUSS-Memory-Mapped-Journal-Type-tp4721494p4721560.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
