I personally don’t mind switching off Chronicle Queue. I have a transformer function to convert the FQL logs to Thrift (don’t judge) and use easy-cas to reply on a cluster… replying FQL from Chronicle Queue was far slower than Thrift and was hard to push the cluster as the client was the bottleneck… switching off it let me actually cause Cassandra to be the bottleneck…
>>> No, there is another perfectly sensible option: just implement a simple >>> serialisation format ourselves. My one issue with this is we need to ask who the target audience is? Trying to add FQL reply to easy-cas was a pain for 2 reasons: Chronicle Queue is slow, custom C* serializers that must be in the class path (that brings a ton of baggage with it)… For me FQL has 2 use cases 1) analytic, what are people actually doing and what are there frequencies? 2) reply In both cases custom serializers are a pain due to baggage they bring and the limiting nature of it… what if I want a Go based FQL reply? I need java code from cassandra-all… I personally favor serializers like protobuf/thrift as they are portable and can be used by users without issues. As for the log format itself… a super simple log format that can be easy to read that is custom is fine by me… I am cool with the log being custom as I don’t know a good portable log format at the top of my head… a simple thing like the following works for me Header: lengths, checksum, etc. Body: std serializer + > On Sep 19, 2024, at 9:14 AM, C. Scott Andreas <sc...@paradoxica.net> wrote: > > Agree with Benedict's proposal here. > > In circumstances when I've needed to capture and work with FQL, I've found it > cumbersome to work with Chronicle. The dial-home functionality and release > process changes put it over the top for me. > > – Scott > >> On Sep 19, 2024, at 8:40 AM, Josh McKenzie <jmcken...@apache.org> wrote: >> >> >>> there is another perfectly sensible option >> My apologies; I wasn't clear. If we choose to continue to use chronicle >> queue, what I enumerated was the only logical option I saw for us. >> >> Altogether I think we should just move away from the library as you've laid >> out here Benedict. >> >> On Thu, Sep 19, 2024, at 11:34 AM, Benedict wrote: >>> >>> No, there is another perfectly sensible option: just implement a simple >>> serialisation format ourselves. >>> >>> I am against forking their code; that is a much higher maintenance burden >>> than just writing something simple ourselves. We’ve spent longer >>> collectively discussing and maintaining this dependency than it would take >>> to implement the features we use. >>> >>> I still have not heard a compelling reason we adopted it as a dependency in >>> the first place. >>> >>>> On 19 Sep 2024, at 16:26, Josh McKenzie <jmcken...@apache.org> wrote: >>>> >>>>> a jerk move, but they started it with this weird release model >>>> I think that's the only option given their release model and lack of >>>> backporting bugfixes to the latest ea. Either you run tip of the spear, >>>> pay them for bugfixes, or run what's effectively an unsupported LTS in the >>>> form of ea. >>>> >>>> So doesn't seem like a jerk move to me as much as it seems like an >>>> eventuality of their release model. >>>> >>>> On Wed, Sep 18, 2024, at 7:02 PM, Nate McCall wrote: >>>>> I feel like a group of us discussed this IRL a bit at ApacheCon in Vegas >>>>> ~ 2019 maybe? Anyhoo, the tidbit sticking in my mind was someone >>>>> explaining the string operations overhead in the JVM of log concatenation >>>>> vs slapping binary to CQ’s off heap-and-append operation was substantial. >>>>> >>>>> We could hostile fork and bring the bits we use in tree (a jerk move, but >>>>> they started it with this weird release model). I’d rather avoid this, >>>>> but it’s an option seeing as how it’s ASFv2. >>>>> >>>>> On Thu, 19 Sep 2024 at 5:08 AM, Jeremiah Jordan >>>>> <jeremiah.jor...@gmail.com <mailto:jeremiah.jor...@gmail.com>> wrote: >>>>> >>>>>> When it comes to alternatives, what about logback + slf4j? It has >>>>>> appenders where we want, it is sync / async, we can code some nio >>>>>> appender too I guess, it logs it as text into a file so we do not need >>>>>> any special tooling to review that. For tailing which Chronicle also >>>>>> offers, I guess "tail -f that.log" just does the job? logback even rolls >>>>>> the files after they are big enough so it rolls the files the same way >>>>>> after some configured period / size as Chronicle does (It even >>>>>> compresses the logs). >>>>> >>>>> >>>>> Yes it was considered. The whole point was to have a binary log because >>>>> serialization to/from (remember replay is part off this) text explodes >>>>> the size on disk and in memory as well as the processing time required >>>>> and does not meet the timing requirements of fqltool. >>>>> >>>>> -Jeremiah >>>> >> > >
