Re: Logstash and Filebeat guaranteed delivery

2023-12-02 Thread Matt Sicker 
Right, I think I misread that. However, that’s what the curation idea is for :)
—
Matt Sicker

> On Dec 1, 2023, at 18:47, Gary Gregory  wrote:
> 
> Of course RE Volkan, my first sentence must have been poorly written. I
> meant to say that I know who Volkan is and that he is great, but, that's
> not the point. I was trying to say that a peronal GitHub account, in
> general, can't be compared to an official Apache account.
> 
> Gary
> 
> On Fri, Dec 1, 2023, 12:25 PM Matt Sicker  wrote:
> 
>> I may be biased, but I wouldn’t consider Volkan a random GitHub user!
>> However, that does raise an interesting point: we could link to third party
>> plugins and such to help curate it.
>> 
>>> On Dec 1, 2023, at 5:00 AM, Gary Gregory  wrote:
>>> 
>>> Hi all,
>>> 
>>> (Don't take his the wrong way Volkan ;-)
>>> Assuming that I don't know who you are, why would I pick a random github
>>> user's custom appender instead of an official Log4j appender? If your
>>> appender is "battle-tested", why not move it to Log4j (or Redis?)
>>> 
>>> Gary
>>> 
>>> 
>>> On Fri, Dec 1, 2023, 4:08 AM Volkan Yazıcı  wrote:
>>> 
 I appreciate your thoughts on this subject. We can eventually convert
>> this
 into a chapter in the Log4j manual. My goal is to be able to make a
 statement as follows:
 
 *When Log4j is configured with Y, Y, Z settings, it can provide
>> guaranteed
 delivery against certain types of log sinks such as A, B, C.*
 
 *A – You need to make sure A has ... feature enabled. Further, it has
>> ...
 caveat.*
 *B – You need to make sure B has ... feature enabled and ...*
 *C – ...*
 
 
 That is, a cookbook for users with recipes for guaranteed delivery.
 
 [I respond to your message below inline.]
 
 On Thu, Nov 30, 2023 at 9:34 PM Ralph Goers >> 
 wrote:
 
> Notice that neither of the links you have provided use the term
> “guaranteed delivery”. That is because that is not really what they are
> providing. In addition, notice that Logstash says "Input plugins that
>> do
> not use a request-response protocol cannot be protected from data
>> loss”,
 
 
 But see the rest of that statement
 <
 
>> https://www.elastic.co/guide/en/logstash/current/persistent-queues.html#persistent-queues-limitations
> 
 : *"Plugins such as beats and http, which do have an acknowledgement
 capability, are well protected by this [Logstash persistent] queue."*
 
 
> and "Data may be lost if an abnormal shutdown occurs before the
 checkpoint
> file has been committed”.
 
 
 See the following statement further down in that page: *"To avoid losing
 data in the persistent queue, you can set `queue.checkpoint.writes: 1`
>> to
 force a checkpoint after each event is written."*
 
 These two make me conclude that, if configured correctly (e.g., using
 `http` plugin in combination with `queue.checkpoint.writes: 1`),
>> Logstash
 can deliver guaranteed delivery. Am I mistaken?
 
 
> As for using Google Cloud that would default the whole point. If your
 data
> center has lost contact with the outside world it won’t be able to get
>> to
> Google Cloud.
> 
 
 But that cannot be an argument against using Google Cloud as a log sink
 with guaranteed delivery. An in-house Flume server can go down too. Let
>> me
 know if I miss your point here.
 
 
> While Redis would work it would require a) an application component
>> that
> interacts with Redis such as a Redis Appender (which we don’t have) b)
>> a
> Redis deployment c) a Logstash (or some other Redis consumer) to
>> forward
> the event. It is a lot simpler to configure Flume than to do all of
>> that.
> 
 
 For one, there is a battle-tested Log4j Redis Appender
 , which enabled us to
>> remove
 `log4j-redis` in `main`.
 
 Indeed, Flume can deliver what Redis+Logstash do. Though my point is,
>> not
 that Redis has a magical feature set, but there *are* several log sink
 stacks one can build using modern stock components and provide
>> guaranteed
 delivery. I would like to document some of those, if not best-practices,
 known-to-work solutions. This way we can enable our users to make a
 well-informed decision and pick the best approach that fits into their
 existing stack.
 
 On Thu, Nov 30, 2023 at 9:34 PM Ralph Goers >> 
 wrote:
 
> Volkan,
> 
> Notice that neither of the links you have provided use the term
> “guaranteed delivery”. That is because that is not really what they are
> providing. In addition, notice that Logstash says "Input plugins that
>> do
> not use a request-response protocol cannot be protected from data
>> loss”,
> and "Data may be lost if an abnormal shutdown occurs before the
 checkpoint
>

Re: [chainsaw] Question: oading/writing settings of ChainsawTabbedPane

2023-12-02 Thread Scott Deboy 
That's great!

Global app settings used to be loaded on startup and saved on exit.

LogPanel-specific settings were persisted on exit and loaded on
startup on a per-tab-name basis, mostly using LogPanelpreferenceModel.

There are a few preferences I'd like to restore - the tab-expression
event routing mechanism and the ability to hide and show columns.

Scott

On 12/2/23, Christian Grobmeier  wrote:
> Hi,
>
> I re-added the load/write feature in ChainsawTabbedPane using JSON and
> Genson:
> https://github.com/apache/logging-chainsaw/blob/master/src/main/java/org/apache/log4j/chainsaw/components/tabbedpane/ChainsawTabbedPane.java#L126-L191
>
> If it works, I will refactor it further so we can reuse the writing and
> reading code for other components.
>
> I could not test it: writing and reading are not used, and I want to know
> when this should happen.
>
> Is it supposed to read this when we start the application and write it when
> the tab was changed?
>
> Kind regards,
> Christian
>
>

[chainsaw] Question: oading/writing settings of ChainsawTabbedPane

2023-12-02 Thread Christian Grobmeier 
Hi,

I re-added the load/write feature in ChainsawTabbedPane using JSON and Genson:
https://github.com/apache/logging-chainsaw/blob/master/src/main/java/org/apache/log4j/chainsaw/components/tabbedpane/ChainsawTabbedPane.java#L126-L191

If it works, I will refactor it further so we can reuse the writing and reading 
code for other components.

I could not test it: writing and reading are not used, and I want to know when 
this should happen.

Is it supposed to read this when we start the application and write it when the 
tab was changed?

Kind regards,
Christian