It may be wise to always include some (configurable) backoff mechanism, to prevent our failover/reconnect logic from essentially initiating a denial of service attack when the service appears to be unavailable.
Not sure if we currently have this anywhere, but if we’re thinking to extract some reusable failover/reconnect API let’s make the backoff mechanism a first-class citizen. Remko. (Shameless plug) Every java main() method deserves http://picocli.info > On Nov 7, 2018, at 3:32, Carter Kozak <[email protected]> wrote: > > It would be helpful if we could implement failover and retrial in a > way that is compatible with disabling immediate flush. The current > implementations may surface a failure to the attempt to log an end of > batch, but don't provide a mechanism to retry queued or buffered > events. > >> On Tue, Nov 6, 2018 at 1:31 PM Gary Gregory <[email protected]> wrote: >> >> On Tue, Nov 6, 2018 at 11:19 AM Ralph Goers <[email protected]> >> wrote: >> >>> The socket appender already has the ability to reconnect. It just needs to >>> ability to load balance or send to an alternate host if the connection >>> fails. >>> >> >> Sure, I just would like reconnection and failover to be abstracted in our >> core framework so that each connector does not have to re-invent the wheel. >> >> Gary >> >>> >>> Ralph >>> >>>> On Nov 6, 2018, at 11:15 AM, Gary Gregory <[email protected]> >>> wrote: >>>> >>>> Speaking of failover kind of things, the JMS Appender has a reconnect >>>> feature and I am wrapping up a similar feature for the JDBC Appender. >>> This >>>> kind of feature is a MUST for services that need to stay up and running >>> for >>>> longer periods of time. >>>> >>>> It would be nice to have a reconnect feature abstracted out so that all >>>> appenders/managers that depend on external resources can survive these >>>> resources going up and down as well as internet connections going up and >>>> down. That would make it much easier to implement this in all appenders >>>> that need it. A start would be to abstract JMS and JDBC reconnect code. >>>> Testing this is quite tricky of course. >>>> >>>> Gary >>>> >>>> On Mon, Nov 5, 2018 at 9:33 PM Ralph Goers <[email protected]> >>>> wrote: >>>> >>>>> Some other features I need/want to do: >>>>> >>>>> 1. LOG4J2-1137 >>>>> 2. Allow the SocketAppender to specify multiple IP addresses and allow >>>>> either round-robining through them or failing to the next if the first >>>>> fails. This will provide better high availability for applications. >>>>> 3. Support a ContextSelector based on Module Layers. >>>>> 4. LOG4J2-2170 >>>>> >>>>> Ralph >>>>> >>>>>> On Nov 5, 2018, at 9:22 PM, Ralph Goers <[email protected]> >>>>> wrote: >>>>>> >>>>>> How much are we impacting the API? I don’t know that package naming is >>>>> required if the API is compatible. I am hoping this doesn’t impact the >>> API >>>>> much. >>>>>> >>>>>> I’d prefer this just be log4j 3.x. Log4j 2 3.x is just really weird. >>>>>> >>>>>> I wouldn’t say a module shouldn’t have any optional dependencies but it >>>>> should be as few as possible. That said, because java is now >>> modularized >>>>> and you only get java.base by default I think log4j-core should only >>>>> require that. This would mean probably only the Properties configuration >>>>> can remain in core. >>>>>> >>>>>> I’m not completely sold on replacing the configuration with Jackson or >>>>> Commons Configuration. First, I really like that we convert the >>>>> configuration to a node tree and then process the node tree the same way >>>>> regardless of the configuration syntax used to construct it. Since we >>>>> already use Jackson for JSON and YAML I am not sure what it means to >>> redo >>>>> the configuration to use something we are already using. >>>>>> >>>>>> I would like to have every Maven module be a JPMS module, but this may >>>>> still be impossible to do as not all of our dependencies have declared >>>>> module names yet. For example, >>>>> https://github.com/LMAX-Exchange/disruptor/issues/234 < >>>>> https://github.com/LMAX-Exchange/disruptor/issues/234> shows the >>>>> disruptor still hasn’t done anything. >>>>>> >>>>>> For me, the main goal would be just “cleaning up” so the modules have >>>>> fewer dependencies. This also should align nicely with generating JPMS >>>>> modules. >>>>>> >>>>>> I do have new features I want to add and they don’t really require 3.0 >>>>> to do them, but I would really like to provide good reasons to upgrade >>> to >>>>> log4j 3.x besides internal cleanup. >>>>>> >>>>>> One new feature that is a high priority for me is to make Log4j more >>>>> “cloud friendly”. This means being able to read and dynamically update >>> the >>>>> logging configuration from something like Spring Cloud Configuration. >>>>> Essentially this just means being able to read and monitor a file via >>> HTTP >>>>> instead of using only the File API. >>>>>> >>>>>> Also, I’d like to make another pass at performance testing to see where >>>>> we still have room for improvement. >>>>>> >>>>>> I would really, like to figure out a way to include location >>> information >>>>> in the log events without the overhead we have now. The only sane way >>> to do >>>>> it is to somehow get the information at compile time, but I just haven’t >>>>> been able to figure out a clever hack to make it work. >>>>>> >>>>>> Ralph >>>>>> >>>>>>> On Nov 5, 2018, at 2:01 PM, Gary Gregory <[email protected]> >>>>> wrote: >>>>>>> >>>>>>> Considerations for 3.0: >>>>>>> >>>>>>> - Currently targeting Java 8, seems OK to keep this for now. >>>>>>> - Remove deprecated code >>>>>>> - Make BC-breaking changes as we see fit to improve impl. >>>>>>> - ? Update root package to include "3" to allow Log4j 1, 2, and 3 to >>>>>>> co-exist peacefully on the claspath. Perhaps >>> org.apache.logging.log4j3. >>>>>>> - Do we need a compatibility layer for 1.2 to 3.0 and 2.x to 3.0? >>>>>>> - Where can we use java.time? >>>>>>> - Is it a goal to have Maven modules with NO optional dependencies? I >>>>> think >>>>>>> so. >>>>>>> - Play nice in the Java 9 module system >>>>>>> - Continue to break up current Maven modules >>>>>>> - How can we make Core smaller? >>>>>>> - Should we redo our config code to use something like Jackson or >>>>> Commons >>>>>>> Configuration? We have a lot of config code... Not sure if everything >>>>> you >>>>>>> can do in XML is doable in JSON and YAML. YAML is gross IMO but some >>>>> people >>>>>>> like it. >>>>>>> >>>>>>> What else? >>>>>>> >>>>>>> Gary >>>>>> >>>>> >>>>> >>>>> >>> >>> >>>
