I’m not surprised. My plan for the socket appender is to look at leveraging Netty. I believe it handles all of what I want to do really well.
Ralph > On Nov 6, 2018, at 4:38 PM, Gary Gregory <[email protected]> wrote: > > On Tue, Nov 6, 2018 at 4:10 PM Remko Popma <[email protected]> wrote: > >> It may be wise to always include some (configurable) backoff mechanism, to >> prevent our failover/reconnect logic from essentially initiating a denial >> of service attack when the service appears to be unavailable. >> >> Not sure if we currently have this anywhere, but if we’re thinking to >> extract some reusable failover/reconnect API let’s make the backoff >> mechanism a first-class citizen. >> > > The Socket, JMS and JDBC appenders all have reconnect logic. All slightly > different. > > Gary > > >> >> Remko. >> >> (Shameless plug) Every java main() method deserves http://picocli.info >> >>> On Nov 7, 2018, at 3:32, Carter Kozak <[email protected]> wrote: >>> >>> It would be helpful if we could implement failover and retrial in a >>> way that is compatible with disabling immediate flush. The current >>> implementations may surface a failure to the attempt to log an end of >>> batch, but don't provide a mechanism to retry queued or buffered >>> events. >>> >>>> On Tue, Nov 6, 2018 at 1:31 PM Gary Gregory <[email protected]> >> wrote: >>>> >>>> On Tue, Nov 6, 2018 at 11:19 AM Ralph Goers <[email protected] >>> >>>> wrote: >>>> >>>>> The socket appender already has the ability to reconnect. It just >> needs to >>>>> ability to load balance or send to an alternate host if the connection >>>>> fails. >>>>> >>>> >>>> Sure, I just would like reconnection and failover to be abstracted in >> our >>>> core framework so that each connector does not have to re-invent the >> wheel. >>>> >>>> Gary >>>> >>>>> >>>>> Ralph >>>>> >>>>>> On Nov 6, 2018, at 11:15 AM, Gary Gregory <[email protected]> >>>>> wrote: >>>>>> >>>>>> Speaking of failover kind of things, the JMS Appender has a reconnect >>>>>> feature and I am wrapping up a similar feature for the JDBC Appender. >>>>> This >>>>>> kind of feature is a MUST for services that need to stay up and >> running >>>>> for >>>>>> longer periods of time. >>>>>> >>>>>> It would be nice to have a reconnect feature abstracted out so that >> all >>>>>> appenders/managers that depend on external resources can survive these >>>>>> resources going up and down as well as internet connections going up >> and >>>>>> down. That would make it much easier to implement this in all >> appenders >>>>>> that need it. A start would be to abstract JMS and JDBC reconnect >> code. >>>>>> Testing this is quite tricky of course. >>>>>> >>>>>> Gary >>>>>> >>>>>> On Mon, Nov 5, 2018 at 9:33 PM Ralph Goers < >> [email protected]> >>>>>> wrote: >>>>>> >>>>>>> Some other features I need/want to do: >>>>>>> >>>>>>> 1. LOG4J2-1137 >>>>>>> 2. Allow the SocketAppender to specify multiple IP addresses and >> allow >>>>>>> either round-robining through them or failing to the next if the >> first >>>>>>> fails. This will provide better high availability for applications. >>>>>>> 3. Support a ContextSelector based on Module Layers. >>>>>>> 4. LOG4J2-2170 >>>>>>> >>>>>>> Ralph >>>>>>> >>>>>>>> On Nov 5, 2018, at 9:22 PM, Ralph Goers <[email protected] >>> >>>>>>> wrote: >>>>>>>> >>>>>>>> How much are we impacting the API? I don’t know that package naming >> is >>>>>>> required if the API is compatible. I am hoping this doesn’t impact >> the >>>>> API >>>>>>> much. >>>>>>>> >>>>>>>> I’d prefer this just be log4j 3.x. Log4j 2 3.x is just really weird. >>>>>>>> >>>>>>>> I wouldn’t say a module shouldn’t have any optional dependencies >> but it >>>>>>> should be as few as possible. That said, because java is now >>>>> modularized >>>>>>> and you only get java.base by default I think log4j-core should only >>>>>>> require that. This would mean probably only the Properties >> configuration >>>>>>> can remain in core. >>>>>>>> >>>>>>>> I’m not completely sold on replacing the configuration with Jackson >> or >>>>>>> Commons Configuration. First, I really like that we convert the >>>>>>> configuration to a node tree and then process the node tree the same >> way >>>>>>> regardless of the configuration syntax used to construct it. Since we >>>>>>> already use Jackson for JSON and YAML I am not sure what it means to >>>>> redo >>>>>>> the configuration to use something we are already using. >>>>>>>> >>>>>>>> I would like to have every Maven module be a JPMS module, but this >> may >>>>>>> still be impossible to do as not all of our dependencies have >> declared >>>>>>> module names yet. For example, >>>>>>> https://github.com/LMAX-Exchange/disruptor/issues/234 < >>>>>>> https://github.com/LMAX-Exchange/disruptor/issues/234> shows the >>>>>>> disruptor still hasn’t done anything. >>>>>>>> >>>>>>>> For me, the main goal would be just “cleaning up” so the modules >> have >>>>>>> fewer dependencies. This also should align nicely with generating >> JPMS >>>>>>> modules. >>>>>>>> >>>>>>>> I do have new features I want to add and they don’t really require >> 3.0 >>>>>>> to do them, but I would really like to provide good reasons to >> upgrade >>>>> to >>>>>>> log4j 3.x besides internal cleanup. >>>>>>>> >>>>>>>> One new feature that is a high priority for me is to make Log4j more >>>>>>> “cloud friendly”. This means being able to read and dynamically >> update >>>>> the >>>>>>> logging configuration from something like Spring Cloud Configuration. >>>>>>> Essentially this just means being able to read and monitor a file via >>>>> HTTP >>>>>>> instead of using only the File API. >>>>>>>> >>>>>>>> Also, I’d like to make another pass at performance testing to see >> where >>>>>>> we still have room for improvement. >>>>>>>> >>>>>>>> I would really, like to figure out a way to include location >>>>> information >>>>>>> in the log events without the overhead we have now. The only sane way >>>>> to do >>>>>>> it is to somehow get the information at compile time, but I just >> haven’t >>>>>>> been able to figure out a clever hack to make it work. >>>>>>>> >>>>>>>> Ralph >>>>>>>> >>>>>>>>> On Nov 5, 2018, at 2:01 PM, Gary Gregory <[email protected]> >>>>>>> wrote: >>>>>>>>> >>>>>>>>> Considerations for 3.0: >>>>>>>>> >>>>>>>>> - Currently targeting Java 8, seems OK to keep this for now. >>>>>>>>> - Remove deprecated code >>>>>>>>> - Make BC-breaking changes as we see fit to improve impl. >>>>>>>>> - ? Update root package to include "3" to allow Log4j 1, 2, and 3 >> to >>>>>>>>> co-exist peacefully on the claspath. Perhaps >>>>> org.apache.logging.log4j3. >>>>>>>>> - Do we need a compatibility layer for 1.2 to 3.0 and 2.x to 3.0? >>>>>>>>> - Where can we use java.time? >>>>>>>>> - Is it a goal to have Maven modules with NO optional >> dependencies? I >>>>>>> think >>>>>>>>> so. >>>>>>>>> - Play nice in the Java 9 module system >>>>>>>>> - Continue to break up current Maven modules >>>>>>>>> - How can we make Core smaller? >>>>>>>>> - Should we redo our config code to use something like Jackson or >>>>>>> Commons >>>>>>>>> Configuration? We have a lot of config code... Not sure if >> everything >>>>>>> you >>>>>>>>> can do in XML is doable in JSON and YAML. YAML is gross IMO but >> some >>>>>>> people >>>>>>>>> like it. >>>>>>>>> >>>>>>>>> What else? >>>>>>>>> >>>>>>>>> Gary >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >>>>> >>
