If you are buffering events in memory you run the risk of losing events if something should fail.
That said, if I had your requirements I would use the FlumeAppender. It has either an in-memory option to buffer as you are suggesting or it can write to a local file to prevent data loss if that is a requirement. It already has the configuration options you are looking for and has been well tested. The only downside is that you need to have either a Flume instance receiving the messages are something that can receive Flume events over Avro, but it is easier just to use Flume and write a custom sink to do what you want with the data. Ralph > On Sep 24, 2016, at 3:13 PM, Gary Gregory <garydgreg...@gmail.com> wrote: > > Hi All, > > I can't believe it, but through a convoluted use-case, I actually need an > in-memory list appender, very much like our test-only ListAppender. > > The requirement is as follows. > > We have a JDBC driver and matching proprietary database that specializes in > data virtualization of mainframe resources like DB2, VSAM, IMS, and all sorts > of non-SQL data sources > (http://www.rocketsoftware.com/products/rocket-data/rocket-data-virtualization > > <http://www.rocketsoftware.com/products/rocket-data/rocket-data-virtualization>) > > > The high level requirement is to merge the driver log into the server's log > for full-end to end tractability and debugging. > > When the driver is running on the z/OS mainframe, it can be configured with a > z/OS specific Appender that can talk to the server log module directly. > > When the driver is running elsewhere, it can talk to the database via a > Syslog socket Appender. This requires more set up on the server side and for > the server to do special magic to know how the incoming log events match up > with server operations. Tricky. > > The customer should also be able to configure the driver such that anytime > the driver communicates to the database, it sends along whatever log events > have accumulated since the last client-server roundtrip. This allows the > server to match exactly the connection and operations the client performed > with the server's own logging. > > In order to do that I need to buffer all log events in an Appender and when > it's time, I need to get the list of events and reset the appender to a new > empty list so events can keep accumulating. > > My proposal is to either turn our ListAppender into such an appender. For > sanity, the appender could be configured with various sizing policies: > > - open: the list grows unbounded > - closed: the list grows to a given size and _new_ events are dropped on the > floor beyond that > - latest: the list grows to a given size and _old_ events are dropped on the > floor beyond that > > Thoughts? > > Gary > > -- > E-Mail: garydgreg...@gmail.com <mailto:garydgreg...@gmail.com> | > ggreg...@apache.org <mailto:ggreg...@apache.org> > Java Persistence with Hibernate, Second Edition > <http://www.manning.com/bauer3/> > JUnit in Action, Second Edition <http://www.manning.com/tahchiev/> > Spring Batch in Action <http://www.manning.com/templier/> > Blog: http://garygregory.wordpress.com <http://garygregory.wordpress.com/> > Home: http://garygregory.com/ <http://garygregory.com/> > Tweet! http://twitter.com/GaryGregory <http://twitter.com/GaryGregory>
