Sounds good. I think a first step would be for us to cooperate on completing the separation of log4j-kafka that I started, and then use that as a template for others.
On Sun, Apr 30, 2017 at 10:23 PM, Matt Sicker <boa...@gmail.com> wrote: > Just to make sure this sounds reasonable, here's my idea for a roadmap: > > 2.9: separate out more modules (not necessarily into their own repo or > repos yet) so that log4j-core contains minimal dependencies. > 2.10: log4j-core-spi or whatever the name, making it easier for other > modules to release on their own without being synced up with log4j-core > versions > > With the current JPMS drama going on, it seems like we may have more time > before Java 9 is released, so we should have time to follow this path > rather than attempting the full on modularization all in one go. > > On 30 April 2017 at 11:56, Matt Sicker <boa...@gmail.com> wrote: > > > I doubt Google uses Git then. > > > > One of the main goals of separating repositories is to make release > > management easier so we can RERO more! Though part of the problem there > is > > that for some reason, the release process requires running all the tests > > and whatnot at least 3 times or more. > > > > On 30 April 2017 at 11:45, Gary Gregory <garydgreg...@gmail.com> wrote: > > > >> Maybe more than one repo isn't such a good idea? I hear Google uses a > >> single repo for all their code... > >> > >> Gary > >> > >> On Apr 30, 2017 9:41 AM, "Matt Sicker" <boa...@gmail.com> wrote: > >> > >> > I've noticed just with the Scala repo that integrating various > >> repositories > >> > into a single coherent website is not so easy anymore (and it wasn't > >> really > >> > all that easy in the first place). While it may be possible to manage > >> each > >> > repository's website individually and use symlinks in the svn repo to > >> keep > >> > the sites linked together, I think there may be easier ways to manage > >> this > >> > if we took a look at alternative site management tools out there. I've > >> > thought about the possibility that we manage our site in a separate > git > >> > repo, but then we'd have to maintain more clear version numbers in the > >> > documentation instead of relying on tagging the docs with the release. > >> > > >> > Besides plain Asciidoc which as been mentioned here before, the only > >> open > >> > source tool I know of that looks interesting here is the one made by > >> vertx: > >> > <https://github.com/vert-x3/vertx-docgen>. See also their site source > >> for > >> > an example on advanced usage: <https://github.com/vert-x3/ve > >> rtx-web-site>. > >> > > >> > On 25 April 2017 at 13:59, Matt Sicker <boa...@gmail.com> wrote: > >> > > >> > > Ideally, the two will align, just like the OSGi modules (which tend > to > >> > > directly correspond with maven modules since that's how they're > >> normally > >> > > assembled). > >> > > > >> > > On 25 April 2017 at 13:39, Gary Gregory <garydgreg...@gmail.com> > >> wrote: > >> > > > >> > >> We are going to have to prefix "module" with "Java" or "Maven" in > >> > >> discussions and documentation to avoid confusion from now on... > >> > >> > >> > >> Gary > >> > >> > >> > >> On Apr 25, 2017 10:59 AM, "Matt Sicker" <boa...@gmail.com> wrote: > >> > >> > >> > >> > If you browse around the Java 9 javadocs, you'll see that they > >> split > >> > >> things > >> > >> > up by module there as well. With that in mind, hopefully it's not > >> too > >> > >> > complicated to support. What I really want to see is inter-module > >> > links > >> > >> > (both Java modules and Maven modules that is) work properly. > >> > >> > > >> > >> > On 25 April 2017 at 11:49, Ralph Goers < > ralph.go...@dslextreme.com > >> > > >> > >> wrote: > >> > >> > > >> > >> > > Ouch. This is where it gets messy. Currently, the javadoc is > >> built > >> > >> > > independently for each module. I’m not sure how to aggregate > them > >> > all > >> > >> > > together but I’m sure Java 9 must be doing that with all the > >> modules > >> > >> they > >> > >> > > are supporting. > >> > >> > > > >> > >> > > Ralph > >> > >> > > > >> > >> > > > On Apr 25, 2017, at 7:03 AM, Mikael Ståldal < > >> > >> mikael.stal...@magine.com > >> > >> > > > >> > >> > > wrote: > >> > >> > > > > >> > >> > > > When adding new modules to the main repo, does each module > need > >> > its > >> > >> own > >> > >> > > > site directory? > >> > >> > > > > >> > >> > > > On Tue, Apr 25, 2017 at 4:02 PM, Mikael Ståldal < > >> > >> > > mikael.stal...@magine.com> > >> > >> > > > wrote: > >> > >> > > > > >> > >> > > >> Yes, they should stay in the main repo, at least for the > time > >> > >> being. > >> > >> > > >> > >> > >> > > >> On Tue, Apr 25, 2017 at 3:56 PM, Gary Gregory < > >> > >> garydgreg...@gmail.com > >> > >> > > > >> > >> > > >> wrote: > >> > >> > > >> > >> > >> > > >>> And all of that should stay in the same repo IMO. > >> > >> > > >>> > >> > >> > > >>> Gary > >> > >> > > >>> > >> > >> > > >>> On Apr 25, 2017 2:51 AM, "Mikael Ståldal" < > >> > >> mikael.stal...@magine.com > >> > >> > > > >> > >> > > >>> wrote: > >> > >> > > >>> > >> > >> > > >>>> I guess that log4-core will become: > >> > >> > > >>>> > >> > >> > > >>>> - log4j-core (will depend on log4j-spi) > >> > >> > > >>>> - log4j-spi > >> > >> > > >>>> - log4j-csv > >> > >> > > >>>> - log4j-xml (XmlLayout) > >> > >> > > >>>> - log4j-json (JsonLayout) > >> > >> > > >>>> - log4j-yaml (YamlLayout) > >> > >> > > >>>> - log4j-kafka > >> > >> > > >>>> - log4j-smtp > >> > >> > > >>>> - log4j-jms > >> > >> > > >>>> - log4j-jdbc (or can this be kept in log4j-core?) > >> > >> > > >>>> - log4j-jpa > >> > >> > > >>>> - log4j-zeromq > >> > >> > > >>>> - log4j-server (already done, not yet released) > >> > >> > > >>>> - log4j-tools (command line tools) > >> > >> > > >>>> > >> > >> > > >>>> > >> > >> > > >>>> Then we should also split log4j-nosql: > >> > >> > > >>>> > >> > >> > > >>>> - log4j-cassandra > >> > >> > > >>>> - log4j-couchdb > >> > >> > > >>>> - log4j-mongodb > >> > >> > > >>>> - log4j-lucene (new, under development) > >> > >> > > >>>> > >> > >> > > >>>> > >> > >> > > >>>> > >> > >> > > >>>> On Mon, Apr 24, 2017 at 7:43 PM, Remko Popma < > >> > >> remko.po...@gmail.com > >> > >> > > > >> > >> > > >>>> wrote: > >> > >> > > >>>> > >> > >> > > >>>>> How many new modules are we talking about, concretely? > >> > >> > > >>>>> > >> > >> > > >>>>> Matt mentioned the StackOverflow questions about > transitive > >> > >> > > >>> dependencies > >> > >> > > >>>>> etc, but I imagine splitting log4j-core into 5 or more > new > >> > >> modules > >> > >> > > >>> will > >> > >> > > >>>>> also cause confusion... It won't be trivial for users to > >> > figure > >> > >> out > >> > >> > > >>> which > >> > >> > > >>>>> of the many modules they do or don't need. The coarse > >> > >> granularity > >> > >> > of > >> > >> > > >>> the > >> > >> > > >>>>> current modules is a good thing for users. > >> > >> > > >>>>> > >> > >> > > >>>>> What problem are we trying to solve? And how can we solve > >> it > >> > >> with > >> > >> > the > >> > >> > > >>>> least > >> > >> > > >>>>> disruption to our users? > >> > >> > > >>>>> > >> > >> > > >>>>> Would it be an idea, for example, to provide separate > jars > >> for > >> > >> the > >> > >> > > >>>> separate > >> > >> > > >>>>> modules, but in addition create a combined jar > >> > (log4j-core-all) > >> > >> > that > >> > >> > > >>>>> contains all the classes in log4j-core as well as the > >> classes > >> > in > >> > >> > the > >> > >> > > >>> new > >> > >> > > >>>>> modules we split out from core? > >> > >> > > >>>>> > >> > >> > > >>>>> > >> > >> > > >>>>> On Tue, Apr 25, 2017 at 2:00 AM, Matt Sicker < > >> > boa...@gmail.com> > >> > >> > > >>> wrote: > >> > >> > > >>>>> > >> > >> > > >>>>>> I agree with Ralph here. I'm sure we'll figure out > rather > >> > >> quickly > >> > >> > > >>> which > >> > >> > > >>>>>> modules are easy to put into rarely updated > repositories. > >> > >> > > >>>>>> > >> > >> > > >>>>>> On 24 April 2017 at 11:39, Ralph Goers < > >> > >> > ralph.go...@dslextreme.com> > >> > >> > > >>>>> wrote: > >> > >> > > >>>>>> > >> > >> > > >>>>>>> I would prefer a hybrid approach. First things should > be > >> > >> moved > >> > >> > to > >> > >> > > >>>>>>> separate modules. Then, if they don’t seem to be > modified > >> > >> > > >>> frequently > >> > >> > > >>>>> they > >> > >> > > >>>>>>> can be moved to a separate repo. For example, I think > it > >> > >> would be > >> > >> > > >>> OK > >> > >> > > >>>>> for > >> > >> > > >>>>>>> the Flume Appender to be in a separate repo. It hasn’t > >> > >> changed in > >> > >> > > >>>>> quite a > >> > >> > > >>>>>>> while and I can’t remember the last time it was > modified > >> due > >> > >> to > >> > >> > > >>>> changes > >> > >> > > >>>>>> in > >> > >> > > >>>>>>> Log4j it has and while continue to change with changes > >> made > >> > in > >> > >> > > >>> Flume > >> > >> > > >>>>>>> releases. I imagine we have quite a few components > that > >> are > >> > >> > > >>> similar. > >> > >> > > >>>>>>> > >> > >> > > >>>>>>> Ralph > >> > >> > > >>>>>>> > >> > >> > > >>>>>>>> On Apr 24, 2017, at 8:39 AM, Gary Gregory < > >> > >> > > >>> garydgreg...@gmail.com> > >> > >> > > >>>>>>> wrote: > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> On Apr 24, 2017 2:38 AM, "Mikael Ståldal" < > >> > >> > > >>>> mikael.stal...@magine.com > >> > >> > > >>>>>> > >> > >> > > >>>>>>> wrote: > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> I fully agree with Matt's both proposals. > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> I'm skeptic to creating more repositories (than we > >> already > >> > >> have) > >> > >> > > >>>>>> though. > >> > >> > > >>>>>>> I > >> > >> > > >>>>>>>> think that we should start by splitting out modules > from > >> > >> > > >>> log4j-core > >> > >> > > >>>>> and > >> > >> > > >>>>>>>> keep those modules in the main repository with > >> synchronized > >> > >> > > >>>>> versioning > >> > >> > > >>>>>>> and > >> > >> > > >>>>>>>> releases, at least for the 2.9 release. We can always > >> move > >> > >> those > >> > >> > > >>>>>> modules > >> > >> > > >>>>>>> to > >> > >> > > >>>>>>>> other repositories later if we want to. > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> I do not like more repos either. Since we have already > >> gone > >> > >> down > >> > >> > > >>>> the > >> > >> > > >>>>>> more > >> > >> > > >>>>>>>> modules road, I say we keep going. > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> Gary > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> It is a lot of administrative work to create a new > >> > repository > >> > >> > > >>> (as > >> > >> > > >>>> we > >> > >> > > >>>>>> have > >> > >> > > >>>>>>>> seen for log4j-scala), I don't want us to do all that > >> work > >> > >> over > >> > >> > > >>> and > >> > >> > > >>>>>> over > >> > >> > > >>>>>>>> again unless really necessary. > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> We have a JIRA ticket for this: > >> > >> > > >>>>>>>> https://issues.apache.org/jira/browse/LOG4J2-1650 > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> I have already started by breaking out log4j-server: > >> > >> > > >>>>>>>> https://issues.apache.org/jira/browse/LOG4J2-1851 > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> I think the next step is to break out plugins (layouts > >> and > >> > >> > > >>>> appenders) > >> > >> > > >>>>>>> with > >> > >> > > >>>>>>>> optional 3rd party dependencies into their own > modules. > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> On Sun, Apr 23, 2017 at 7:45 PM, Matt Sicker < > >> > >> boa...@gmail.com> > >> > >> > > >>>>> wrote: > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>>> I think I brought this topic up like 3 years ago > when I > >> > was > >> > >> > > >>>> working > >> > >> > > >>>>> on > >> > >> > > >>>>>>>>> initial OSGi support, but now that we have 3 more > years > >> > >> worth > >> > >> > > >>> of > >> > >> > > >>>>> code > >> > >> > > >>>>>>>>> additions and optional features, I think this might > be > >> a > >> > >> more > >> > >> > > >>>>>>> appropriate > >> > >> > > >>>>>>>>> time to discuss it again in light of experience. > >> > >> > > >>>>>>>>> > >> > >> > > >>>>>>>>> Building log4j-core itself already takes a long time, > >> and > >> > >> many > >> > >> > > >>>>> plugins > >> > >> > > >>>>>>>>> aren't updated very often at all. In the past, > >> requiring > >> > >> users > >> > >> > > >>> to > >> > >> > > >>>>>> simply > >> > >> > > >>>>>>>>> add log4j-core plus any transitive dependencies to > use > >> > >> optional > >> > >> > > >>>>>> features > >> > >> > > >>>>>>>>> seemed to work well enough, but I still think that's > a > >> > >> > > >>> confusing > >> > >> > > >>>>>>>>> distribution mechanism as demonstrated by the > numerous > >> bug > >> > >> > > >>> reports > >> > >> > > >>>>> and > >> > >> > > >>>>>>>>> Stack Overflow posts regarding missing transitive > >> > >> dependencies > >> > >> > > >>> for > >> > >> > > >>>>>>> various > >> > >> > > >>>>>>>>> features. I spent some time experimenting with Log4j > >> Boot > >> > a > >> > >> > > >>> little > >> > >> > > >>>>>> while > >> > >> > > >>>>>>>>> ago to help alleviate this problem, but this may be > >> > >> > > >>> unnecessary if > >> > >> > > >>>>> we > >> > >> > > >>>>>>> can > >> > >> > > >>>>>>>>> agree to modularize log4j-core itself. > >> > >> > > >>>>>>>>> > >> > >> > > >>>>>>>>> I have two different proposals, both of which can be > >> used > >> > at > >> > >> > > >>> the > >> > >> > > >>>>> same > >> > >> > > >>>>>>>> time. > >> > >> > > >>>>>>>>> > >> > >> > > >>>>>>>>> 1. Split out everything from log4j-core that requires > >> 3rd > >> > >> party > >> > >> > > >>>>>>>>> dependencies (except for AsyncLogger, though perhaps > we > >> > >> could > >> > >> > > >>>>> consider > >> > >> > > >>>>>>>>> shading and renaming those classes like some other > low > >> > level > >> > >> > > >>>>> libraries > >> > >> > > >>>>>>> do > >> > >> > > >>>>>>>>> with JCTools). Ideally, I'd like to see each module > >> have > >> > >> > > >>> required > >> > >> > > >>>>>>>>> dependencies instead of optional ones, so that if, > for > >> > >> > > >>> instance, I > >> > >> > > >>>>>>> include > >> > >> > > >>>>>>>>> a "log4j-config-yaml" dependency, I know that Log4j > >> will > >> > >> > > >>> support > >> > >> > > >>>>> YAML > >> > >> > > >>>>>>>>> configuration without having to specify the > individual > >> > >> Jackson > >> > >> > > >>>>>>>>> dependencies. > >> > >> > > >>>>>>>>> > >> > >> > > >>>>>>>>> 2. Split out from log4j-core a sort of log4j-spi > module > >> > >> which > >> > >> > > >>>>> defines > >> > >> > > >>>>>>>>> interfaces, abstract classes, and annotations for > >> plugins > >> > >> that > >> > >> > > >>>> would > >> > >> > > >>>>>> be > >> > >> > > >>>>>>>>> promoted to the same level of backwards compatibility > >> > >> > > >>> guarantees > >> > >> > > >>>> as > >> > >> > > >>>>>>>>> log4j-api. This would aid in cementing what we really > >> wish > >> > >> to > >> > >> > > >>>>> maintain > >> > >> > > >>>>>>>>> compatibility with in the backend while allowing > other > >> > >> modules > >> > >> > > >>> to > >> > >> > > >>>>> have > >> > >> > > >>>>>>>> less > >> > >> > > >>>>>>>>> strict guarantees. > >> > >> > > >>>>>>>>> > >> > >> > > >>>>>>>>> With proposal #1, I'd think that we could more easily > >> > start > >> > >> > > >>> moving > >> > >> > > >>>>>>> modules > >> > >> > > >>>>>>>>> into separate repositories and release trains. > Without > >> #2, > >> > >> > > >>> though, > >> > >> > > >>>>>> this > >> > >> > > >>>>>>>>> makes version support more annoying to handle, but > >> that's > >> > >> what > >> > >> > > >>>> we'll > >> > >> > > >>>>>>> face > >> > >> > > >>>>>>>>> regardless as we separate more repositories. If we go > >> this > >> > >> > > >>> route, > >> > >> > > >>>>> then > >> > >> > > >>>>>>>>> there will be no need for a Log4j Boot subproject. > >> > >> > > >>>>>>>>> > >> > >> > > >>>>>>>>> What do you all think? > >> > >> > > >>>>>>>>> > >> > >> > > >>>>>>>>> -- > >> > >> > > >>>>>>>>> Matt Sicker <boa...@gmail.com> > >> > >> > > >>>>>>>>> > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> -- > >> > >> > > >>>>>>>> [image: MagineTV] > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> *Mikael Ståldal* > >> > >> > > >>>>>>>> Senior software developer > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> *Magine TV* > >> > >> > > >>>>>>>> mikael.stal...@magine.com > >> > >> > > >>>>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | > >> > >> > > >>> www.magine.com > >> > >> > > >>>>>>>> > >> > >> > > >>>>>>>> Privileged and/or Confidential Information may be > >> contained > >> > >> in > >> > >> > > >>> this > >> > >> > > >>>>>>>> message. If you are not the addressee indicated in > this > >> > >> message > >> > >> > > >>>>>>>> (or responsible for delivery of the message to such a > >> > >> person), > >> > >> > > >>> you > >> > >> > > >>>>> may > >> > >> > > >>>>>>> not > >> > >> > > >>>>>>>> copy or deliver this message to anyone. In such case, > >> > >> > > >>>>>>>> you should destroy this message and kindly notify the > >> > sender > >> > >> by > >> > >> > > >>>> reply > >> > >> > > >>>>>>>> email. > >> > >> > > >>>>>>> > >> > >> > > >>>>>>> > >> > >> > > >>>>>>> > >> > >> > > >>>>>> > >> > >> > > >>>>>> > >> > >> > > >>>>>> -- > >> > >> > > >>>>>> Matt Sicker <boa...@gmail.com> > >> > >> > > >>>>>> > >> > >> > > >>>>> > >> > >> > > >>>> > >> > >> > > >>>> > >> > >> > > >>>> > >> > >> > > >>>> -- > >> > >> > > >>>> [image: MagineTV] > >> > >> > > >>>> > >> > >> > > >>>> *Mikael Ståldal* > >> > >> > > >>>> Senior software developer > >> > >> > > >>>> > >> > >> > > >>>> *Magine TV* > >> > >> > > >>>> mikael.stal...@magine.com > >> > >> > > >>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | > >> > www.magine.com > >> > >> > > >>>> > >> > >> > > >>>> Privileged and/or Confidential Information may be > contained > >> in > >> > >> this > >> > >> > > >>>> message. If you are not the addressee indicated in this > >> message > >> > >> > > >>>> (or responsible for delivery of the message to such a > >> person), > >> > >> you > >> > >> > may > >> > >> > > >>> not > >> > >> > > >>>> copy or deliver this message to anyone. In such case, > >> > >> > > >>>> you should destroy this message and kindly notify the > >> sender by > >> > >> > reply > >> > >> > > >>>> email. > >> > >> > > >>>> > >> > >> > > >>> > >> > >> > > >> > >> > >> > > >> > >> > >> > > >> > >> > >> > > >> -- > >> > >> > > >> [image: MagineTV] > >> > >> > > >> > >> > >> > > >> *Mikael Ståldal* > >> > >> > > >> Senior software developer > >> > >> > > >> > >> > >> > > >> *Magine TV* > >> > >> > > >> mikael.stal...@magine.com > >> > >> > > >> Grev Turegatan 3 | 114 46 Stockholm, Sweden | > >> www.magine.com > >> > >> > > >> > >> > >> > > >> Privileged and/or Confidential Information may be contained > in > >> > this > >> > >> > > >> message. If you are not the addressee indicated in this > >> message > >> > >> > > >> (or responsible for delivery of the message to such a > person), > >> > you > >> > >> may > >> > >> > > not > >> > >> > > >> copy or deliver this message to anyone. In such case, > >> > >> > > >> you should destroy this message and kindly notify the sender > >> by > >> > >> reply > >> > >> > > >> email. > >> > >> > > >> > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > -- > >> > >> > > > [image: MagineTV] > >> > >> > > > > >> > >> > > > *Mikael Ståldal* > >> > >> > > > Senior software developer > >> > >> > > > > >> > >> > > > *Magine TV* > >> > >> > > > mikael.stal...@magine.com > >> > >> > > > Grev Turegatan 3 | 114 46 Stockholm, Sweden | > >> www.magine.com > >> > >> > > > > >> > >> > > > Privileged and/or Confidential Information may be contained > in > >> > this > >> > >> > > > message. If you are not the addressee indicated in this > message > >> > >> > > > (or responsible for delivery of the message to such a > person), > >> you > >> > >> may > >> > >> > > not > >> > >> > > > copy or deliver this message to anyone. In such case, > >> > >> > > > you should destroy this message and kindly notify the sender > by > >> > >> reply > >> > >> > > > email. > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > >> > >> > > >> > >> > -- > >> > >> > Matt Sicker <boa...@gmail.com> > >> > >> > > >> > >> > >> > > > >> > > > >> > > > >> > > -- > >> > > Matt Sicker <boa...@gmail.com> > >> > > > >> > > >> > > >> > > >> > -- > >> > Matt Sicker <boa...@gmail.com> > >> > > >> > > > > > > > > -- > > Matt Sicker <boa...@gmail.com> > > > > > > -- > Matt Sicker <boa...@gmail.com> > -- [image: MagineTV] *Mikael Ståldal* Senior software developer *Magine TV* mikael.stal...@magine.com Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com Privileged and/or Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such a person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email.
