I am fine with your goals for 2.9. I still have my doubts that separating out an spi can be done.
Ralph > On Apr 30, 2017, at 1:23 PM, Matt Sicker <boa...@gmail.com> wrote: > > Just to make sure this sounds reasonable, here's my idea for a roadmap: > > 2.9: separate out more modules (not necessarily into their own repo or > repos yet) so that log4j-core contains minimal dependencies. > 2.10: log4j-core-spi or whatever the name, making it easier for other > modules to release on their own without being synced up with log4j-core > versions > > With the current JPMS drama going on, it seems like we may have more time > before Java 9 is released, so we should have time to follow this path > rather than attempting the full on modularization all in one go. > > On 30 April 2017 at 11:56, Matt Sicker <boa...@gmail.com> wrote: > >> I doubt Google uses Git then. >> >> One of the main goals of separating repositories is to make release >> management easier so we can RERO more! Though part of the problem there is >> that for some reason, the release process requires running all the tests >> and whatnot at least 3 times or more. >> >> On 30 April 2017 at 11:45, Gary Gregory <garydgreg...@gmail.com> wrote: >> >>> Maybe more than one repo isn't such a good idea? I hear Google uses a >>> single repo for all their code... >>> >>> Gary >>> >>> On Apr 30, 2017 9:41 AM, "Matt Sicker" <boa...@gmail.com> wrote: >>> >>>> I've noticed just with the Scala repo that integrating various >>> repositories >>>> into a single coherent website is not so easy anymore (and it wasn't >>> really >>>> all that easy in the first place). While it may be possible to manage >>> each >>>> repository's website individually and use symlinks in the svn repo to >>> keep >>>> the sites linked together, I think there may be easier ways to manage >>> this >>>> if we took a look at alternative site management tools out there. I've >>>> thought about the possibility that we manage our site in a separate git >>>> repo, but then we'd have to maintain more clear version numbers in the >>>> documentation instead of relying on tagging the docs with the release. >>>> >>>> Besides plain Asciidoc which as been mentioned here before, the only >>> open >>>> source tool I know of that looks interesting here is the one made by >>> vertx: >>>> <https://github.com/vert-x3/vertx-docgen>. See also their site source >>> for >>>> an example on advanced usage: <https://github.com/vert-x3/ve >>> rtx-web-site>. >>>> >>>> On 25 April 2017 at 13:59, Matt Sicker <boa...@gmail.com> wrote: >>>> >>>>> Ideally, the two will align, just like the OSGi modules (which tend to >>>>> directly correspond with maven modules since that's how they're >>> normally >>>>> assembled). >>>>> >>>>> On 25 April 2017 at 13:39, Gary Gregory <garydgreg...@gmail.com> >>> wrote: >>>>> >>>>>> We are going to have to prefix "module" with "Java" or "Maven" in >>>>>> discussions and documentation to avoid confusion from now on... >>>>>> >>>>>> Gary >>>>>> >>>>>> On Apr 25, 2017 10:59 AM, "Matt Sicker" <boa...@gmail.com> wrote: >>>>>> >>>>>>> If you browse around the Java 9 javadocs, you'll see that they >>> split >>>>>> things >>>>>>> up by module there as well. With that in mind, hopefully it's not >>> too >>>>>>> complicated to support. What I really want to see is inter-module >>>> links >>>>>>> (both Java modules and Maven modules that is) work properly. >>>>>>> >>>>>>> On 25 April 2017 at 11:49, Ralph Goers <ralph.go...@dslextreme.com >>>> >>>>>> wrote: >>>>>>> >>>>>>>> Ouch. This is where it gets messy. Currently, the javadoc is >>> built >>>>>>>> independently for each module. I’m not sure how to aggregate them >>>> all >>>>>>>> together but I’m sure Java 9 must be doing that with all the >>> modules >>>>>> they >>>>>>>> are supporting. >>>>>>>> >>>>>>>> Ralph >>>>>>>> >>>>>>>>> On Apr 25, 2017, at 7:03 AM, Mikael Ståldal < >>>>>> mikael.stal...@magine.com >>>>>>>> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> When adding new modules to the main repo, does each module need >>>> its >>>>>> own >>>>>>>>> site directory? >>>>>>>>> >>>>>>>>> On Tue, Apr 25, 2017 at 4:02 PM, Mikael Ståldal < >>>>>>>> mikael.stal...@magine.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Yes, they should stay in the main repo, at least for the time >>>>>> being. >>>>>>>>>> >>>>>>>>>> On Tue, Apr 25, 2017 at 3:56 PM, Gary Gregory < >>>>>> garydgreg...@gmail.com >>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> And all of that should stay in the same repo IMO. >>>>>>>>>>> >>>>>>>>>>> Gary >>>>>>>>>>> >>>>>>>>>>> On Apr 25, 2017 2:51 AM, "Mikael Ståldal" < >>>>>> mikael.stal...@magine.com >>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> I guess that log4-core will become: >>>>>>>>>>>> >>>>>>>>>>>> - log4j-core (will depend on log4j-spi) >>>>>>>>>>>> - log4j-spi >>>>>>>>>>>> - log4j-csv >>>>>>>>>>>> - log4j-xml (XmlLayout) >>>>>>>>>>>> - log4j-json (JsonLayout) >>>>>>>>>>>> - log4j-yaml (YamlLayout) >>>>>>>>>>>> - log4j-kafka >>>>>>>>>>>> - log4j-smtp >>>>>>>>>>>> - log4j-jms >>>>>>>>>>>> - log4j-jdbc (or can this be kept in log4j-core?) >>>>>>>>>>>> - log4j-jpa >>>>>>>>>>>> - log4j-zeromq >>>>>>>>>>>> - log4j-server (already done, not yet released) >>>>>>>>>>>> - log4j-tools (command line tools) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Then we should also split log4j-nosql: >>>>>>>>>>>> >>>>>>>>>>>> - log4j-cassandra >>>>>>>>>>>> - log4j-couchdb >>>>>>>>>>>> - log4j-mongodb >>>>>>>>>>>> - log4j-lucene (new, under development) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Apr 24, 2017 at 7:43 PM, Remko Popma < >>>>>> remko.po...@gmail.com >>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> How many new modules are we talking about, concretely? >>>>>>>>>>>>> >>>>>>>>>>>>> Matt mentioned the StackOverflow questions about transitive >>>>>>>>>>> dependencies >>>>>>>>>>>>> etc, but I imagine splitting log4j-core into 5 or more new >>>>>> modules >>>>>>>>>>> will >>>>>>>>>>>>> also cause confusion... It won't be trivial for users to >>>> figure >>>>>> out >>>>>>>>>>> which >>>>>>>>>>>>> of the many modules they do or don't need. The coarse >>>>>> granularity >>>>>>> of >>>>>>>>>>> the >>>>>>>>>>>>> current modules is a good thing for users. >>>>>>>>>>>>> >>>>>>>>>>>>> What problem are we trying to solve? And how can we solve >>> it >>>>>> with >>>>>>> the >>>>>>>>>>>> least >>>>>>>>>>>>> disruption to our users? >>>>>>>>>>>>> >>>>>>>>>>>>> Would it be an idea, for example, to provide separate jars >>> for >>>>>> the >>>>>>>>>>>> separate >>>>>>>>>>>>> modules, but in addition create a combined jar >>>> (log4j-core-all) >>>>>>> that >>>>>>>>>>>>> contains all the classes in log4j-core as well as the >>> classes >>>> in >>>>>>> the >>>>>>>>>>> new >>>>>>>>>>>>> modules we split out from core? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Apr 25, 2017 at 2:00 AM, Matt Sicker < >>>> boa...@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> I agree with Ralph here. I'm sure we'll figure out rather >>>>>> quickly >>>>>>>>>>> which >>>>>>>>>>>>>> modules are easy to put into rarely updated repositories. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 24 April 2017 at 11:39, Ralph Goers < >>>>>>> ralph.go...@dslextreme.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I would prefer a hybrid approach. First things should be >>>>>> moved >>>>>>> to >>>>>>>>>>>>>>> separate modules. Then, if they don’t seem to be modified >>>>>>>>>>> frequently >>>>>>>>>>>>> they >>>>>>>>>>>>>>> can be moved to a separate repo. For example, I think it >>>>>> would be >>>>>>>>>>> OK >>>>>>>>>>>>> for >>>>>>>>>>>>>>> the Flume Appender to be in a separate repo. It hasn’t >>>>>> changed in >>>>>>>>>>>>> quite a >>>>>>>>>>>>>>> while and I can’t remember the last time it was modified >>> due >>>>>> to >>>>>>>>>>>> changes >>>>>>>>>>>>>> in >>>>>>>>>>>>>>> Log4j it has and while continue to change with changes >>> made >>>> in >>>>>>>>>>> Flume >>>>>>>>>>>>>>> releases. I imagine we have quite a few components that >>> are >>>>>>>>>>> similar. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Ralph >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Apr 24, 2017, at 8:39 AM, Gary Gregory < >>>>>>>>>>> garydgreg...@gmail.com> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Apr 24, 2017 2:38 AM, "Mikael Ståldal" < >>>>>>>>>>>> mikael.stal...@magine.com >>>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I fully agree with Matt's both proposals. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm skeptic to creating more repositories (than we >>> already >>>>>> have) >>>>>>>>>>>>>> though. >>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>> think that we should start by splitting out modules from >>>>>>>>>>> log4j-core >>>>>>>>>>>>> and >>>>>>>>>>>>>>>> keep those modules in the main repository with >>> synchronized >>>>>>>>>>>>> versioning >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>> releases, at least for the 2.9 release. We can always >>> move >>>>>> those >>>>>>>>>>>>>> modules >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>> other repositories later if we want to. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I do not like more repos either. Since we have already >>> gone >>>>>> down >>>>>>>>>>>> the >>>>>>>>>>>>>> more >>>>>>>>>>>>>>>> modules road, I say we keep going. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Gary >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> It is a lot of administrative work to create a new >>>> repository >>>>>>>>>>> (as >>>>>>>>>>>> we >>>>>>>>>>>>>> have >>>>>>>>>>>>>>>> seen for log4j-scala), I don't want us to do all that >>> work >>>>>> over >>>>>>>>>>> and >>>>>>>>>>>>>> over >>>>>>>>>>>>>>>> again unless really necessary. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We have a JIRA ticket for this: >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/LOG4J2-1650 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I have already started by breaking out log4j-server: >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/LOG4J2-1851 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think the next step is to break out plugins (layouts >>> and >>>>>>>>>>>> appenders) >>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>> optional 3rd party dependencies into their own modules. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Sun, Apr 23, 2017 at 7:45 PM, Matt Sicker < >>>>>> boa...@gmail.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I think I brought this topic up like 3 years ago when I >>>> was >>>>>>>>>>>> working >>>>>>>>>>>>> on >>>>>>>>>>>>>>>>> initial OSGi support, but now that we have 3 more years >>>>>> worth >>>>>>>>>>> of >>>>>>>>>>>>> code >>>>>>>>>>>>>>>>> additions and optional features, I think this might be >>> a >>>>>> more >>>>>>>>>>>>>>> appropriate >>>>>>>>>>>>>>>>> time to discuss it again in light of experience. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Building log4j-core itself already takes a long time, >>> and >>>>>> many >>>>>>>>>>>>> plugins >>>>>>>>>>>>>>>>> aren't updated very often at all. In the past, >>> requiring >>>>>> users >>>>>>>>>>> to >>>>>>>>>>>>>> simply >>>>>>>>>>>>>>>>> add log4j-core plus any transitive dependencies to use >>>>>> optional >>>>>>>>>>>>>> features >>>>>>>>>>>>>>>>> seemed to work well enough, but I still think that's a >>>>>>>>>>> confusing >>>>>>>>>>>>>>>>> distribution mechanism as demonstrated by the numerous >>> bug >>>>>>>>>>> reports >>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> Stack Overflow posts regarding missing transitive >>>>>> dependencies >>>>>>>>>>> for >>>>>>>>>>>>>>> various >>>>>>>>>>>>>>>>> features. I spent some time experimenting with Log4j >>> Boot >>>> a >>>>>>>>>>> little >>>>>>>>>>>>>> while >>>>>>>>>>>>>>>>> ago to help alleviate this problem, but this may be >>>>>>>>>>> unnecessary if >>>>>>>>>>>>> we >>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>> agree to modularize log4j-core itself. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I have two different proposals, both of which can be >>> used >>>> at >>>>>>>>>>> the >>>>>>>>>>>>> same >>>>>>>>>>>>>>>> time. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1. Split out everything from log4j-core that requires >>> 3rd >>>>>> party >>>>>>>>>>>>>>>>> dependencies (except for AsyncLogger, though perhaps we >>>>>> could >>>>>>>>>>>>> consider >>>>>>>>>>>>>>>>> shading and renaming those classes like some other low >>>> level >>>>>>>>>>>>> libraries >>>>>>>>>>>>>>> do >>>>>>>>>>>>>>>>> with JCTools). Ideally, I'd like to see each module >>> have >>>>>>>>>>> required >>>>>>>>>>>>>>>>> dependencies instead of optional ones, so that if, for >>>>>>>>>>> instance, I >>>>>>>>>>>>>>> include >>>>>>>>>>>>>>>>> a "log4j-config-yaml" dependency, I know that Log4j >>> will >>>>>>>>>>> support >>>>>>>>>>>>> YAML >>>>>>>>>>>>>>>>> configuration without having to specify the individual >>>>>> Jackson >>>>>>>>>>>>>>>>> dependencies. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 2. Split out from log4j-core a sort of log4j-spi module >>>>>> which >>>>>>>>>>>>> defines >>>>>>>>>>>>>>>>> interfaces, abstract classes, and annotations for >>> plugins >>>>>> that >>>>>>>>>>>> would >>>>>>>>>>>>>> be >>>>>>>>>>>>>>>>> promoted to the same level of backwards compatibility >>>>>>>>>>> guarantees >>>>>>>>>>>> as >>>>>>>>>>>>>>>>> log4j-api. This would aid in cementing what we really >>> wish >>>>>> to >>>>>>>>>>>>> maintain >>>>>>>>>>>>>>>>> compatibility with in the backend while allowing other >>>>>> modules >>>>>>>>>>> to >>>>>>>>>>>>> have >>>>>>>>>>>>>>>> less >>>>>>>>>>>>>>>>> strict guarantees. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> With proposal #1, I'd think that we could more easily >>>> start >>>>>>>>>>> moving >>>>>>>>>>>>>>> modules >>>>>>>>>>>>>>>>> into separate repositories and release trains. Without >>> #2, >>>>>>>>>>> though, >>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>> makes version support more annoying to handle, but >>> that's >>>>>> what >>>>>>>>>>>> we'll >>>>>>>>>>>>>>> face >>>>>>>>>>>>>>>>> regardless as we separate more repositories. If we go >>> this >>>>>>>>>>> route, >>>>>>>>>>>>> then >>>>>>>>>>>>>>>>> there will be no need for a Log4j Boot subproject. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> What do you all think? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> Matt Sicker <boa...@gmail.com> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> [image: MagineTV] >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> *Mikael Ståldal* >>>>>>>>>>>>>>>> Senior software developer >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> *Magine TV* >>>>>>>>>>>>>>>> mikael.stal...@magine.com >>>>>>>>>>>>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | >>>>>>>>>>> www.magine.com >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Privileged and/or Confidential Information may be >>> contained >>>>>> in >>>>>>>>>>> this >>>>>>>>>>>>>>>> message. If you are not the addressee indicated in this >>>>>> message >>>>>>>>>>>>>>>> (or responsible for delivery of the message to such a >>>>>> person), >>>>>>>>>>> you >>>>>>>>>>>>> may >>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>> copy or deliver this message to anyone. In such case, >>>>>>>>>>>>>>>> you should destroy this message and kindly notify the >>>> sender >>>>>> by >>>>>>>>>>>> reply >>>>>>>>>>>>>>>> email. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Matt Sicker <boa...@gmail.com> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> [image: MagineTV] >>>>>>>>>>>> >>>>>>>>>>>> *Mikael Ståldal* >>>>>>>>>>>> Senior software developer >>>>>>>>>>>> >>>>>>>>>>>> *Magine TV* >>>>>>>>>>>> mikael.stal...@magine.com >>>>>>>>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | >>>> www.magine.com >>>>>>>>>>>> >>>>>>>>>>>> Privileged and/or Confidential Information may be contained >>> in >>>>>> this >>>>>>>>>>>> message. If you are not the addressee indicated in this >>> message >>>>>>>>>>>> (or responsible for delivery of the message to such a >>> person), >>>>>> you >>>>>>> may >>>>>>>>>>> not >>>>>>>>>>>> copy or deliver this message to anyone. In such case, >>>>>>>>>>>> you should destroy this message and kindly notify the >>> sender by >>>>>>> reply >>>>>>>>>>>> email. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> [image: MagineTV] >>>>>>>>>> >>>>>>>>>> *Mikael Ståldal* >>>>>>>>>> Senior software developer >>>>>>>>>> >>>>>>>>>> *Magine TV* >>>>>>>>>> mikael.stal...@magine.com >>>>>>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | >>> www.magine.com >>>>>>>>>> >>>>>>>>>> Privileged and/or Confidential Information may be contained in >>>> this >>>>>>>>>> message. If you are not the addressee indicated in this >>> message >>>>>>>>>> (or responsible for delivery of the message to such a person), >>>> you >>>>>> may >>>>>>>> not >>>>>>>>>> copy or deliver this message to anyone. In such case, >>>>>>>>>> you should destroy this message and kindly notify the sender >>> by >>>>>> reply >>>>>>>>>> email. >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> [image: MagineTV] >>>>>>>>> >>>>>>>>> *Mikael Ståldal* >>>>>>>>> Senior software developer >>>>>>>>> >>>>>>>>> *Magine TV* >>>>>>>>> mikael.stal...@magine.com >>>>>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | >>> www.magine.com >>>>>>>>> >>>>>>>>> Privileged and/or Confidential Information may be contained in >>>> this >>>>>>>>> message. If you are not the addressee indicated in this message >>>>>>>>> (or responsible for delivery of the message to such a person), >>> you >>>>>> may >>>>>>>> not >>>>>>>>> copy or deliver this message to anyone. In such case, >>>>>>>>> you should destroy this message and kindly notify the sender by >>>>>> reply >>>>>>>>> email. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Matt Sicker <boa...@gmail.com> >>>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Matt Sicker <boa...@gmail.com> >>>>> >>>> >>>> >>>> >>>> -- >>>> Matt Sicker <boa...@gmail.com> >>>> >>> >> >> >> >> -- >> Matt Sicker <boa...@gmail.com> >> > > > > -- > Matt Sicker <boa...@gmail.com>
