Just to make sure this sounds reasonable, here's my idea for a roadmap: 2.9: separate out more modules (not necessarily into their own repo or repos yet) so that log4j-core contains minimal dependencies. 2.10: log4j-core-spi or whatever the name, making it easier for other modules to release on their own without being synced up with log4j-core versions
With the current JPMS drama going on, it seems like we may have more time before Java 9 is released, so we should have time to follow this path rather than attempting the full on modularization all in one go. On 30 April 2017 at 11:56, Matt Sicker <boa...@gmail.com> wrote: > I doubt Google uses Git then. > > One of the main goals of separating repositories is to make release > management easier so we can RERO more! Though part of the problem there is > that for some reason, the release process requires running all the tests > and whatnot at least 3 times or more. > > On 30 April 2017 at 11:45, Gary Gregory <garydgreg...@gmail.com> wrote: > >> Maybe more than one repo isn't such a good idea? I hear Google uses a >> single repo for all their code... >> >> Gary >> >> On Apr 30, 2017 9:41 AM, "Matt Sicker" <boa...@gmail.com> wrote: >> >> > I've noticed just with the Scala repo that integrating various >> repositories >> > into a single coherent website is not so easy anymore (and it wasn't >> really >> > all that easy in the first place). While it may be possible to manage >> each >> > repository's website individually and use symlinks in the svn repo to >> keep >> > the sites linked together, I think there may be easier ways to manage >> this >> > if we took a look at alternative site management tools out there. I've >> > thought about the possibility that we manage our site in a separate git >> > repo, but then we'd have to maintain more clear version numbers in the >> > documentation instead of relying on tagging the docs with the release. >> > >> > Besides plain Asciidoc which as been mentioned here before, the only >> open >> > source tool I know of that looks interesting here is the one made by >> vertx: >> > <https://github.com/vert-x3/vertx-docgen>. See also their site source >> for >> > an example on advanced usage: <https://github.com/vert-x3/ve >> rtx-web-site>. >> > >> > On 25 April 2017 at 13:59, Matt Sicker <boa...@gmail.com> wrote: >> > >> > > Ideally, the two will align, just like the OSGi modules (which tend to >> > > directly correspond with maven modules since that's how they're >> normally >> > > assembled). >> > > >> > > On 25 April 2017 at 13:39, Gary Gregory <garydgreg...@gmail.com> >> wrote: >> > > >> > >> We are going to have to prefix "module" with "Java" or "Maven" in >> > >> discussions and documentation to avoid confusion from now on... >> > >> >> > >> Gary >> > >> >> > >> On Apr 25, 2017 10:59 AM, "Matt Sicker" <boa...@gmail.com> wrote: >> > >> >> > >> > If you browse around the Java 9 javadocs, you'll see that they >> split >> > >> things >> > >> > up by module there as well. With that in mind, hopefully it's not >> too >> > >> > complicated to support. What I really want to see is inter-module >> > links >> > >> > (both Java modules and Maven modules that is) work properly. >> > >> > >> > >> > On 25 April 2017 at 11:49, Ralph Goers <ralph.go...@dslextreme.com >> > >> > >> wrote: >> > >> > >> > >> > > Ouch. This is where it gets messy. Currently, the javadoc is >> built >> > >> > > independently for each module. I’m not sure how to aggregate them >> > all >> > >> > > together but I’m sure Java 9 must be doing that with all the >> modules >> > >> they >> > >> > > are supporting. >> > >> > > >> > >> > > Ralph >> > >> > > >> > >> > > > On Apr 25, 2017, at 7:03 AM, Mikael Ståldal < >> > >> mikael.stal...@magine.com >> > >> > > >> > >> > > wrote: >> > >> > > > >> > >> > > > When adding new modules to the main repo, does each module need >> > its >> > >> own >> > >> > > > site directory? >> > >> > > > >> > >> > > > On Tue, Apr 25, 2017 at 4:02 PM, Mikael Ståldal < >> > >> > > mikael.stal...@magine.com> >> > >> > > > wrote: >> > >> > > > >> > >> > > >> Yes, they should stay in the main repo, at least for the time >> > >> being. >> > >> > > >> >> > >> > > >> On Tue, Apr 25, 2017 at 3:56 PM, Gary Gregory < >> > >> garydgreg...@gmail.com >> > >> > > >> > >> > > >> wrote: >> > >> > > >> >> > >> > > >>> And all of that should stay in the same repo IMO. >> > >> > > >>> >> > >> > > >>> Gary >> > >> > > >>> >> > >> > > >>> On Apr 25, 2017 2:51 AM, "Mikael Ståldal" < >> > >> mikael.stal...@magine.com >> > >> > > >> > >> > > >>> wrote: >> > >> > > >>> >> > >> > > >>>> I guess that log4-core will become: >> > >> > > >>>> >> > >> > > >>>> - log4j-core (will depend on log4j-spi) >> > >> > > >>>> - log4j-spi >> > >> > > >>>> - log4j-csv >> > >> > > >>>> - log4j-xml (XmlLayout) >> > >> > > >>>> - log4j-json (JsonLayout) >> > >> > > >>>> - log4j-yaml (YamlLayout) >> > >> > > >>>> - log4j-kafka >> > >> > > >>>> - log4j-smtp >> > >> > > >>>> - log4j-jms >> > >> > > >>>> - log4j-jdbc (or can this be kept in log4j-core?) >> > >> > > >>>> - log4j-jpa >> > >> > > >>>> - log4j-zeromq >> > >> > > >>>> - log4j-server (already done, not yet released) >> > >> > > >>>> - log4j-tools (command line tools) >> > >> > > >>>> >> > >> > > >>>> >> > >> > > >>>> Then we should also split log4j-nosql: >> > >> > > >>>> >> > >> > > >>>> - log4j-cassandra >> > >> > > >>>> - log4j-couchdb >> > >> > > >>>> - log4j-mongodb >> > >> > > >>>> - log4j-lucene (new, under development) >> > >> > > >>>> >> > >> > > >>>> >> > >> > > >>>> >> > >> > > >>>> On Mon, Apr 24, 2017 at 7:43 PM, Remko Popma < >> > >> remko.po...@gmail.com >> > >> > > >> > >> > > >>>> wrote: >> > >> > > >>>> >> > >> > > >>>>> How many new modules are we talking about, concretely? >> > >> > > >>>>> >> > >> > > >>>>> Matt mentioned the StackOverflow questions about transitive >> > >> > > >>> dependencies >> > >> > > >>>>> etc, but I imagine splitting log4j-core into 5 or more new >> > >> modules >> > >> > > >>> will >> > >> > > >>>>> also cause confusion... It won't be trivial for users to >> > figure >> > >> out >> > >> > > >>> which >> > >> > > >>>>> of the many modules they do or don't need. The coarse >> > >> granularity >> > >> > of >> > >> > > >>> the >> > >> > > >>>>> current modules is a good thing for users. >> > >> > > >>>>> >> > >> > > >>>>> What problem are we trying to solve? And how can we solve >> it >> > >> with >> > >> > the >> > >> > > >>>> least >> > >> > > >>>>> disruption to our users? >> > >> > > >>>>> >> > >> > > >>>>> Would it be an idea, for example, to provide separate jars >> for >> > >> the >> > >> > > >>>> separate >> > >> > > >>>>> modules, but in addition create a combined jar >> > (log4j-core-all) >> > >> > that >> > >> > > >>>>> contains all the classes in log4j-core as well as the >> classes >> > in >> > >> > the >> > >> > > >>> new >> > >> > > >>>>> modules we split out from core? >> > >> > > >>>>> >> > >> > > >>>>> >> > >> > > >>>>> On Tue, Apr 25, 2017 at 2:00 AM, Matt Sicker < >> > boa...@gmail.com> >> > >> > > >>> wrote: >> > >> > > >>>>> >> > >> > > >>>>>> I agree with Ralph here. I'm sure we'll figure out rather >> > >> quickly >> > >> > > >>> which >> > >> > > >>>>>> modules are easy to put into rarely updated repositories. >> > >> > > >>>>>> >> > >> > > >>>>>> On 24 April 2017 at 11:39, Ralph Goers < >> > >> > ralph.go...@dslextreme.com> >> > >> > > >>>>> wrote: >> > >> > > >>>>>> >> > >> > > >>>>>>> I would prefer a hybrid approach. First things should be >> > >> moved >> > >> > to >> > >> > > >>>>>>> separate modules. Then, if they don’t seem to be modified >> > >> > > >>> frequently >> > >> > > >>>>> they >> > >> > > >>>>>>> can be moved to a separate repo. For example, I think it >> > >> would be >> > >> > > >>> OK >> > >> > > >>>>> for >> > >> > > >>>>>>> the Flume Appender to be in a separate repo. It hasn’t >> > >> changed in >> > >> > > >>>>> quite a >> > >> > > >>>>>>> while and I can’t remember the last time it was modified >> due >> > >> to >> > >> > > >>>> changes >> > >> > > >>>>>> in >> > >> > > >>>>>>> Log4j it has and while continue to change with changes >> made >> > in >> > >> > > >>> Flume >> > >> > > >>>>>>> releases. I imagine we have quite a few components that >> are >> > >> > > >>> similar. >> > >> > > >>>>>>> >> > >> > > >>>>>>> Ralph >> > >> > > >>>>>>> >> > >> > > >>>>>>>> On Apr 24, 2017, at 8:39 AM, Gary Gregory < >> > >> > > >>> garydgreg...@gmail.com> >> > >> > > >>>>>>> wrote: >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> On Apr 24, 2017 2:38 AM, "Mikael Ståldal" < >> > >> > > >>>> mikael.stal...@magine.com >> > >> > > >>>>>> >> > >> > > >>>>>>> wrote: >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> I fully agree with Matt's both proposals. >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> I'm skeptic to creating more repositories (than we >> already >> > >> have) >> > >> > > >>>>>> though. >> > >> > > >>>>>>> I >> > >> > > >>>>>>>> think that we should start by splitting out modules from >> > >> > > >>> log4j-core >> > >> > > >>>>> and >> > >> > > >>>>>>>> keep those modules in the main repository with >> synchronized >> > >> > > >>>>> versioning >> > >> > > >>>>>>> and >> > >> > > >>>>>>>> releases, at least for the 2.9 release. We can always >> move >> > >> those >> > >> > > >>>>>> modules >> > >> > > >>>>>>> to >> > >> > > >>>>>>>> other repositories later if we want to. >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> I do not like more repos either. Since we have already >> gone >> > >> down >> > >> > > >>>> the >> > >> > > >>>>>> more >> > >> > > >>>>>>>> modules road, I say we keep going. >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> Gary >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> It is a lot of administrative work to create a new >> > repository >> > >> > > >>> (as >> > >> > > >>>> we >> > >> > > >>>>>> have >> > >> > > >>>>>>>> seen for log4j-scala), I don't want us to do all that >> work >> > >> over >> > >> > > >>> and >> > >> > > >>>>>> over >> > >> > > >>>>>>>> again unless really necessary. >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> We have a JIRA ticket for this: >> > >> > > >>>>>>>> https://issues.apache.org/jira/browse/LOG4J2-1650 >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> I have already started by breaking out log4j-server: >> > >> > > >>>>>>>> https://issues.apache.org/jira/browse/LOG4J2-1851 >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> I think the next step is to break out plugins (layouts >> and >> > >> > > >>>> appenders) >> > >> > > >>>>>>> with >> > >> > > >>>>>>>> optional 3rd party dependencies into their own modules. >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> On Sun, Apr 23, 2017 at 7:45 PM, Matt Sicker < >> > >> boa...@gmail.com> >> > >> > > >>>>> wrote: >> > >> > > >>>>>>>> >> > >> > > >>>>>>>>> I think I brought this topic up like 3 years ago when I >> > was >> > >> > > >>>> working >> > >> > > >>>>> on >> > >> > > >>>>>>>>> initial OSGi support, but now that we have 3 more years >> > >> worth >> > >> > > >>> of >> > >> > > >>>>> code >> > >> > > >>>>>>>>> additions and optional features, I think this might be >> a >> > >> more >> > >> > > >>>>>>> appropriate >> > >> > > >>>>>>>>> time to discuss it again in light of experience. >> > >> > > >>>>>>>>> >> > >> > > >>>>>>>>> Building log4j-core itself already takes a long time, >> and >> > >> many >> > >> > > >>>>> plugins >> > >> > > >>>>>>>>> aren't updated very often at all. In the past, >> requiring >> > >> users >> > >> > > >>> to >> > >> > > >>>>>> simply >> > >> > > >>>>>>>>> add log4j-core plus any transitive dependencies to use >> > >> optional >> > >> > > >>>>>> features >> > >> > > >>>>>>>>> seemed to work well enough, but I still think that's a >> > >> > > >>> confusing >> > >> > > >>>>>>>>> distribution mechanism as demonstrated by the numerous >> bug >> > >> > > >>> reports >> > >> > > >>>>> and >> > >> > > >>>>>>>>> Stack Overflow posts regarding missing transitive >> > >> dependencies >> > >> > > >>> for >> > >> > > >>>>>>> various >> > >> > > >>>>>>>>> features. I spent some time experimenting with Log4j >> Boot >> > a >> > >> > > >>> little >> > >> > > >>>>>> while >> > >> > > >>>>>>>>> ago to help alleviate this problem, but this may be >> > >> > > >>> unnecessary if >> > >> > > >>>>> we >> > >> > > >>>>>>> can >> > >> > > >>>>>>>>> agree to modularize log4j-core itself. >> > >> > > >>>>>>>>> >> > >> > > >>>>>>>>> I have two different proposals, both of which can be >> used >> > at >> > >> > > >>> the >> > >> > > >>>>> same >> > >> > > >>>>>>>> time. >> > >> > > >>>>>>>>> >> > >> > > >>>>>>>>> 1. Split out everything from log4j-core that requires >> 3rd >> > >> party >> > >> > > >>>>>>>>> dependencies (except for AsyncLogger, though perhaps we >> > >> could >> > >> > > >>>>> consider >> > >> > > >>>>>>>>> shading and renaming those classes like some other low >> > level >> > >> > > >>>>> libraries >> > >> > > >>>>>>> do >> > >> > > >>>>>>>>> with JCTools). Ideally, I'd like to see each module >> have >> > >> > > >>> required >> > >> > > >>>>>>>>> dependencies instead of optional ones, so that if, for >> > >> > > >>> instance, I >> > >> > > >>>>>>> include >> > >> > > >>>>>>>>> a "log4j-config-yaml" dependency, I know that Log4j >> will >> > >> > > >>> support >> > >> > > >>>>> YAML >> > >> > > >>>>>>>>> configuration without having to specify the individual >> > >> Jackson >> > >> > > >>>>>>>>> dependencies. >> > >> > > >>>>>>>>> >> > >> > > >>>>>>>>> 2. Split out from log4j-core a sort of log4j-spi module >> > >> which >> > >> > > >>>>> defines >> > >> > > >>>>>>>>> interfaces, abstract classes, and annotations for >> plugins >> > >> that >> > >> > > >>>> would >> > >> > > >>>>>> be >> > >> > > >>>>>>>>> promoted to the same level of backwards compatibility >> > >> > > >>> guarantees >> > >> > > >>>> as >> > >> > > >>>>>>>>> log4j-api. This would aid in cementing what we really >> wish >> > >> to >> > >> > > >>>>> maintain >> > >> > > >>>>>>>>> compatibility with in the backend while allowing other >> > >> modules >> > >> > > >>> to >> > >> > > >>>>> have >> > >> > > >>>>>>>> less >> > >> > > >>>>>>>>> strict guarantees. >> > >> > > >>>>>>>>> >> > >> > > >>>>>>>>> With proposal #1, I'd think that we could more easily >> > start >> > >> > > >>> moving >> > >> > > >>>>>>> modules >> > >> > > >>>>>>>>> into separate repositories and release trains. Without >> #2, >> > >> > > >>> though, >> > >> > > >>>>>> this >> > >> > > >>>>>>>>> makes version support more annoying to handle, but >> that's >> > >> what >> > >> > > >>>> we'll >> > >> > > >>>>>>> face >> > >> > > >>>>>>>>> regardless as we separate more repositories. If we go >> this >> > >> > > >>> route, >> > >> > > >>>>> then >> > >> > > >>>>>>>>> there will be no need for a Log4j Boot subproject. >> > >> > > >>>>>>>>> >> > >> > > >>>>>>>>> What do you all think? >> > >> > > >>>>>>>>> >> > >> > > >>>>>>>>> -- >> > >> > > >>>>>>>>> Matt Sicker <boa...@gmail.com> >> > >> > > >>>>>>>>> >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> -- >> > >> > > >>>>>>>> [image: MagineTV] >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> *Mikael Ståldal* >> > >> > > >>>>>>>> Senior software developer >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> *Magine TV* >> > >> > > >>>>>>>> mikael.stal...@magine.com >> > >> > > >>>>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | >> > >> > > >>> www.magine.com >> > >> > > >>>>>>>> >> > >> > > >>>>>>>> Privileged and/or Confidential Information may be >> contained >> > >> in >> > >> > > >>> this >> > >> > > >>>>>>>> message. If you are not the addressee indicated in this >> > >> message >> > >> > > >>>>>>>> (or responsible for delivery of the message to such a >> > >> person), >> > >> > > >>> you >> > >> > > >>>>> may >> > >> > > >>>>>>> not >> > >> > > >>>>>>>> copy or deliver this message to anyone. In such case, >> > >> > > >>>>>>>> you should destroy this message and kindly notify the >> > sender >> > >> by >> > >> > > >>>> reply >> > >> > > >>>>>>>> email. >> > >> > > >>>>>>> >> > >> > > >>>>>>> >> > >> > > >>>>>>> >> > >> > > >>>>>> >> > >> > > >>>>>> >> > >> > > >>>>>> -- >> > >> > > >>>>>> Matt Sicker <boa...@gmail.com> >> > >> > > >>>>>> >> > >> > > >>>>> >> > >> > > >>>> >> > >> > > >>>> >> > >> > > >>>> >> > >> > > >>>> -- >> > >> > > >>>> [image: MagineTV] >> > >> > > >>>> >> > >> > > >>>> *Mikael Ståldal* >> > >> > > >>>> Senior software developer >> > >> > > >>>> >> > >> > > >>>> *Magine TV* >> > >> > > >>>> mikael.stal...@magine.com >> > >> > > >>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | >> > www.magine.com >> > >> > > >>>> >> > >> > > >>>> Privileged and/or Confidential Information may be contained >> in >> > >> this >> > >> > > >>>> message. If you are not the addressee indicated in this >> message >> > >> > > >>>> (or responsible for delivery of the message to such a >> person), >> > >> you >> > >> > may >> > >> > > >>> not >> > >> > > >>>> copy or deliver this message to anyone. In such case, >> > >> > > >>>> you should destroy this message and kindly notify the >> sender by >> > >> > reply >> > >> > > >>>> email. >> > >> > > >>>> >> > >> > > >>> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> -- >> > >> > > >> [image: MagineTV] >> > >> > > >> >> > >> > > >> *Mikael Ståldal* >> > >> > > >> Senior software developer >> > >> > > >> >> > >> > > >> *Magine TV* >> > >> > > >> mikael.stal...@magine.com >> > >> > > >> Grev Turegatan 3 | 114 46 Stockholm, Sweden | >> www.magine.com >> > >> > > >> >> > >> > > >> Privileged and/or Confidential Information may be contained in >> > this >> > >> > > >> message. If you are not the addressee indicated in this >> message >> > >> > > >> (or responsible for delivery of the message to such a person), >> > you >> > >> may >> > >> > > not >> > >> > > >> copy or deliver this message to anyone. In such case, >> > >> > > >> you should destroy this message and kindly notify the sender >> by >> > >> reply >> > >> > > >> email. >> > >> > > >> >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > -- >> > >> > > > [image: MagineTV] >> > >> > > > >> > >> > > > *Mikael Ståldal* >> > >> > > > Senior software developer >> > >> > > > >> > >> > > > *Magine TV* >> > >> > > > mikael.stal...@magine.com >> > >> > > > Grev Turegatan 3 | 114 46 Stockholm, Sweden | >> www.magine.com >> > >> > > > >> > >> > > > Privileged and/or Confidential Information may be contained in >> > this >> > >> > > > message. If you are not the addressee indicated in this message >> > >> > > > (or responsible for delivery of the message to such a person), >> you >> > >> may >> > >> > > not >> > >> > > > copy or deliver this message to anyone. In such case, >> > >> > > > you should destroy this message and kindly notify the sender by >> > >> reply >> > >> > > > email. >> > >> > > >> > >> > > >> > >> > > >> > >> > >> > >> > >> > >> > -- >> > >> > Matt Sicker <boa...@gmail.com> >> > >> > >> > >> >> > > >> > > >> > > >> > > -- >> > > Matt Sicker <boa...@gmail.com> >> > > >> > >> > >> > >> > -- >> > Matt Sicker <boa...@gmail.com> >> > >> > > > > -- > Matt Sicker <boa...@gmail.com> > -- Matt Sicker <boa...@gmail.com>
