And yes, one repo per module is another option, a la Maven as you point out, but what is being proposed is some halfway solution which to me feels confusing and arbitrary just because today this and that module is in favor but that some others.
Gaty On Sun, May 1, 2022, 07:54 Gary Gregory <[email protected]> wrote: > Hi Volkan, > > I see how you build and how you do version control as orthogonal concerns. > Currently we build and deliver everything from the root of the git checkout > directory. It does not have to be that way is all I'm saying. One could > just cd to a sub directory and work there. You could still have the option > to build the world from the root as a convenient way to make sure > everything works together and there is no breakage. > > https://en.m.wikipedia.org/wiki/Monorepo > > Google,[5] Facebook,[6] Microsoft,[7] Uber,[8] Airbnb, and Twitter[9] all > employ very large monorepos with varying strategies to scale build systems > and version control software with a large volume of code and daily changes. > > Log4j is tiny in comparison. > > Gary > > On Sat, Apr 30, 2022, 16:20 Volkan Yazıcı <[email protected]> wrote: > >> I disagree that multi-repo setup is a recipe for disaster. It works >> perfectly well for Maven. Actually their setup is way more complicated: >> dedicated repo for each module! But let's assume you're right and >> multi-repos decrease the shelf time. Isn't that something good? I like >> repos that rot, it gives a clear message about that project: it simply is >> dead. If we migrate to multiple repos and some never get to make a >> release, >> what does this mean? Apparently nothing interesting is happening here. If >> there is a demand from the community to warrant a release, I am sure that >> release will come. After all, the very same community attempted to >> resurrect Log4j 1. >> >> I also like single repo setups where build and test times are within >> bearable limits, though Log4j is not one of them. Note that multi-repo >> discussion started, because we don't know any other way to solve the >> current set of problems. If you can think of how to address these concerns >> in some other way (e.g., "move folders around"), I am all ears. >> >> On Sat, Apr 30, 2022 at 2:54 PM Gary Gregory <[email protected]> >> wrote: >> >> > Breaking our mono repo into a bunch of smaller repo pieces is a recipe >> for >> > software rot and orphanage IMO. >> > >> > For example, we have never released out of the tools repo, and I have >> folks >> > at work at use code from that repo for some internal facing work. Now, I >> > could take the time and do a release of course but... priorities. OTOH, >> I >> > could have gotten a release for "free" if that code was in the mono >> repo (I >> > am not advocating to put it back). >> > >> > This email might be better articulated in a call but I am trying to be >> > constructive: I think we can keep our mono repo and develop faster by >> > reorganizing what we have now. I like having everything in one place >> > myself. I am not sure how to actually move folders around to achieve >> > whatever new release cycle/split up we want, but I am sure it is do >> able. >> > >> > Gary >> > >> > On Fri, Apr 29, 2022, 16:03 Volkan Yazıcı <[email protected]> wrote: >> > >> > > I have raised this issue in the past. I even came up with a proposal >> on >> > the >> > > repository structure >> > > <https://lists.apache.org/thread/sdv92pw6m9bst8zzn88jyln8z8ozshw3> >> to be >> > > followed. I am a big proponent of breaking up the logging-log4j2 >> > repository >> > > into multiple ones. The current size of the code base is simply >> > > incomprehensible and slows down the development cycle tremendously >> due to >> > > excessive compilation and test times. >> > > >> > > *The proposal* >> > > >> > > Let me share the break down I have in mind: >> > > >> > > *logging-log4j2-api* >> > > log4j-1.2-api >> > > log4j-api >> > > >> > > *logging-log4j2* >> > > log4j-bom >> > > log4j-core >> > > log4j-core-its >> > > log4j-gctests >> > > log4j-iostreams >> > > log4j-perf >> > > log4j-plugins >> > > >> > > *logging-log4j2-appender* >> > > log4j-cassandra >> > > log4j-couchdb >> > > log4j-csv >> > > log4j-flume-ng >> > > log4j-mongodb3 >> > > log4j-mongodb4 >> > > log4j-smtp >> > > >> > > *logging-log4j2-appender-jdbc* >> > > log4j-jdbc (?) >> > > log4j-jdbc-dbcp2 (?) >> > > log4j-jpa >> > > >> > > *logging-log4j2-appender-queue* >> > > log4j-jeromq >> > > log4j-jms >> > > log4j-kafka >> > > log4j-redis >> > > >> > > *logging-log4j2-binding* >> > > log4j-jcl >> > > log4j-jpl >> > > log4j-jul >> > > log4j-liquibase >> > > log4j-slf4j18-impl >> > > log4j-slf4j-impl >> > > log4j-to-slf4j >> > > >> > > *logging-log4j2-container* >> > > log4j-docker >> > > log4j-kubernetes >> > > >> > > *logging-log4j2-gui* >> > > log4j-jmx-gui >> > > >> > > *logging-log4j2-jee* >> > > log4j-appserver >> > > log4j-taglib >> > > log4j-web >> > > >> > > *logging-log4j2-layout-json* >> > > log4j-layout-gelf >> > > log4j-layout-jackson >> > > log4j-layout-jackson-json >> > > log4j-layout-jackson-xml >> > > log4j-layout-jackson-yaml >> > > log4j-layout-template-json >> > > >> > > *logging-log4j2-osgi* >> > > log4j-osgi >> > > >> > > *logging-log4j2-spring* >> > > log4j-spring-boot >> > > log4j-spring-cloud-config >> > > >> > > Note that above breakdown contains every available module in master, >> > except >> > > log4j-samples, which has the following sub-modules: >> > > >> > > log4j-samples-loggerProperties (contains 2 very trivial example >> classes, >> > > log4j-core [tests] contains more comprehensive alternatives) >> > > log4j-samples-flume-remote (flume-specific) >> > > log4j-samples-flume-embedded (flume-specific) >> > > log4j-samples-flume-common (flume-specific) >> > > log4j-samples-configuration (contains 3 very trivial example classes, >> > > log4j-core [tests] contains more comprehensive alternatives) >> > > >> > > I propose removing the log4j-samples module and, if necessary, moving >> > > Flume-related parts to the log4j-flume-ng. >> > > >> > > *The problem with multiple repositories* >> > > >> > > I made this proposal in June 2021. Since then the more my familiarity >> > with >> > > the website and manual generation increased, the less I became >> inclined >> > to >> > > move forward. I am not much concerned about the increased release >> labor >> > the >> > > Ralph has mentioned – a majority of these repos will hardly get an >> update >> > > to warrant a release. Though the fact that current website generation >> > > depends on the presence of these modules to be in the same repository >> and >> > > with multiple repositories, the manual and source codes will be in >> > > different repositories... These concern me a lot. I think, the steps >> to >> > > overhaul the project structure should better be in the following >> order: >> > > >> > > 1. Replace `site` phase (`site` phase won't be needed anymore. >> Website >> > > will independently be generated in a single step via JBake.) >> > > 2. Rewrite the manual >> > > 3. Migrate to a multi-repo structure (Each repo will have its own >> > > website and manual, ala Maven. Here the code in steps 1 and 2 will >> be >> > > reused.) >> > > >> > > >> > > >> > > On Wed, Apr 27, 2022 at 1:35 PM Ralph Goers < >> [email protected]> >> > > wrote: >> > > >> > > > I wouldn’t necessarily want to penalize them. At the same time, I >> don’t >> > > > think we want a ton of extra repos each of which needs its own >> release. >> > > > >> > > > Ralph >> > > > >> > > > > On Apr 27, 2022, at 12:26 PM, Piotr P. Karwasz < >> > > [email protected]> >> > > > wrote: >> > > > > >> > > > > Hello, >> > > > > >> > > > > On Wed, 27 Apr 2022 at 11:33, Volkan Yazıcı <[email protected]> >> wrote: >> > > > > >> > > > >> +1 with moving Cassandra et al. to their own repo >> > > > >> >> > > > > >> > > > > I agree: Cassandra's tests were one of the most common reasons for >> > > > failing >> > > > > CI. >> > > > > Maybe it could share a repository with other database appenders. >> > > > > >> > > > > Piotr >> > > > >> > > > >> > > >> > >> >
