Re: [log4cxx] Next steps / release?

[Tobias Frost]

On Thu, Dec 29, 2022 at 05:21:34PM -0500, Robert Middleton wrote:
> With some fixes that have been accomplished over the past few weeks,
> we now have only 1-2 issues in JIRA that would need to be finished for
> a release, and I think both of those are effectively done at this
> point.
> 
> I'm pretty happy with where things are at the moment - there's always
> more tweaking that can be done, but I can't think of anything major
> that needs to be done that could be done within a reasonable
> timeframe.  The major thing that was done for the next release was to
> make things ABI stable, so as long as we commit to that going
> forward(and only breaking it with proper versioning) we should be
> good.
> 
> The last time we talked about this Tobias Frost said that the
> soft-freeze for Debian is the 12th of January[1], so after that point
> an updated library wouldn't make it into Debian.  I would like to get
> this version into Debian if possible(as that is the distribution I
> use), but that depends on Tobias' availability.

Yes, I'm generally available.
> With that in mind, should we do a release in the immediate
> future(within the next ~7 days), or should we wait a bit for some more
> tweaking and/or features?
> 

As said, if the new version does _not_ need an SONAME bump, the deadline
is February 12th (at which time it must be in "testing", so effectivly I
have to upload it minmum 10 days earlier. (12 to be safe).

IF an SONAME bump is required, it will become more challenging, as other
Debian teams are involved (and other maintainers if it casues breakages in
reverse dependencies)
I would basically needs something (it does not need to be the final
release, just something with the target SONAME) now to kick of the process. 
Fixes
(without SONAME bump) can still then be provided until February.

-- 
Cheers,
tobi

 
> -Robert Middleton
> 
> [1]: https://lists.apache.org/thread/zg150rbkdkzgog1bnd403052818nncs7

Re: Maven site failure

[Ralph Goers]

> On Dec 29, 2022, at 5:44 PM, Ralph Goers  wrote:
> 
> I am trying to build the web site and it is failing in the javadoc plugin in 
> log4j-layout-template-json-test because there are public or protected classes 
> to document. Either the javadoc plugin needs to be disabled for the module or 
> some classes need to be made public.

Personally, I don’t think any of the test modules need javadoc generated.

Ralph

Maven site failure

[Ralph Goers]

I am trying to build the web site and it is failing in the javadoc plugin in 
log4j-layout-template-json-test because there are public or protected classes 
to document. Either the javadoc plugin needs to be disabled for the module or 
some classes need to be made public.

Ralph

Re: [log4cxx] Next steps / release?

[Stephen Webb]

I think the next_stable branch should be released now.

I am not aware of any issue that would prevent a release.

I will do some more tests with production code this weekend to ensure there
is no anything I have missed.

On Fri, Dec 30, 2022 at 9:21 AM Robert Middleton 
wrote:

> With some fixes that have been accomplished over the past few weeks,
> we now have only 1-2 issues in JIRA that would need to be finished for
> a release, and I think both of those are effectively done at this
> point.
>
> I'm pretty happy with where things are at the moment - there's always
> more tweaking that can be done, but I can't think of anything major
> that needs to be done that could be done within a reasonable
> timeframe.  The major thing that was done for the next release was to
> make things ABI stable, so as long as we commit to that going
> forward(and only breaking it with proper versioning) we should be
> good.
>
> The last time we talked about this Tobias Frost said that the
> soft-freeze for Debian is the 12th of January[1], so after that point
> an updated library wouldn't make it into Debian.  I would like to get
> this version into Debian if possible(as that is the distribution I
> use), but that depends on Tobias' availability.
>
> With that in mind, should we do a release in the immediate
> future(within the next ~7 days), or should we wait a bit for some more
> tweaking and/or features?
>
> -Robert Middleton
>
> [1]: https://lists.apache.org/thread/zg150rbkdkzgog1bnd403052818nncs7
>

[log4cxx] Next steps / release?

[Robert Middleton]

With some fixes that have been accomplished over the past few weeks,
we now have only 1-2 issues in JIRA that would need to be finished for
a release, and I think both of those are effectively done at this
point.

I'm pretty happy with where things are at the moment - there's always
more tweaking that can be done, but I can't think of anything major
that needs to be done that could be done within a reasonable
timeframe.  The major thing that was done for the next release was to
make things ABI stable, so as long as we commit to that going
forward(and only breaking it with proper versioning) we should be
good.

The last time we talked about this Tobias Frost said that the
soft-freeze for Debian is the 12th of January[1], so after that point
an updated library wouldn't make it into Debian.  I would like to get
this version into Debian if possible(as that is the distribution I
use), but that depends on Tobias' availability.

With that in mind, should we do a release in the immediate
future(within the next ~7 days), or should we wait a bit for some more
tweaking and/or features?

-Robert Middleton

[1]: https://lists.apache.org/thread/zg150rbkdkzgog1bnd403052818nncs7

Re: Log4Net 2.014

[Ralph Goers]

Rupak and Kevin,

Please note that your emails contain a classification that these are internal 
Schwab emails but that you have sent them to a public mailing list. We 
obviously ignore such classifications since there is no way to make them 
private once you have done that.

I can’t comment on your specific issue as I don’t work on Log4Net so I will 
leave that to those that do. 

Ralph

> On Dec 29, 2022, at 1:24 PM, Seal, Rupak  
> wrote:
> 
> HI @Tools Support is there any ways to have 
> this work being escalated as this impacted production stability and damaging 
> the logging server in Production SNE environment
> 
> From: Gardenhire, Kevin 
> Date: Thursday, December 29, 2022 at 2:12 PM
> To: dev@logging.apache.org 
> Cc: Joshi, Hemant , Vishwakarma, Amit 
> , Seal, Rupak , 
> Shambhumane, Meenakshi 
> Subject: Log4Net 2.014
> Hi,
> 
> We recently upgraded our Log4Net package from 1.2.13 to 2.0.14 to remediate a 
> security flaw found in the old package. After upgrading we have found an 
> issue that is not allowing 2 instances of an application to write to the same 
> file location. We have a blue/green pool deployment strategy where each 
> server has an active and inactive application instance and on the first 
> deployment we see that the active version logs as expected, but after 
> swapping the pools the new active application is unable to log as the now 
> inactive application seems to have a lock on the file.
> 
> We were able to make it work with 2.0.3, but 2.0.10 is the oldest version 
> that does not have the flaw. Any versions of the library without the flaw has 
> this locking issue.
> 
> Can you help us to raise an issue to be taken up for development in future 
> releases?
> 
> Thanks,
> Kevin Gardenhire
> Charles Schwab & Co.  |  Enterprise Middleware and Online Security Technology 
> (EMOST)
> 
> 
> 
> Classification: Schwab Internal
> 
> 
> Classification: Schwab Internal

Re: Log4Net 2.014

[Seal, Rupak]

HI @Tools Support is there any ways to have 
this work being escalated as this impacted production stability and damaging 
the logging server in Production SNE environment

From: Gardenhire, Kevin 
Date: Thursday, December 29, 2022 at 2:12 PM
To: dev@logging.apache.org 
Cc: Joshi, Hemant , Vishwakarma, Amit 
, Seal, Rupak , 
Shambhumane, Meenakshi 
Subject: Log4Net 2.014
Hi,

We recently upgraded our Log4Net package from 1.2.13 to 2.0.14 to remediate a 
security flaw found in the old package. After upgrading we have found an issue 
that is not allowing 2 instances of an application to write to the same file 
location. We have a blue/green pool deployment strategy where each server has 
an active and inactive application instance and on the first deployment we see 
that the active version logs as expected, but after swapping the pools the new 
active application is unable to log as the now inactive application seems to 
have a lock on the file.

We were able to make it work with 2.0.3, but 2.0.10 is the oldest version that 
does not have the flaw. Any versions of the library without the flaw has this 
locking issue.

Can you help us to raise an issue to be taken up for development in future 
releases?

Thanks,
Kevin Gardenhire
Charles Schwab & Co.  |  Enterprise Middleware and Online Security Technology 
(EMOST)



Classification: Schwab Internal


Classification: Schwab Internal

Log4Net 2.014

[Gardenhire, Kevin]

Hi,

We recently upgraded our Log4Net package from 1.2.13 to 2.0.14 to remediate a 
security flaw found in the old package. After upgrading we have found an issue 
that is not allowing 2 instances of an application to write to the same file 
location. We have a blue/green pool deployment strategy where each server has 
an active and inactive application instance and on the first deployment we see 
that the active version logs as expected, but after swapping the pools the new 
active application is unable to log as the now inactive application seems to 
have a lock on the file.

We were able to make it work with 2.0.3, but 2.0.10 is the oldest version that 
does not have the flaw. Any versions of the library without the flaw has this 
locking issue.

Can you help us to raise an issue to be taken up for development in future 
releases?

Thanks,
Kevin Gardenhire
Charles Schwab & Co.  |  Enterprise Middleware and Online Security Technology 
(EMOST)



Classification: Schwab Internal

Re: [log4j] Question about the initial configuration created in LoggerContext

[Matt Sicker]

Yeah, I think you’re right, Ralph. Thanks for reasoning this over with me. 
Returning quickly is still a good idea.

—
Matt Sicker

> On Dec 29, 2022, at 01:16, Ralph Goers  wrote:
> 
> The reason a DefaultConfiguration is created and NOT the identified 
> configuration is that a) we would have to block while the configuration is 
> created and b) creating a Configuration isn’t necessarily very fast depending 
> on what is in it (some of which we cannot control). We do want logging to be 
> available quickly and only logging errors to the console during the 
> initialization period seems like a fair compromise.  It is not at all 
> inconceivable that an application will have started multiple threads that 
> perform logging while logging is initializing and in general, we want Log4j 
> to be as lock free as possible.
> 
> So your “fairly innocuous” change is not something I would support as I 
> explicitly rejected that idea when I implemented it in the first place. I am 
> also not in favor of a configuration option that makes the start up logic 
> even more complicated and fragile.
> 
> Ralph
> 
>> On Dec 28, 2022, at 6:23 PM, Matt Sicker  wrote:
>> 
>> And with this detail about the possibility for a Logger to be returned while 
>> LoggerContext::start is being executed is indeed a reason for why 
>> DefaultConfiguration _normally_ exists. I’m looking at two additional 
>> scenarios:
>> 
>> 1. What if you provide a configuration location String or URI to the 
>> LoggerContext constructor? We already have a configuration source in this 
>> case (or at least we do if the URI corresponds to a local file), but we 
>> ignore this source until we start the LoggerContext. It is this situation 
>> where I thought it may be possible to start loading ConfigurationFactory 
>> plugins here to load and parse the specified configuration as the initial 
>> Configuration instance rather than an instance of DefaultConfiguration. This 
>> wasn’t previously possible due to various code dependencies, but given the 
>> fact that we have access to an initialized Injector instance inside the 
>> LoggerContext constructor (this may be a key detail I haven’t been too 
>> explicit about) means that we can start loading whatever subset of plugins 
>> we want at this point.
>> 
>> 2. If you don’t provide a configuration location, we could still attempt to 
>> load the ConfigurationFactory plugins to bootstrap here. This is even more 
>> realistic now that we’ve removed the plugin package scanning feature, so we 
>> know that plugin service classes will already be available to load at this 
>> time.
>> 
>> What I’m thinking here is a fairly innocuous-looking change may allow us to 
>> load the configuration earlier. As it is now, the LoggerContext is 
>> getOrCreated by the ContextSelector before Log4jContextSelector checks its 
>> state to see if it should be started (in which case it will then load 
>> ConfigurationFactory plugins to start the real configuration). What I think 
>> might be possible is moving this ConfigurationFactory bit deeper into the 
>> call chain by creating it when the LoggerContext is being created. This 
>> would mean the LoggerContext would have the custom configuration instance 
>> _before_ LoggerContext::start is invoked.
>> 
>> One of the main tradeoffs here would be startup time I suppose. By deferring 
>> configuration loading until LoggerContext::start is invoked, we do indeed 
>> get out of the way as soon as possible to start accepting log messages and 
>> returning control to user code. So I suppose even if it turns out to be 
>> possible to load the configuration right away, this would by definition 
>> cause blocking as the configuration is loaded, so this would probably make 
>> sense as a configurable option.
>> —
>> Matt Sicker
>> 
 On Dec 28, 2022, at 11:44, Ralph Goers  wrote:
>>> 
>>> 
>>> 
 On Dec 28, 2022, at 10:37 AM, Ralph Goers  
 wrote:
 
 
 
 Ralph
 
> On Dec 28, 2022, at 7:01 AM, Piotr P. Karwasz  
> wrote:
> 
> Hi Matt,
> 
> On Sun, 18 Dec 2022 at 20:30, Matt Sicker  wrote:
>> During this bootstrapping, if the configuration location is available 
>> (such as for a unit test), should LoggerContext set up the configuration 
>> provided? Or is there some sort of cyclic dependency here preventing us 
>> from loading ConfigurationFactory right away? So far, the only cyclic 
>> dependencies I’ve found are related to plugins created in the 
>> DefaultConfiguration (or the NullConfiguration in some cases), but those 
>> are already commented as such (like in PatternLayout).
> 
> I think we should rely more on our `LifeCycle` abstraction:
> `Configuration` starts in the "initializing" state and does not have
> any subcomponents (especially those that require a `LoggerContext` to
> be present) until `initialize()` is called.
 
 Actually, a LoggerContext is hard-wired with a 

Re: [log4j] The grand infra revamp

[Volkan Yazıcı]

Daniel Gruno stated in INFRA-23996 that they will implement the ticket in a
couple of days, I will wait for a week.

You can go ahead and merge your changes with necessary `changes.xml`
modifications, just like what we have done before. I will take care of
resolving `changes.xml`-related merge conflicts myself.

On Thu, Dec 29, 2022 at 8:21 AM Ralph Goers 
wrote:

> Volkan,
>
> Please do not wait to release log4j-tools.  I am already holding off
> adding more changelog entries so as to avoid creating more work when the PR
> is merged.  As far as I can tell there is no requirement that log4j-tools
> MUST be released via CI.
>
> I really would have liked to have all of this done already as this would
> have been an ideal time to release 2.19.1.
>
> Ralph
>
> > On Dec 28, 2022, at 6:34 PM, Matt Sicker  wrote:
> >
> > This sounds amazing! Thanks for championing this effort! For signing,
> there’s some project out there that lets you use OIDC for signing things
> and keeping a sort of certificate transparency log so that you can verify
> that not only was an artifact signed by a valid signature, but it’s also
> done by the expected key for a particular release. See
> https://www.sigstore.dev/ for details. I’m not sure if it works with GPG
> keys or Maven or anything, but it could be a useful basis for further
> release automation in the future (even if it’s something more custom to the
> ASF).
> > —
> > Matt Sicker
> >
> >> On Dec 28, 2022, at 14:42, Volkan Yazıcı  wrote:
> >>
> >> Hello,
> >>
> >> I want to share some updates from my side on what I have been working on
> >> for the last couple of months. In particular, I expect certain changes
> to
> >> have an ASF-wide impact! Sounds interesting? Continue reading.
> >>
> >> *What is up with `maven-changes-plugin`?*
> >>
> >> The current `site` phase takes ages to complete. This significantly
> hinders
> >> releases and more frequent website updates. The main culprit is `site`
> >> phase plugins need to be executed for 50+ modules. As my investigation
> >> turned out, we can actually either drop or replace all such plugins,
> except
> >> one: `maven-changes-plugin`, that is used to generate the changelog from
> >> `changes.xml`. `maven-changes-plugin` is also a major source of
> >> merge-conflicts in between feature branches and PRs, since they all
> need to
> >> touch to the same file: `changes.xml`.
> >>
> >> *Is it only about overhauling the `site` phase?*
> >>
> >> No. As we agreed on, we want to migrate the issue tracking from JIRA to
> >> GitHub Issues. `maven-changes-plugin` blocks us there too, since it only
> >> supports a single issue tracking system.
> >>
> >> *Enter `log4j-changelog`*
> >>
> >> I have written simple Java command line applications to perform the
> >> following tasks:
> >>
> >>  1. Populate
> >>  `src/site/changelog//_.xml` files
> >>  from `changes.xml`
> >>  2. Generate AsciiDoc-formatted changelog files from the populated
> >>  `src/site/changelog`
> >>  3. Make `src/site/changelog` ready before a release
> >>
> >> These technical tools will not only help us to replace
> >> `maven-changes-plugin`, support multiple issue trackers, and enable
> great
> >> simplification of the `site` phase, due to its
> one-changelog-file-per-issue
> >> convention, they will make changelog merge conflicts a thing of the past
> >> too!
> >>
> >> Okay, great! Since the `maven-changes-plugin` successor is there, what
> are
> >> we exactly waiting for?
> >>
> >> *Enter `log4j-tools`*
> >>
> >> I have implemented `log4j-changelog` as a not-deployed Log4j module,
> though
> >> Ralph insisted on not having this within Log4j to make it available for
> >> other Log4j projects and since it needs to be shared by (and hence,
> sync'ed
> >> in between) `release-2.x` and `master` branches. He proposed that we
> host
> >> this in `log4j-tools` .
> >> Since `log4j-tools` repository never had a release before, I let my
> >> imagination go wild there:
> >>
> >>  - Adopted the Maven-recommended way of setting up a BOM:
> >>  `logging-parent` parents `log4j-bom`, which parents `log4j-parent`,
> which
> >>  parents all other modules. There the effective POM of `log4j-bom` is
> >>  stripped-down from its parent and all unnecessary weight via
> >>  `flatten-maven-plugin` voodoo.
> >>  - Switched to CI-friendly Maven versioning
> >>  
> >>  - Configured GitHub Actions to make releases!
> >>
> >> (As you might guess, if experiment succeeds, I will port all these fancy
> >> stuff to Log4j too.)
> >>
> >> *Releasing... via GitHub Actions!?!*
> >>
> >> ASF project artifacts are required to be signed and up until now that
> has
> >> **always** been performed manually by release maintainers. Releasing
> **and
> >> signing** artifacts in a fully-automated fashion in CI... "Surely You're
> >> Joking, Mr. Feynman!" But I am not! I have shared a technical
> >> implementation proposal in