What about scanning the classpath and removing any known logging backends so that ours is the only one present?
On Wed, Feb 5, 2025 at 11:37 AM Shunping Huang <mark.sphu...@gmail.com> wrote: > I agree that, ideally, users should delegate control of the logging > backend to the runner or SDK harness to ensure logs are correctly captured. > Dataflow, as you know, has a mechanism for collecting logs written by the > JUL backend (via the default slf4j-jdk14 bridge) and sending them to Cloud > Logging. This approach may not work natively with other logging backends. > For example, when slf4j-simple is used, logs will be written to stdout or > stderr, and eventually appear in a different Cloud Logging category. > > However, we have received customer support tickets (internal bug IDs: > 307965737, 319291955, and 381398168) requesting support for alternative > backends like Log4j or Logback. These customers often have transitive Log4j > or Logback dependencies and user-customized logging configurations within > their pipelines that are difficult to get rid of. Rather than failing > their pipeline runs (e.g., a crash loop can happen when users have both > the log4j-slf4j-impl bridge and the Log4j backend, due to our prepending > log4j-to-slf4j to the classpath) or forcing the default JUL backend (due to > the previous higher priority of slf4j-jdk14 on the classpath), we propose > allowing them to use their own dependencies at their own discretion and > risk. > > Honestly, if we want to fully support other logging backends in Dataflow > and Cloud Logging, there will be some adjustments to the Dataflow > codebase. > > On Wed, Feb 5, 2025 at 10:04 AM Kenneth Knowles <k...@apache.org> wrote: > >> Thanks for the very nice document. Logging has been a source of >> complexity since the beginning of Beam, and upgrading to SLF4J2 is getting >> urgent. >> >> I do have big picture questions/concerns: >> >> - For non-portable runners/pipelines, shouldn't the runner totally >> control logging backends so that it can capture logs correctly? >> >> - For portable runners, shouldn't the SDK harness totally control >> logging backends so it can ship logs out of the container correctly? >> >> Can you explain the intended use of having the user override the backend? >> (I totally understand that since we don't control dependencies we need all >> the frontend bridges) >> >> Kenn >> >> On Tue, Feb 4, 2025 at 3:15 PM Shunping Huang <mark.sphu...@gmail.com> >> wrote: >> >>> Hi everyone, >>> >>> I put together a short doc to summarize the existing logging >>> infrastructure(dependencies) in Beam Java and outline a plan to improve it. >>> Basically, we are on the path towards slf4j 2.x. >>> >>> >>> https://docs.google.com/document/d/1IkbiM4m8D-aB3NYI1aErFZHt6M7BQ-8eCULh284Davs/edit?usp=sharing >>> >>> If you are interested in this topic, please take a look and share any >>> feedback. >>> >>> Regards, >>> >>> Shunping >>> >>
