That is not guaranteed to work. The backend classes may sit inside any staged JARs from the users. For example, users can make a uber jar of their pipeline and its dependency including the backend classes. Then the class path will only has that uber jar, and we cannot tell what’s inside without decompressing it.
On Thu, Feb 6, 2025 at 10:17 AM Kenneth Knowles <k...@apache.org> wrote: > What about scanning the classpath and removing any known logging backends > so that ours is the only one present? > > On Wed, Feb 5, 2025 at 11:37 AM Shunping Huang <mark.sphu...@gmail.com> > wrote: > >> I agree that, ideally, users should delegate control of the logging >> backend to the runner or SDK harness to ensure logs are correctly captured. >> Dataflow, as you know, has a mechanism for collecting logs written by the >> JUL backend (via the default slf4j-jdk14 bridge) and sending them to Cloud >> Logging. This approach may not work natively with other logging backends. >> For example, when slf4j-simple is used, logs will be written to stdout or >> stderr, and eventually appear in a different Cloud Logging category. >> >> However, we have received customer support tickets (internal bug IDs: >> 307965737, 319291955, and 381398168) requesting support for alternative >> backends like Log4j or Logback. These customers often have transitive Log4j >> or Logback dependencies and user-customized logging configurations >> within their pipelines that are difficult to get rid of. Rather than >> failing their pipeline runs (e.g., a crash loop can happen when users >> have both the log4j-slf4j-impl bridge and the Log4j backend, due to our >> prepending log4j-to-slf4j to the classpath) or forcing the default JUL >> backend (due to the previous higher priority of slf4j-jdk14 on the >> classpath), we propose allowing them to use their own dependencies at >> their own discretion and risk. >> >> Honestly, if we want to fully support other logging backends in Dataflow >> and Cloud Logging, there will be some adjustments to the Dataflow >> codebase. >> >> On Wed, Feb 5, 2025 at 10:04 AM Kenneth Knowles <k...@apache.org> wrote: >> >>> Thanks for the very nice document. Logging has been a source of >>> complexity since the beginning of Beam, and upgrading to SLF4J2 is getting >>> urgent. >>> >>> I do have big picture questions/concerns: >>> >>> - For non-portable runners/pipelines, shouldn't the runner totally >>> control logging backends so that it can capture logs correctly? >>> >>> - For portable runners, shouldn't the SDK harness totally control >>> logging backends so it can ship logs out of the container correctly? >>> >>> Can you explain the intended use of having the user override the >>> backend? (I totally understand that since we don't control dependencies we >>> need all the frontend bridges) >>> >>> Kenn >>> >>> On Tue, Feb 4, 2025 at 3:15 PM Shunping Huang <mark.sphu...@gmail.com> >>> wrote: >>> >>>> Hi everyone, >>>> >>>> I put together a short doc to summarize the existing logging >>>> infrastructure(dependencies) in Beam Java and outline a plan to improve it. >>>> Basically, we are on the path towards slf4j 2.x. >>>> >>>> >>>> https://docs.google.com/document/d/1IkbiM4m8D-aB3NYI1aErFZHt6M7BQ-8eCULh284Davs/edit?usp=sharing >>>> >>>> If you are interested in this topic, please take a look and share any >>>> feedback. >>>> >>>> Regards, >>>> >>>> Shunping >>>> >>>
