peay, Let me know if you get this working, I've run into the same issue.
On Wed, May 17, 2017 at 5:09 PM Lukasz Cwik <[email protected]> wrote: > Have you tried installing a logger onto the root JUL logger once the > pipeline starts executing in the worker (so inside one of your DoFn(s) > setup methods)? > > ROOT_LOGGER_NAME = "" > > LogManager.getLogManager().getLogger(ROOT_LOGGER_NAME).addHandler(myCustomHandler); > > Also, the logging integration seems to be heavily tied to Google Cloud > Logging integration so even if you get this working, its effectively > unsupported and likely to break in the future. Your best bet is once Apache > Beam uses docker containers and Google Cloud Dataflow supports user > containers. Then you'll have a lot of control over the environment. > > On Wed, May 17, 2017 at 1:35 PM, peay <[email protected]> wrote: > >> Hello, >> >> I've been trying to use a custom appender with slf4j and either log4j, >> logback or `java.util.logging` as a backend (I am not picky), to report log >> entries above a certain level to a third party service. >> >> I can set a custom appender in those backends, and that works fine when >> submitting the jobs. However, no matter what I do, at runtime in the >> Dataflow pipeline workers, all my slf4j loggers get replaced by >> `java.util.logging.Logger` loggers, with a single handler set to >> `com.google.cloud.dataflow.worker.runners.worker.logging.DataflowWorkerLoggingHandler`. >> >> I have not found a good workaround. I can manually add my >> `java.util.logging` handler for the loggers I create, but this is a bit >> hacky and I don't get all the loggers related to the pipeline worker >> itself. For instance, if an exception and `UserCodeException` are thrown, >> the error is logged by the pipeline worker and my custom appender/handler >> won't apply. >> >> This is more about some undocumented internals of the Dataflow pipeline >> worker, so this may be slightly out of scope for this mailer, but Google >> Support couldn't help and I am curious to see if anyone has some experience >> with that. >> >> Thanks! >> >> >
