A fix, calling basicconfig in pipeline and pipelineoptions:
https://github.com/apache/beam/pull/10396
On Tue, Dec 17, 2019 at 3:17 PM Robert Bradshaw wrote:
> The generally expected behavior is that if you don't do anything,
> logging goes to stderr. Logging to non-root loggers breaks this.
>
The generally expected behavior is that if you don't do anything,
logging goes to stderr. Logging to non-root loggers breaks this.
(Arguably it's a bug in the Python logging libraries to have this
inconsistency, but so be it...)
On the other hand, if you do set something up, that is respected.
I
The Python basicConfig[1] sets up a StreamHandler[2], which by default
publishes to stderr. This configuration is applied by default whenever
someone logs anything on the root logger (e.g. logging.info('abc')). With
the module-based logging changes, the basic config is never called on the
pipeline
In Beam Java, the expectation has always been that pipeline authors are
responsible for setting up logging correctly during pipeline construction
time and that the Beam SDK is responsible for setting up logging at
pipeline execution time.
Is this something we can solve by documenting and telling
It should not affect debuggability at pipeline runtime, as the sdk_worker
already does the appropriate setup of handlers, but it may affect debugging
of e.g. ptransform expansion, and pipeline construction issues.
Best
-P.
On Tue, Dec 17, 2019 at 1:59 PM Udi Meiri wrote:
> Pablo, does the issue
Pablo, does the issue affect debuggability of pipelines?
On Mon, Dec 16, 2019 at 6:23 PM Chad Dombrova wrote:
>
>
> On Mon, Dec 16, 2019 at 5:59 PM Pablo Estrada wrote:
>
>> +chad...@gmail.com is this consistent with behavior
>> that you observed?
>>
>
> I honestly can't recall, sorry. I
On Mon, Dec 16, 2019 at 5:59 PM Pablo Estrada wrote:
> +chad...@gmail.com is this consistent with behavior
> that you observed?
>
I honestly can't recall, sorry. I just remember that while I was testing I
updated sdk version and some logging stopped. I *think* I was missing the
state/message
+chad...@gmail.com is this consistent with behavior
that you observed?
+Udi Meiri sorry about the short notice on this. I know
you are working on RC1. But do you think this should be a blocker for
2.18.0? I have https://issues.apache.org/jira/browse/BEAM-8976 to track
that.
On Mon, Dec 16,
Hi all,
So there are two places where we log:
1. At pipeline construction.
2. At worker startup, to execute user code in workers.
The workers have always set up their logging handlers properly, but
pipeline construction setup is up to the user.
If the user doesn't set it up, we could have a
On Fri, Dec 13, 2019 at 1:08 PM Robert Bradshaw wrote:
> The default behavior of Python logging is
>
> (1) Logging handlers may get added (usually in main or very close to it).
> (2) Logging goes to those handlers, or a default handler if none were
> added.
>
> With this proposal, we would have
The default behavior of Python logging is
(1) Logging handlers may get added (usually in main or very close to it).
(2) Logging goes to those handlers, or a default handler if none were added.
With this proposal, we would have
(0) A default handler gets added.
(1) Logging handlers may get added
I looked at the documentation for basicConfig:
https://docs.python.org/3/library/logging.html#logging.basicConfig
Specifically, the following line:
> This function does nothing if the root logger already has handlers
configured, unless the keyword argument force is set to True.
That would mean
Thanks for looking into this.
I'm not sure unconditionally calling logging.basicConfig() on module
import is the correct solution--this prevents modules that wish to set
up handlers in place of the default handler from being able to do so.
(This is why logging.basicConfig is lazily done at the
Hello all,
It has been pointed out to me by Chad, and also by others, that my logging
changes have caused logs to start getting lost.
It seems that by never logging on the root logger, initialization for a
root handler is skipped; and that's what causes the failures.
I will work on a fix for
14 matches
Mail list logo
