GitHub user xintongsong added a comment to the discussion: [Feature]
Per-Event-Type Configurable Log Levels for Event Log
Thanks for addressing my comments.
> Hierarchical Inheritance vs. Wildcards: Instead of explicit wildcard patterns
> like foo.bar.*, I implemented standard hierarchical logger inheritance
> (similar to log4j). event-log.org.apache.flink.agents.api.event.level: OFF
> will silence all events in that package. The root event-log.level serves as
> the global default, so no default keyword or explicit .* is needed.
Sounds good to me. This also avoids the special character `*` which might cause
unexpected behaviors in the yaml file. However, once we migrate to the
language-independent events, the current design pretty much means that users
need to use the dot-separated pattern for the event types, or they need to set
log level for each individual event. I'm not saying this is something we need
to resolve immediately. Just want to mention this for the record.
A few more questions and comments:
1. How exactly does `max-length` affects the truncation?
> The specific thresholds for each strategy are implementation details that may
> be tuned over time.
It seems we have internal thresholds for truncating long string, long list and
deep nested objects. After applying these thresholds, the resulting event is
final and will not be adjusted w.r.t. `max-length`. Then why introducing
`max-length` at all?
2. Parsability
> Truncated content is not independently parseable
I think we need the truncated contents to be valid json objects. Otherwise,
further analysis on the event logs are practically impossible. We can wrap the
truncated fields with a wrapper object, to indicate that the field has been
truncated. Something like the following.
```
{
"x": {"truncatedString": "This string is trun...", "omittedChars": 100},
"y": {"truncatedList": [1, 2, 3, 4, 5], "omittedElements": 100},
"z": {"truncatedObject": {"foo": "myFoo", "bar": "myBar"}, "omittedFields":
10}
}
```
3. EventFilter
I'm leaning towards remove this. This was designed based on the assumption that
each event is either logged or not. This binary classification did not consider
the possibility of logging truncatedly.
I would not be worried about the compatibility. We've been claiming flink
agents 0.x versions as preview releases that do not guarantee compatibility.
That means if we identify any API that should be modified, we'd better do it
now. Moreover, users should not directly interact with `EventFilter`, unless
they need to implement their own event logger, which I don't think anyone is
already doing so.
4. Validation
> On logger initialization, configured event type names are validated against
> known event classes.
How do we get all the known event classes on initialization? A user may define
a custom event that is not listened by any action, only for the logging
purpose. In that case, the framework won't be aware of the event class until
`ctx.send_event()` is called in runtime.
GitHub link:
https://github.com/apache/flink-agents/discussions/552#discussioncomment-15970941
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
