rosemarYuan commented on PR #821: URL: https://github.com/apache/flink-agents/pull/821#issuecomment-4914989040
For `Object normalizedAttrs = normalizeValue(event.getAttributes(), 0);`, I think here have two question. The first one is the large-number failure. That part has been fixed: when a number larger than int64 enters condition evaluation, we now convert it to `double` instead of failing during activation building. The second one is reducing the amount of data passed into condition evaluation. I agree that we should avoid normalizing every field when the condition only needs a small subset. There are two possible directions here. **1. Shrink the map we build for condition evaluation.** At plan/open time, we can analyze each condition and collect the attribute paths it may read. At runtime, we build the input map only from the union of fields needed by the candidate conditions for that event, and normalize only those selected values. If a condition cannot be narrowed safely, we can fall back to the current full-map behavior. **2. Lazy attribute access from the expression engine itself.** Java already has the needed capability through CEL Java’s variable resolver path, so values can be normalized only when the expression actually reads them. The problem is Python: the current community implementation does not provide an equivalent lazy access path. The newer Google-backed Python implementation is promising because it wraps the official C++ runtime, but the Python binding does not expose this capability yet, and it currently requires Python 3.11 while we still support Python 3.10. So my plan is to implement the first approach now: partial map construction with conservative fallback to full normalization. Later, once the Google CEL Python implementation supports Python 3.10 and exposes the required lazy access capability, we can consider migrating to the second approach. @wenjin272 does this direction sound reasonable? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
