Baymine opened a new issue, #66439: URL: https://github.com/apache/doris/issues/66439
## Problem ICU's `UnicodeString::toLower()` / `toUpper()` with no explicit `Locale` argument internally calls `locale_get_default_69()`, which acquires a global mutex. Under concurrent multi-threaded LOWER/UPPER/INITCAP evaluation on UTF-8 data (e.g. a 20 TB Hudi scan with `lower(event_id) LIKE '%pattern%'` predicates), this mutex becomes a severe CPU bottleneck. Flame graph evidence from a production incident: | Signal | Measured value | |---|---:| | Total CPU samples | 95,064 | | `TransferImpl<NameToLower>::vector` inclusive | 95.11% | | ICU `UnicodeString::toLower` inclusive | 91.12% | | ICU `locale_get_default_69` inclusive | 91.08% | | `native_queued_spin_lock_slowpath` inclusive/self | 82.44% / 82.07% | | Actual substring search (LIKE) | 0.44% | 82% of CPU is wasted on a spinlock inside ICU's default-locale lookup — not on actual case conversion or string matching. ## Root Cause `TransferImpl::to_upper_utf8`, `TransferImpl::to_lower_utf8`, and `InitcapImpl::to_initcap_utf8` in `be/src/exprs/function/function_string.cpp` all call `unicode_str.toUpper()` / `unicode_str.toLower()` without an explicit `Locale` parameter. ICU internally resolves the default locale via `locale_get_default_69()`, which takes a global mutex. When dozens of scanner threads hit LOWER/UPPER/INITCAP on UTF-8 data simultaneously, this mutex becomes the dominant CPU bottleneck. ## Solution Pass `icu::Locale::getRoot()` explicitly to all three call sites. This: 1. **Eliminates the lock contention** — bypasses `locale_get_default_69()` entirely 2. **Makes case conversion deterministic across BE hosts** — results no longer depend on the process environment locale 3. **Fixes a latent correctness bug** — under a Turkish (tr_TR) default locale, ICU maps ASCII `I` → `ı` (U+0131, dotless i) instead of `i`, producing incorrect results for SQL LOWER/UPPER/INITCAP ## Behavior Change Case conversion now uses the ICU root locale instead of the process default locale. Deployments running BE under a non-root locale will see different results. This is the intended fix — the old locale-dependent behavior was both a performance bottleneck and a correctness hazard. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
