[GitHub] [incubator-doris] zbtzbtzbt opened a new issue, #9194: [Enhancement] HLL registers use chunk allocator for better memory usage

[GitBox]

zbtzbtzbt opened a new issue, #9194:
URL: https://github.com/apache/incubator-doris/issues/9194

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Description
   
   pr https://github.com/apache/incubator-doris/pull/9188
   
   <img width="795" alt="test1" 
src="https://user-images.githubusercontent.com/35688959/164967142-cb4bb3ec-c716-43d7-bad8-7694afd83207.png";>
   
   ```
   select 
       dt, HLL_UNION_AGG(union_id)
   from
       xxx
   where
       dt between a and b
   group by dt;
   ```
   
   ### speed
   hll_chunkAlloc: **547.794** seconds
   apache:master: **712.229** seconds
   
   ### mem.memused
   hll_chunkAlloc: used **38G** memory avg backend node
   apache:master: used **46G** memory avg backend node
   
   ### mem.memused.percent
   hll_chunkAlloc:  **15%**
   apache:master:  **18%**
   
   ------
   hll_chunkAlloc:
   ChunkAlloctor::allocate is still a heavy function, but it's better than use 
`new` as we know, and it does have some boost
   (`new` ->  `SystemAlloctor` -> `system call` wil not record in perf report )
   
   <img width="1193" alt="chunck_concu_2" 
src="https://user-images.githubusercontent.com/35688959/164969030-8bd725f4-53a4-46cb-a4aa-e1711bc2f87f.png";>
   
   apache:master
   <img width="1525" alt="hll2" 
src="https://user-images.githubusercontent.com/35688959/164969428-3c16fad0-eb25-442b-85bd-b9530b834a9b.png";>
   
   
   
   ### Solution
   
   https://github.com/apache/incubator-doris/pull/9188
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org