[GitHub] [beam] bhisevishal opened a new pull request, #26313: Prevent hashmap growing too large. Improves performance for too many …

Mon, 17 Apr 2023 17:07:12 -0700 


bhisevishal opened a new pull request, #26313:
URL: https://github.com/apache/beam/pull/26313

   …Unique Keys cases.
   
   Flush all key-values if map size reach to 12K entries + keep using LRU cache
   
   Micro BenchMark Result: 5x to 6x improvement for uniqueKeys 
PrecombineGroupingTableBenchmark, Minor impact on other
   End To End WordCount Pipeline: Approx 25% improvement
   
   Before
   
   Benchmark                                                 (distribution)  
(globallyWindowed)   Mode  Cnt   Score   Error  Units
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine         uniform     
           true  thrpt   15  52.584 ± 5.631  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine         uniform     
          false  thrpt   15  48.427 ± 4.090  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine          normal     
           true  thrpt   15  36.470 ± 3.498  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine          normal     
          false  thrpt   15  35.610 ± 0.940  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine          hotKey     
           true  thrpt   15  55.111 ± 2.996  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine          hotKey     
          false  thrpt   15  49.423 ± 2.859  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine      uniqueKeys     
           true  thrpt   15   5.319 ± 0.655  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine      uniqueKeys     
          false  thrpt   15   5.094 ± 0.337  ops/s
   
   After
   
   Benchmark                                                 (distribution)  
(globallyWindowed)   Mode  Cnt   Score   Error  Units
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine         uniform     
           true  thrpt   15  52.442 ± 1.937  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine         uniform     
          false  thrpt   15  44.824 ± 3.504  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine          normal     
           true  thrpt   15  33.719 ± 2.688  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine          normal     
          false  thrpt   15  30.081 ± 1.278  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine          hotKey     
           true  thrpt   15  51.839 ± 3.127  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine          hotKey     
          false  thrpt   15  46.264 ± 1.691  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine      uniqueKeys     
           true  thrpt   15  32.422 ± 1.269  ops/s
   PrecombineGroupingTableBenchmark.sumIntegerBinaryCombine      uniqueKeys     
          false  thrpt   15  29.210 ± 1.757  ops/s
   
   **Please** add a meaningful description for your change here
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us 
incorporate your contribution quickly and easily:
   
    - [ ] Mention the appropriate issue in your description (for example: 
`addresses #123`), if applicable. This will automatically add a link to the 
pull request in the issue. If you would like the issue to automatically close 
on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more 
tips on [how to make review process 
smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
   
   To check the build health, please visit 
[https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
   
   GitHub Actions Tests Status (on master branch)
   
------------------------------------------------------------------------------------------------
   [![Build python source distribution and 
wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python 
tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java 
tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go 
tests](https://github.com/apache/beam/workflows/Go%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more 
information about GitHub Actions CI.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to