NightOwl888 opened a new pull request, #1207: URL: https://github.com/apache/lucenenet/pull/1207
<!-- Thank you for submitting a pull request to our repo. --> <!-- Please do NOT submit PRs for features in newer versions of Lucene. We should keep the target version consistent across our repository to make the upgrade process to newer versions of Lucene go smoothly. --> <!-- If this is your first PR in the Lucene.NET repo, please run through the checklist below to ensure a smooth review and merge process for your PR. --> - [x] You've read the [Contributor Guide](https://github.com/apache/lucenenet/blob/master/CONTRIBUTING.md) and [Code of Conduct](https://www.apache.org/foundation/policies/conduct.html). - [ ] You've included unit or integration tests for your change, where applicable. - [x] You've included inline docs for your change, where applicable. - [ ] There's an open issue for the PR that you are making. If you'd like to propose a change, please [open an issue](https://github.com/apache/lucenenet/issues/new/choose) to discuss the change or find an existing issue. <!-- Once all that is done, you're ready to go. Open the PR with the content below. --> `Lucene.Net.Store (DataInput + DataOutput):` Use stack or array pool for temporary byte buffers ## Description This optimizes the temporary buffers used when converting between UTF16 and UTF8 by putting them on the stack with fallback to the array pool for large strings. ## Benchmarks I increased the payload to 100 string fields per document (a GUID value) and this was the result. The biggest impact is the amount allocated while writing an index. ### Index Files ``` ini BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19045 Intel Core i7-8850H CPU 2.60GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores .NET Core SDK=9.0.304 [Host] : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 000-4.8.0-beta00014 : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 001-4.8.0-beta00015 : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 002-4.8.0-beta00016 : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 003-4.8.0-beta00017 : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 004-1 - before : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 005-2 - unicodeutil : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 006-3 - utf8 : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT InvocationCount=1 IterationCount=15 LaunchCount=2 UnrollFactor=1 WarmupCount=10 ``` | Method | Job | NuGetReferences | Mean | Error | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |----------- |-------------------- |---------------------------------------------- |-----------:|---------:|---------:|-----------:|----------:|----------:|----------:| | IndexFiles | 000-4.8.0-beta00014 | Lucene.Net.Analysis.Common 4.8.0-beta00014 | 1,006.4 ms | 53.97 ms | 79.11 ms | 44000.0000 | 7000.0000 | 6000.0000 | 231.96 MB | | IndexFiles | 001-4.8.0-beta00015 | Lucene.Net.Analysis.Common 4.8.0-beta00015 | 725.8 ms | 28.72 ms | 41.19 ms | 44000.0000 | 7000.0000 | 6000.0000 | 232 MB | | IndexFiles | 002-4.8.0-beta00016 | Lucene.Net.Analysis.Common 4.8.0-beta00016 | 734.8 ms | 33.88 ms | 48.59 ms | 44000.0000 | 7000.0000 | 6000.0000 | 232.05 MB | | IndexFiles | 003-4.8.0-beta00017 | Lucene.Net.Analysis.Common 4.8.0-beta00017 | 710.4 ms | 50.10 ms | 71.85 ms | 44000.0000 | 8000.0000 | 7000.0000 | 223.4 MB | | IndexFiles | 004 - before | Lucene.Net.Analysis.Common 4.8.0-ci0000003128 | 780.6 ms | 42.24 ms | 63.22 ms | 44000.0000 | 8000.0000 | 7000.0000 | 223.08 MB | | IndexFiles | 005 - after | Lucene.Net.Analysis.Common 4.8.0-ci0000003129 | 734.2 ms | 29.08 ms | 42.63 ms | 43000.0000 | 8000.0000 | 7000.0000 | 217.16 MB | ## ReadStoredFields ``` ini BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19045 Intel Core i7-8850H CPU 2.60GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores .NET Core SDK=9.0.304 [Host] : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 000-4.8.0-beta00014 : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 001-4.8.0-beta00015 : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 002-4.8.0-beta00016 : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 003-4.8.0-beta00017 : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 004-1 - before : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 005-2 - unicodeutil : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT 006-3 - utf8 : .NET Core 8.0.19 (CoreCLR 8.0.1925.36514, CoreFX 8.0.1925.36514), X64 RyuJIT IterationCount=15 LaunchCount=2 WarmupCount=10 ``` | Method | Job | NuGetReferences | Mean | Error | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |----------------- |-------------------- |---------------------------------------------------------------------------------------- |---------:|---------:|---------:|----------:|----------:|------:|----------:| | ReadStoredFields | 000-4.8.0-beta00014 | Lucene.Net.Analysis.Common 4.8.0-beta00014,Lucene.Net.QueryParser 4.8.0-beta00014 | 104.2 ms | 1.90 ms | 2.79 ms | 6200.0000 | 200.0000 | - | 29.37 MB | | ReadStoredFields | 001-4.8.0-beta00015 | Lucene.Net.Analysis.Common 4.8.0-beta00015,Lucene.Net.QueryParser 4.8.0-beta00015 | 104.7 ms | 2.26 ms | 3.38 ms | 6200.0000 | 200.0000 | - | 29.37 MB | | ReadStoredFields | 002-4.8.0-beta00016 | Lucene.Net.Analysis.Common 4.8.0-beta00016,Lucene.Net.QueryParser 4.8.0-beta00016 | 103.8 ms | 1.81 ms | 2.65 ms | 6200.0000 | 200.0000 | - | 29.37 MB | | ReadStoredFields | 003-4.8.0-beta00017 | Lucene.Net.Analysis.Common 4.8.0-beta00017,Lucene.Net.QueryParser 4.8.0-beta00017 | 104.9 ms | 1.71 ms | 2.34 ms | 6200.0000 | 200.0000 | - | 29.37 MB | | ReadStoredFields | 004 - before | Lucene.Net.Analysis.Common 4.8.0-ci0000003128,Lucene.Net.QueryParser 4.8.0-ci0000003128 | 278.9 ms | 10.73 ms | 15.39 ms | 6000.0000 | 1000.0000 | - | 29.37 MB | | ReadStoredFields | 005 - after | Lucene.Net.Analysis.Common 4.8.0-ci0000003129,Lucene.Net.QueryParser 4.8.0-ci0000003129 | 293.6 ms | 6.68 ms | 9.57 ms | 6000.0000 | 1000.0000 | - | 29.35 MB | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
