hamilton-earthscope opened a new pull request, #643: URL: https://github.com/apache/iceberg-go/pull/643
## Add Comprehensive Partitioned Write Performance Benchmarks Expands benchmark suite to measure partitioned write performance across different schema complexities and partition counts. ### Benchmarks Added **Schema Complexity Tests** - All partitioned by `day(ts)` and `host` identity: 1. **Simple** - 6 primitive fields (int64, timestamp, string, int32) - baseline performance 2. **ListPrimitive** - Simple schema + `list<string>` field - tests nested list handling 3. **ListStruct** - Simple schema + `list<struct<type: string, id: list<string>>>` field - tests deeply nested structures with lists of structs containing lists 4. **MapPrimitive** - Simple schema + `map<string, string>` field - tests map type handling Each schema tested at 100K, 500K, and 2.5M record counts. **Partition Scaling Test** - Simple 4-field schema partitioned by single `partition_key` field, testing 100K records across 25, 100, 250, and 1000 partitions to measure how write performance scales with partition count. ### Purpose - Establishes baseline metrics for write performance across different Arrow/Iceberg schemas - Measure performance improvement/degradation as libraries evolve. ### Notes Pulls up arrow-go to v18.5.0-rc0 in order to support writing Map types. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
