RussellSpitzer commented on issue #3157:
URL: https://github.com/apache/iceberg/issues/3157#issuecomment-923046069
Ok checking the interior workings, we are timing out on an iteration. The
setup phase time is included with the iteration time so if the time limit is
set to 10 minutes, and setup takes 27 minutes you will never start.
I turned on the UI and saw we start this test by writing 10 files during the
setup phase. On my excellent MBP (2020) it takes almost 3 minutes to write each
file. The test itself completes nearly immediately following the final file
being written taking almost no time at all for any of the read iterations.
```
# Warmup: 3 iterations, single-shot each
# Measurement: 5 iterations, single-shot each
# Timeout: 30 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark:
org.apache.iceberg.spark.source.parquet.vectorized.VectorizedReadDictionaryEncodedFlatParquetDataBenchmark.readDatesIcebergVectorized5k
# Run progress: 0.00% complete, ETA 00:00:00
# Fork: 1 of 1
# Warmup Iteration 1: 3.800 s/op
# Warmup Iteration 2: 3.178 s/op
# Warmup Iteration 3: 3.038 s/op
Iteration 1: 3.122 s/op
Iteration 2: ok 3.014 s/op
Iteration 3: 3.057 s/op
Iteration 4: 3.182 s/op
Iteration 5: 2.979 s/op
Result
"org.apache.iceberg.spark.source.parquet.vectorized.VectorizedReadDictionaryEncodedFlatParquetDataBenchmark.readDatesIcebergVectorized5k":
N = 5
mean = 3.071 ±(99.9%) 0.314 s/op
Histogram, s/op:
[2.900, 2.925) = 0
[2.925, 2.950) = 0
[2.950, 2.975) = 0
[2.975, 3.000) = 1
[3.000, 3.025) = 1
[3.025, 3.050) = 0
[3.050, 3.075) = 1
[3.075, 3.100) = 0
[3.100, 3.125) = 1
[3.125, 3.150) = 0
[3.150, 3.175) = 0
Percentiles, s/op:
p(0.0000) = 2.979 s/op
p(50.0000) = 3.057 s/op
p(90.0000) = 3.182 s/op
p(95.0000) = 3.182 s/op
p(99.0000) = 3.182 s/op
p(99.9000) = 3.182 s/op
p(99.9900) = 3.182 s/op
p(99.9990) = 3.182 s/op
p(99.9999) = 3.182 s/op
p(100.0000) = 3.182 s/op
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
