JiayaoS commented on PR #22347:
URL: https://github.com/apache/kafka/pull/22347#issuecomment-4517866422
I used the following loop to re-run both affected tests 10 times and
summarize their runtimes. It removes only the test result directories between
runs, so Gradle can keep compilation tasks up-to-date while still re-running
the tests.
```bash
OUT=/tmp/fence-epoch-runs.tsv
XML=storage/build/test-results/test/TEST-org.apache.kafka.tiered.storage.integration.TransactionsTest.xml
: > "$OUT"
for run in $(seq 1 10); do
echo "===== run ${run}/10 ====="
rm -rf storage/build/test-results/test storage/build/reports/tests/test
storage/build/test-results/binary/test
./gradlew :storage:test \
--tests
org.apache.kafka.tiered.storage.integration.TransactionsTest.testFailureToFenceEpochWithTV1
\
--tests
org.apache.kafka.tiered.storage.integration.TransactionsTest.testFailureToFenceEpochWithTV2
sed -n 's/.*<testcase
name="\([^"]*testFailureToFenceEpochWithTV[12][^"]*\)".*
time="\([^"]*\)".*/\1\t\2/p' "$XML" |
awk -F "\t" -v run="$run" '{
test = $1
sub(/^.*testFailureToFenceEpochWithTV/,
"testFailureToFenceEpochWithTV", test)
sub(/ .*/, "", test)
print run "\t" test "\t" $2
}' >> "$OUT"
done
summarize() {
local label="$1"
awk -v label="$label" '
{ times[++count] = $1; sum += $1; if ($1 > max) max = $1 }
END {
if (count == 0) {
printf "%s: count=0\n", label
exit
}
if (count % 2 == 1) {
median = times[(count + 1) / 2]
} else {
median = (times[count / 2] + times[count / 2 + 1]) / 2
}
printf "%s: count=%d, max=%.3fs, median=%.3fs, avg=%.3fs\n", label,
count, max, median, sum / count
}
'
}
for test_name in testFailureToFenceEpochWithTV1
testFailureToFenceEpochWithTV2; do
awk -v test="$test_name" '$2 == test { print $3 }' "$OUT" | sort -n |
summarize "$test_name"
done
awk '{ print $3 }' "$OUT" | sort -n | summarize "ALL"
```
Results:
| Test | Count | Max | Median | Average |
| --- | ---: | ---: | ---: | ---: |
| `testFailureToFenceEpochWithTV1` | 10 | 19.959s | 18.305s | 18.570s |
| `testFailureToFenceEpochWithTV2` | 10 | 16.832s | 16.718s | 16.718s |
| All runs | 20 | 19.959s | 17.341s | 17.644s |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
