dianaclarke commented on a change in pull request #9843:
URL: https://github.com/apache/arrow/pull/9843#discussion_r604325602
##########
File path: dev/archery/archery/tests/test_benchmarks.py
##########
@@ -94,10 +94,16 @@ def test_static_runner_from_json():
archery_result['suites'][0]['benchmarks'][0]['values'][0] *= 2
baseline = StaticBenchmarkRunner.from_json(json.dumps(archery_result))
- artificial_reg, normal = RunnerComparator(contender, baseline).comparisons
+ comparisons = list(RunnerComparator(contender, baseline).comparisons)
+
+ # can't assume return order
Review comment:
While dictionary order is preserved in Python 3, set order isn't (to the
best of my knowledge).
```
>>> foo = {'a': 1, 'b': 2, 'c': 3}
>>> bar = {'a': 1, 'b': 2, 'c': 3}
>>> type(foo.keys() & bar.keys())
<class 'set'>
```
I suspect it's the set usage in here, that makes the resulting order
non-deterministic.
```
def pairwise_compare(contender, baseline):
dict_contender = {e.name: e for e in contender}
dict_baseline = {e.name: e for e in baseline}
for name in (dict_contender.keys() & dict_baseline.keys()): #
<------ for name in set
yield name, (dict_contender[name], dict_baseline[name])
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
