[
https://issues.apache.org/jira/browse/BEAM-7121?focusedWorklogId=239390&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-239390
]
ASF GitHub Bot logged work on BEAM-7121:
----------------------------------------
Author: ASF GitHub Bot
Created on: 08/May/19 18:39
Start Date: 08/May/19 18:39
Worklog Time Spent: 10m
Work Description: yifanmai commented on issue #8377: [BEAM-7121] Add
deterministic proto coder
URL: https://github.com/apache/beam/pull/8377#issuecomment-490602509
Added microbenchmarks:
```
small_message_with_map, ProtoCoder, 1000 element(s): run 1 of 20, per
element time cost: 1.59318e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 2 of 20, per
element time cost: 1.4535e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 3 of 20, per
element time cost: 1.4518e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 4 of 20, per
element time cost: 1.38509e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 5 of 20, per
element time cost: 1.60599e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 6 of 20, per
element time cost: 1.2928e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 7 of 20, per
element time cost: 1.17121e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 8 of 20, per
element time cost: 1.18802e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 9 of 20, per
element time cost: 1.24202e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 10 of 20, per
element time cost: 1.71099e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 11 of 20, per
element time cost: 1.61979e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 12 of 20, per
element time cost: 1.1765e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 13 of 20, per
element time cost: 1.28272e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 14 of 20, per
element time cost: 1.13361e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 15 of 20, per
element time cost: 1.11871e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 16 of 20, per
element time cost: 1.20051e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 17 of 20, per
element time cost: 1.2253e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 18 of 20, per
element time cost: 1.1971e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 19 of 20, per
element time cost: 1.1992e-05 sec
small_message_with_map, ProtoCoder, 1000 element(s): run 20 of 20, per
element time cost: 1.24671e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 1 of 20, per
element time cost: 3.93291e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 2 of 20, per
element time cost: 3.4997e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 3 of 20, per
element time cost: 4.39081e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 4 of 20, per
element time cost: 3.50349e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 5 of 20, per
element time cost: 3.8208e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 6 of 20, per
element time cost: 3.45418e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 7 of 20, per
element time cost: 3.6797e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 8 of 20, per
element time cost: 3.8079e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 9 of 20, per
element time cost: 3.90592e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 10 of 20, per
element time cost: 3.8846e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 11 of 20, per
element time cost: 3.8914e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 12 of 20, per
element time cost: 3.8697e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 13 of 20, per
element time cost: 3.9917e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 14 of 20, per
element time cost: 4.14879e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 15 of 20, per
element time cost: 3.91412e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 16 of 20, per
element time cost: 4.4534e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 17 of 20, per
element time cost: 3.98979e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 18 of 20, per
element time cost: 4.05421e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 19 of 20, per
element time cost: 4.1333e-05 sec
large_message_with_map, ProtoCoder, 1000 element(s): run 20 of 20, per
element time cost: 4.23539e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 1 of
20, per element time cost: 1.46439e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 2 of
20, per element time cost: 1.8162e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 3 of
20, per element time cost: 1.2713e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 4 of
20, per element time cost: 1.26729e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 5 of
20, per element time cost: 1.2538e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 6 of
20, per element time cost: 1.27809e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 7 of
20, per element time cost: 1.2578e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 8 of
20, per element time cost: 1.29151e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 9 of
20, per element time cost: 1.384e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 10 of
20, per element time cost: 1.9423e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 11 of
20, per element time cost: 1.5723e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 12 of
20, per element time cost: 1.33312e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 13 of
20, per element time cost: 1.2516e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 14 of
20, per element time cost: 1.25978e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 15 of
20, per element time cost: 1.2943e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 16 of
20, per element time cost: 1.32768e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 17 of
20, per element time cost: 1.31061e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 18 of
20, per element time cost: 1.3788e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 19 of
20, per element time cost: 1.27299e-05 sec
small_message_with_map, DeterministicProtoCoder, 1000 element(s): run 20 of
20, per element time cost: 1.5681e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 1 of
20, per element time cost: 4.14329e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 2 of
20, per element time cost: 4.75399e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 3 of
20, per element time cost: 6.64539e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 4 of
20, per element time cost: 5.57411e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 5 of
20, per element time cost: 5.8033e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 6 of
20, per element time cost: 4.9984e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 7 of
20, per element time cost: 4.93789e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 8 of
20, per element time cost: 5.10612e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 9 of
20, per element time cost: 4.96809e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 10 of
20, per element time cost: 4.8228e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 11 of
20, per element time cost: 4.77509e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 12 of
20, per element time cost: 4.04549e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 13 of
20, per element time cost: 5.21371e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 14 of
20, per element time cost: 4.62301e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 15 of
20, per element time cost: 4.9046e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 16 of
20, per element time cost: 4.38449e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 17 of
20, per element time cost: 5.07569e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 18 of
20, per element time cost: 5.47261e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 19 of
20, per element time cost: 4.92008e-05 sec
large_message_with_map, DeterministicProtoCoder, 1000 element(s): run 20 of
20, per element time cost: 4.90022e-05 sec
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 239390)
Time Spent: 2h 20m (was: 2h 10m)
> Provide deterministic version of Python's ProtoCoder
> ----------------------------------------------------
>
> Key: BEAM-7121
> URL: https://issues.apache.org/jira/browse/BEAM-7121
> Project: Beam
> Issue Type: Improvement
> Components: runner-core
> Reporter: Yifan Mai
> Priority: Minor
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Passing deterministic=true to proto's
> [SerializeToString|https://github.com/protocolbuffers/protobuf/blob/60b66a119d17f0a2a595a231bea87cd4f4cf2689/python/google/protobuf/message.py#L189-L204]
> will result in deterministic encoding of maps in protos. This can be used to
> provide a deterministic version of ProtoCoder.
> This would allow protos to be used as a key for grouping by key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
