[
https://issues.apache.org/jira/browse/BEAM-5995?focusedWorklogId=251413&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-251413
]
ASF GitHub Bot logged work on BEAM-5995:
----------------------------------------
Author: ASF GitHub Bot
Created on: 31/May/19 00:10
Start Date: 31/May/19 00:10
Worklog Time Spent: 10m
Work Description: pabloem commented on pull request #8664: [BEAM-5995]
Add hot key to Python Synthetic Sources and use it in Load Tests
URL: https://github.com/apache/beam/pull/8664#discussion_r289217851
##########
File path: sdks/python/apache_beam/testing/synthetic_pipeline.py
##########
@@ -238,13 +241,25 @@ def get_range_tracker(self, start_position,
stop_position):
tracker = range_trackers.UnsplittableRangeTracker(tracker)
return tracker
+ def _gen_kv_pair(self, index):
+ r = np.random.RandomState(index)
+ rand = r.random_sample()
+
+ # Determines whether to generate hot key or not.
+ if rand < self._hot_key_fraction:
+ # Generate hot key.
+ # An integer is randomly selected from the range [0, numHotKeys-1]
+ # with equal probability.
+ r_hot = np.random.RandomState(self._num_hot_keys)
Review comment:
If we use `index`, then we will have many different seeds, so we'll jave
many different 'hot keys' instead of restricting to a total of `num_hot_keys` -
I think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 251413)
Time Spent: 50h 10m (was: 50h)
> Create Jenkins jobs to run the load tests
> -----------------------------------------
>
> Key: BEAM-5995
> URL: https://issues.apache.org/jira/browse/BEAM-5995
> Project: Beam
> Issue Type: Sub-task
> Components: testing
> Reporter: Kasia Kucharczyk
> Assignee: Kasia Kucharczyk
> Priority: Major
> Time Spent: 50h 10m
> Remaining Estimate: 0h
>
> (/) Add SMOKE test
> Add GBK load tests.
> Add CoGBK load tests.
> Add Pardo load tests.
> Add SideInput tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
