[
https://issues.apache.org/jira/browse/BEAM-5995?focusedWorklogId=250328&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-250328
]
ASF GitHub Bot logged work on BEAM-5995:
----------------------------------------
Author: ASF GitHub Bot
Created on: 29/May/19 17:43
Start Date: 29/May/19 17:43
Worklog Time Spent: 10m
Work Description: pabloem commented on pull request #8664: [BEAM-5995]
Add hot key to Python Synthetic Sources and use it in Load Tests
URL: https://github.com/apache/beam/pull/8664#discussion_r288689157
##########
File path: sdks/python/apache_beam/testing/synthetic_pipeline.py
##########
@@ -238,13 +241,25 @@ def get_range_tracker(self, start_position,
stop_position):
tracker = range_trackers.UnsplittableRangeTracker(tracker)
return tracker
+ def _gen_kv_pair(self, index):
+ r = np.random.RandomState(index)
+ rand = r.random_sample()
+
+ # Determines whether to generate hot key or not.
+ if rand < self._hot_key_fraction:
+ # Generate hot key.
+ # An integer is randomly selected from the range [0, numHotKeys-1]
+ # with equal probability.
+ r_hot = np.random.RandomState(self._num_hot_keys)
Review comment:
Doesn't this generate the key from the same seed always? So it's always a
single key? Maybe using something like `index % num_hot_keys`? Or something
like that..
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 250328)
Time Spent: 49h 10m (was: 49h)
> Create Jenkins jobs to run the load tests
> -----------------------------------------
>
> Key: BEAM-5995
> URL: https://issues.apache.org/jira/browse/BEAM-5995
> Project: Beam
> Issue Type: Sub-task
> Components: testing
> Reporter: Kasia Kucharczyk
> Assignee: Kasia Kucharczyk
> Priority: Major
> Time Spent: 49h 10m
> Remaining Estimate: 0h
>
> (/) Add SMOKE test
> Add GBK load tests.
> Add CoGBK load tests.
> Add Pardo load tests.
> Add SideInput tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
