[
https://issues.apache.org/jira/browse/BEAM-5995?focusedWorklogId=251815&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-251815
]
ASF GitHub Bot logged work on BEAM-5995:
----------------------------------------
Author: ASF GitHub Bot
Created on: 31/May/19 14:40
Start Date: 31/May/19 14:40
Worklog Time Spent: 10m
Work Description: kkucharc commented on pull request #8664: [BEAM-5995]
Add hot key to Python Synthetic Sources and use it in Load Tests
URL: https://github.com/apache/beam/pull/8664#discussion_r289418456
##########
File path: sdks/python/apache_beam/testing/synthetic_pipeline.py
##########
@@ -238,13 +241,25 @@ def get_range_tracker(self, start_position,
stop_position):
tracker = range_trackers.UnsplittableRangeTracker(tracker)
return tracker
+ def _gen_kv_pair(self, index):
+ r = np.random.RandomState(index)
+ rand = r.random_sample()
+
+ # Determines whether to generate hot key or not.
+ if rand < self._hot_key_fraction:
+ # Generate hot key.
+ # An integer is randomly selected from the range [0, numHotKeys-1]
+ # with equal probability.
+ r_hot = np.random.RandomState(self._num_hot_keys)
Review comment:
That totally makes sense, thanks a lot! And it also answers my second
comment here, I guess :) I'll change it this way.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 251815)
Time Spent: 50.5h (was: 50h 20m)
> Create Jenkins jobs to run the load tests
> -----------------------------------------
>
> Key: BEAM-5995
> URL: https://issues.apache.org/jira/browse/BEAM-5995
> Project: Beam
> Issue Type: Sub-task
> Components: testing
> Reporter: Kasia Kucharczyk
> Assignee: Kasia Kucharczyk
> Priority: Major
> Time Spent: 50.5h
> Remaining Estimate: 0h
>
> (/) Add SMOKE test
> Add GBK load tests.
> Add CoGBK load tests.
> Add Pardo load tests.
> Add SideInput tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
