[
https://issues.apache.org/jira/browse/BEAM-10029?focusedWorklogId=578286&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-578286
]
ASF GitHub Bot logged work on BEAM-10029:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Apr/21 11:15
Start Date: 07/Apr/21 11:15
Worklog Time Spent: 10m
Work Description: mszb commented on a change in pull request #13857:
URL: https://github.com/apache/beam/pull/13857#discussion_r608560235
##########
File path: .test-infra/jenkins/job_PerformanceTests_SpannerIO_Python.groovy
##########
@@ -93,7 +93,7 @@ def spannerio_read_test_2gb = [
influx_db_name : InfluxDBCredentialsHelper.InfluxDBDatabaseName,
influx_hostname : InfluxDBCredentialsHelper.InfluxDBHostUrl,
input_options : '\'{' +
- '"num_records": 10485760,' +
+ '"num_records": 2097152,' +
Review comment:
Yes, I've fixed the data size for 2GB.
The 10GB read was failing because of insufficient data size.
Read test works in a two-step.
1: create (write pipeline) test dataset. Pass this step if the database
already exists.
2: perform a read test.
Due to system timeout, (step-1) the write pipeline was canceled before
finishing which results in insufficient data size and since we reuse the same
data for the test it will always fail, so I have deleted the database manually
in order to recreate it with the required size.
For the timeout issue, I increased it to 480 mins (8 Hours), but somehow job
still canceling in around 4 hours which causes the current failures.
```
def executeJob = { scope, testConfig ->
commonJobProperties.setTopLevelMainJobProperties(scope, 'master', 480)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 578286)
Time Spent: 17h 10m (was: 17h)
> Add Spanner IO Performance Tests for Python SDK
> -----------------------------------------------
>
> Key: BEAM-10029
> URL: https://issues.apache.org/jira/browse/BEAM-10029
> Project: Beam
> Issue Type: Test
> Components: io-py-gcp
> Reporter: Shehzaad Nakhoda
> Assignee: Shoaib Zafar
> Priority: P3
> Time Spent: 17h 10m
> Remaining Estimate: 0h
>
> Add performance tests so that the SpannerIO functionality can move into
> production (i.e. out of experimental).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
