[
https://issues.apache.org/jira/browse/BEAM-7721?focusedWorklogId=293176&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-293176
]
ASF GitHub Bot logged work on BEAM-7721:
----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Aug/19 14:45
Start Date: 12/Aug/19 14:45
Worklog Time Spent: 10m
Work Description: lgajowy commented on pull request #9295: [BEAM-7721]
Add write test to BigQueryIOIT
URL: https://github.com/apache/beam/pull/9295#discussion_r312956786
##########
File path:
sdks/java/io/bigquery-io-tests/src/test/java/org/apache/beam/sdk/bigqueryiotests/BigQueryIOIT.java
##########
@@ -94,15 +96,20 @@
void setSourceOptions(String value);
- @Description("BQ dataset with source data")
- String getBigQuerySourceDataset();
+ @Description("BQ dataset for the test data")
+ String getBigQueryTestDataset();
- void setBigQuerySourceDataset(String dataset);
+ void setBigQueryTestDataset(String dataset);
- @Description("BQ table with source data")
- String getBigQuerySourceTable();
+ @Description("BQ table for test data")
+ String getBigQueryTestTable();
- void setBigQuerySourceTable(String table);
+ void setBigQueryTestTable(String table);
+
+ @Description("Should test use streaming writes or batch loads to BQ")
+ Boolean getUseStreamingWrites();
+
+ void setUseStreamingWrites(Boolean value);
Review comment:
Can we use String or the enum itself for that option? So that users could
explicitly tell:
`--writeMethod=STREAMING_WRITES` or `--writeMethod=BATCH_LOADS`
This is similar to TextIOIT case and compression type case:
[TextIOIT](https://github.com/apache/beam/blob/e0cbd2aa7e371c75511544ab78075d54f3f086ca/sdks/java/io/file-based-io-tests/src/test/java/org/apache/beam/sdk/io/text/TextIOIT.java#L98)
Possible alternative: I never tried this but maybe it makes sense to use
[InstanceFactory](https://github.com/apache/beam/blob/279a05604b83a54e8e5a79e13d8761f94841f326/sdks/java/core/src/main/java/org/apache/beam/sdk/options/Default.java#L145)
here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 293176)
Time Spent: 6.5h (was: 6h 20m)
> Measure throughput in BigQueryIO read in Java
> ---------------------------------------------
>
> Key: BEAM-7721
> URL: https://issues.apache.org/jira/browse/BEAM-7721
> Project: Beam
> Issue Type: Improvement
> Components: testing
> Reporter: Michal Walenia
> Assignee: Michal Walenia
> Priority: Major
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> Collect bytes/time metric in BQ read tests in Java SDK
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
