swaroopak commented on a change in pull request #3: PHOENIX-5232:
PhoenixDataWriter in Phoenix-Spark connector does not commit when mutation
batch size is reached
URL: https://github.com/apache/phoenix-connectors/pull/3#discussion_r274201307
##########
File path:
phoenix-spark/src/main/java/org/apache/phoenix/spark/datasource/v2/writer/PhoenixDatasourceWriter.java
##########
@@ -47,4 +65,25 @@ public void commit(WriterCommitMessage[] messages) {
@Override
public void abort(WriterCommitMessage[] messages) {
}
+
+ PhoenixDataSourceWriteOptions getOptions() {
+ return options;
+ }
+
+ private PhoenixDataSourceWriteOptions
createPhoenixDataSourceWriteOptions(DataSourceOptions options,
+
StructType schema) {
+ String scn = options.get(CURRENT_SCN_VALUE).orElse(null);
+ String tenantId =
options.get(PhoenixRuntime.TENANT_ID_ATTRIB).orElse(null);
+ String zkUrl = options.get(ZOOKEEPER_URL).get();
+ boolean skipNormalizingIdentifier =
options.getBoolean(SKIP_NORMALIZING_IDENTIFIER, false);
+ return new PhoenixDataSourceWriteOptions.Builder()
+ .setTableName(options.tableName().get())
+ .setZkUrl(zkUrl)
+ .setScn(scn)
+ .setTenantId(tenantId)
+ .setSchema(schema)
+ .setSkipNormalizingIdentifier(skipNormalizingIdentifier)
+
.setOverriddenProps(extractPhoenixHBaseConfFromOptions(options))
+ .build();
+ }
Review comment:
nit: additional line after the closing bracket
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
