[jira] [Updated] (HUDI-723) SqlTransformer's schema sometimes is not registered.
[ https://issues.apache.org/jira/browse/HUDI-723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-723: - Labels: bug-bash-0.6.0 pull-request-available (was: pull-request-available) > SqlTransformer's schema sometimes is not registered. > - > > Key: HUDI-723 > URL: https://issues.apache.org/jira/browse/HUDI-723 > Project: Apache Hudi (incubating) > Issue Type: Bug > Components: DeltaStreamer >Reporter: Alexander Filipchik >Priority: Major > Labels: bug-bash-0.6.0, pull-request-available > Fix For: 0.6.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > If schema is inferred from RowBasedSchemaProvider when SQL transformer is > used it also needs to be registered. > > Current way only works if SchemaProvider has a valid target schema. Is one > wants to use schema from SQL transformation, the result of > RowBasedSchemaProvider.getTargetSchema needs to be passed into something like: > {code:java} > private void setupWriteClient(SchemaProvider schemaProvider) { > LOG.info("Setting up Hoodie Write Client"); > registerAvroSchemas(schemaProvider); > HoodieWriteConfig hoodieCfg = getHoodieClientConfig(schemaProvider); > writeClient = new HoodieWriteClient<>(jssc, hoodieCfg, true); > onInitializingHoodieWriteClient.apply(writeClient); > } > {code} > Existent method will not work as it is checking for: > {code:java} > if ((null != schemaProvider) && (null == writeClient)) { > {code} > and writeClient is already configured. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HUDI-723) SqlTransformer's schema sometimes is not registered.
[ https://issues.apache.org/jira/browse/HUDI-723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-723: Labels: pull-request-available (was: ) > SqlTransformer's schema sometimes is not registered. > - > > Key: HUDI-723 > URL: https://issues.apache.org/jira/browse/HUDI-723 > Project: Apache Hudi (incubating) > Issue Type: Bug > Components: DeltaStreamer >Reporter: Alexander Filipchik >Priority: Major > Labels: pull-request-available > Fix For: 0.6.0 > > > If schema is inferred from RowBasedSchemaProvider when SQL transformer is > used it also needs to be registered. > > Current way only works if SchemaProvider has a valid target schema. Is one > wants to use schema from SQL transformation, the result of > RowBasedSchemaProvider.getTargetSchema needs to be passed into something like: > {code:java} > private void setupWriteClient(SchemaProvider schemaProvider) { > LOG.info("Setting up Hoodie Write Client"); > registerAvroSchemas(schemaProvider); > HoodieWriteConfig hoodieCfg = getHoodieClientConfig(schemaProvider); > writeClient = new HoodieWriteClient<>(jssc, hoodieCfg, true); > onInitializingHoodieWriteClient.apply(writeClient); > } > {code} > Existent method will not work as it is checking for: > {code:java} > if ((null != schemaProvider) && (null == writeClient)) { > {code} > and writeClient is already configured. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HUDI-723) SqlTransformer's schema sometimes is not registered.
[ https://issues.apache.org/jira/browse/HUDI-723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lamber-ken updated HUDI-723: Status: Open (was: New) > SqlTransformer's schema sometimes is not registered. > - > > Key: HUDI-723 > URL: https://issues.apache.org/jira/browse/HUDI-723 > Project: Apache Hudi (incubating) > Issue Type: Bug > Components: DeltaStreamer >Reporter: Alexander Filipchik >Assignee: lamber-ken >Priority: Major > Fix For: 0.6.0 > > > If schema is inferred from RowBasedSchemaProvider when SQL transformer is > used it also needs to be registered. > > Current way only works if SchemaProvider has a valid target schema. Is one > wants to use schema from SQL transformation, the result of > RowBasedSchemaProvider.getTargetSchema needs to be passed into something like: > {code:java} > private void setupWriteClient(SchemaProvider schemaProvider) { > LOG.info("Setting up Hoodie Write Client"); > registerAvroSchemas(schemaProvider); > HoodieWriteConfig hoodieCfg = getHoodieClientConfig(schemaProvider); > writeClient = new HoodieWriteClient<>(jssc, hoodieCfg, true); > onInitializingHoodieWriteClient.apply(writeClient); > } > {code} > Existent method will not work as it is checking for: > {code:java} > if ((null != schemaProvider) && (null == writeClient)) { > {code} > and writeClient is already configured. > -- This message was sent by Atlassian Jira (v8.3.4#803005)