zyperd commented on issue #10561:
URL: https://github.com/apache/hudi/issues/10561#issuecomment-1909649918
I have gone through the class in emr
```find . -name 'MySqlDebeziumAvroPayload.class' | xargs javap -p```
the output is
```
Compiled from "MysqlDebeziumSource.java"
public class org.apache.hudi.utilities.sources.debezium.MysqlDebeziumSource
extends org.apache.hudi.utilities.sources.debezium.DebeziumSource {
private final org.apache.spark.sql.SQLContext sqlContext;
private final java.lang.String generateUniqueSeqUdfFn;
public
org.apache.hudi.utilities.sources.debezium.MysqlDebeziumSource(org.apache.hudi.common.config.TypedProperties,
org.apache.spark.api.java.JavaSparkContext, org.apache.spark.sql.SparkSession,
org.apache.hudi.utilities.schema.SchemaProvider,
org.apache.hudi.utilities.ingestion.HoodieIngestionMetrics);
protected org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>
processDataset(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>);
private static java.lang.String generateUniqueSequence(java.lang.String,
java.lang.Long);
private static java.lang.Object
$deserializeLambda$(java.lang.invoke.SerializedLambda);
}
```
In utilhelper.java
```
public static Source createSource(String sourceClass, TypedProperties cfg,
JavaSparkContext jssc,
SparkSession sparkSession, SchemaProvider schemaProvider,
HoodieIngestionMetrics metrics) throws IOException {
try {
try {
return (Source) ReflectionUtils.loadClass(sourceClass,
new Class<?>[] {TypedProperties.class, JavaSparkContext.class,
SparkSession.class, SchemaProvider.class,
HoodieIngestionMetrics.class},
cfg, jssc, sparkSession, schemaProvider, metrics);
} catch (HoodieException e) {
return (Source) ReflectionUtils.loadClass(sourceClass,
new Class<?>[] {TypedProperties.class, JavaSparkContext.class,
SparkSession.class, SchemaProvider.class},
cfg, jssc, sparkSession, schemaProvider);
}
} catch (Throwable e) {
throw new IOException("Could not load source class " + sourceClass, e);
}
}
```
there is a option to call
```
(TypedProperties.class, JavaSparkContext.class,
SparkSession.class, SchemaProvider.class},cfg, jssc,
sparkSession, schemaProvider)
```
but the declaration in the catch is not available in MysqlDebeziumSource.java
**please correct me if i’m wrong**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
