Kavin88 commented on issue #2406:
URL: https://github.com/apache/hudi/issues/2406#issuecomment-953064052
@nsivabalan @vinothchandar Facing below Null Pointer Exception in
MultitableDeltastreamer. As per above comments, this issue had been fixed.
Still am getting the error.
Hudi - 0.8.0
spark-submit --jars
"/usr/lib/hudi/hudi-utilities-bundle_2.12-0.8.0-amzn-0.jar,/usr/lib/spark/external/lib/spark-avro_2.12-3.1.2-amzn-0.jar"
\
--master yarn --deploy-mode cluster \
--class
org.apache.hudi.utilities.deltastreamer.HoodieMultiTableDeltaStreamer
/usr/lib/hudi/hudi-utilities-bundle_2.12-0.8.0-amzn-0.jar \
--props s3://test/file.properties \
--config-folder s3://test/config/ \
--source-class org.apache.hudi.utilities.sources.ParquetDFSSource \
--target-table dummy_table \
--schemaprovider-class
org.apache.hudi.utilities.schema.FilebasedSchemaProvider \
--table-type COPY_ON_WRITE
Property file :
hoodie.deltastreamer.ingestion.tablesToBeIngested=default.table1,default.table2
hoodie.deltastreamer.ingestion.default.table1.configFile=s3://test/config/config_table1.properties
hoodie.deltastreamer.ingestion.default.table2.configFile=s3://test/config/config_table2.properties
Config file for table1 :
hoodie.deltastreamer.source.dfs.root=s3://input/
hoodie.datasource.write.partitionpath.field=
hoodie.deltastreamer.ingestion.targetBasePath=/tmp/hudi/table1/
hoodie.deltastreamer.schemaprovider.source.schema.file=s3://test/source.avsc
hoodie.deltastreamer.schemaprovider.source.schema.file=s3://test/target.avsc
hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.NonpartitionedKeyGenerator
INFO ApplicationMaster: Unregistering ApplicationMaster with FAILED (diag
message: User class threw exception: java.lang.NullPointerException
at
org.apache.hudi.utilities.deltastreamer.HoodieMultiTableDeltaStreamer.resetTarget(HoodieMultiTableDeltaStreamer.java:345)
at
org.apache.hudi.utilities.deltastreamer.HoodieMultiTableDeltaStreamer.populateTableExecutionContextList(HoodieMultiTableDeltaStreamer.java:124)
at
org.apache.hudi.utilities.deltastreamer.HoodieMultiTableDeltaStreamer.<init>(HoodieMultiTableDeltaStreamer.java:81)
at
org.apache.hudi.utilities.deltastreamer.HoodieMultiTableDeltaStreamer.main(HoodieMultiTableDeltaStreamer.java:204)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
