codope commented on pull request #3668: URL: https://github.com/apache/hudi/pull/3668#issuecomment-939993169
@xiarixiaoyao It would really help if you could share a gist showing the schema evolution steps. For example, earlier I tried [this](https://gist.github.com/codope/9ebb60d5763d998aa2f11914729263c6) and add/drop/reorder was working but rename was not working. Now, after the latest push, when I try the same but with the below spark-sql command, i don't see schema directory being created. ``` ./bin/spark-sql --jars ${JARS} \ --packages org.apache.spark:spark-avro_2.11:2.4.7,com.github.ben-manes.caffeine:caffeine:2.9.1 \ --conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer' \ --conf 'spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension' \ --conf 'spark.kryoserializer.buffer.max=1024m' --conf "spark.memory.storageFraction=0.8" \ --conf "hoodie.schema.evolution.enable=true" --conf spark.rdd.compress=true --driver-memory 2g --conf "spark.memory.fraction=0.8" ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
