mithalee commented on issue #3336:
URL: https://github.com/apache/hudi/issues/3336#issuecomment-895405470
@nsivabalan @codope Hi, I found the issue. Spark on K8 3.1.1 has some bugs
and throws the "Premature Content" error. I tried the Spark on K8 3.1.2 which
is the latest release and the
mithalee commented on issue #3336:
URL: https://github.com/apache/hudi/issues/3336#issuecomment-893711383
@nsivabalan @codope
I came across this issue:
https://issues.apache.org/jira/browse/HADOOP-17338
This may be the root cause of the error I am running into. I am running into
mithalee commented on issue #3336:
URL: https://github.com/apache/hudi/issues/3336#issuecomment-892112742
@codope Hi, I did try the HoodieDeltaStreamer on below version of EMR:
Release label:emr-6.3.0
Hadoop distribution:Amazon 3.2.1
Applications:Tez 0.9.2, Spark 3.1.1, Hive 3.1.2,
mithalee commented on issue #3336:
URL: https://github.com/apache/hudi/issues/3336#issuecomment-888699402
> @mithalee Can you try with the latest master branch? I built the master
code and tried to reproduce the scenario in a local docker environment. It runs
fine. For example, after
mithalee commented on issue #3336:
URL: https://github.com/apache/hudi/issues/3336#issuecomment-887202873
Does the source ordering config and precombine field has to be a timestamp
field?
mithalee commented on issue #3336:
URL: https://github.com/apache/hudi/issues/3336#issuecomment-887201790
I changed the input data set a bit so that I can provide 3 different fields
for the 3 differnt mentioned configs.
./spark-submit --master
