nsivabalan commented on issue #4635:
URL: https://github.com/apache/hudi/issues/4635#issuecomment-1073112112
sorry. could not get time to repro this. I don't have exp w/ pyspark. I just
saved the script to rep.py and tried to execute the spark-submit. but running
into failures.
```
:: resolution report :: resolve 255ms :: artifacts dl 5ms
:: modules in use:
org.apache.hudi#hudi-spark-bundle_2.11;0.10.1 from local-m2-cache in
[default]
org.apache.spark#spark-avro_2.11;2.4.4 from local-m2-cache in [default]
org.spark-project.spark#unused;1.0.0 from local-m2-cache in [default]
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 3 | 0 | 0 | 0 || 3 | 0 |
---------------------------------------------------------------------
:: retrieving ::
org.apache.spark#spark-submit-parent-a3077db6-05fc-4539-ab07-7fcdba4e85ba
confs: [default]
0 artifacts copied, 3 already retrieved (0kB/5ms)
22/03/19 14:05:28 WARN Utils: Your hostname, Sivabalans-MacBook-Pro.local
resolves to a loopback address: 127.0.0.1; using 192.168.1.5 instead (on
interface en0)
22/03/19 14:05:28 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to
another address
22/03/19 14:05:28 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
File "/tmp/rep.py", line 26
'hoodie.write.lock.zookeeper.lock_key': f"{table_name}",
^
SyntaxError: invalid syntax
log4j:WARN No appenders could be found for logger
(org.apache.spark.util.ShutdownHookManager).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
```
Or should I launch pyspark and run through the commands you have given
above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
