FANNG1 commented on PR #9588:
URL: https://github.com/apache/gravitino/pull/9588#issuecomment-3811960365
some problems while test the rewrite job
1. The Gravitino job jar doesn't include the Gravitino API package will
introduce class not found error when executiong Spark jobs
2. missing jar or archive handle logic, so I had to place Iceberg Spark
runtime jar to the jar under Spark home
3. missing the logs or output of spark jobs, it's hard to debug
For the escaping issues, it seems we should wrap `where`, `options`, and
`spark_config` in single quotes (`''`). There’s no need to handle double quotes
(`"`). For example, with `{"target-file-size-bytes":"536870912"}`, the Spark
CLI requires it to be formatted as `'{"target-file-size-bytes":"536870912"}'`.
Do you think we should wrap the string on the client side, or should we modify
the job template from `"--options {{options}}"` to `"--options '{{options}}'"`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
