I'm hoping to find some information about using "ADD FILES <PATH>" when using
the spark execution engine. I've seen some jira tickets reference this
functionality, but little else. We have written some custom UDFs which require
some external resources. When using the MR execution engine, we can reference
the file paths using a relative path and they are properly distributed and
resolved. When I try to do the same under spark engine, I receive an error
saying the file is unavailable.
Does "ADD FILES <PATH>" work on spark, and if so, how should I properly
reference those files in order to read them in the executors?
Thanks much for your help,