Yes, that is the exact same scenario I have, just instead of jar files I am using CSV files stored in my client machine's file system and no protocol is supporting to access those files.
On Sat, Jul 28, 2018 at 3:42 AM, Harsch, Tim <[email protected]> wrote: > When you say not able to use my files in Spark Session what do you mean? > > > > I think I’m seeing a similar problem. If I submit jars using a client > machine that is not the same as the server (so file system is not shared) > and I submit the jar it’s looks like it gets transmitted to the server but > then I can’t reference any of the classes. If I look at the Spark UI that > gets launched for the session then I can see references to paths in my > client machine, which aren’t resolvable on the server. Is this the > scenario you have? I think it might occur in a docker env if you are > doing client stuff on the host os, and livy is running in a container. > > > > > > > > *From: *Divya Arya <[email protected]> > *Reply-To: *"[email protected]" <[email protected]. > org> > *Date: *Thursday, July 26, 2018 at 11:45 PM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Query: Local Path cannot read by Livy (Running with Docker) > > > > *[External Email] * > > > > Please help me out. I am really looking forward. > > > > On Thu, Jul 26, 2018 at 5:49 PM, Divya Arya <[email protected]> wrote: > > Hello Team, > > > > I need a little support from you, I am trying to bundle up my application > which is working on Apache Livy, Spark (SCALA), so it can run on a > Windows-based system, for that, I am using Apache Docker. > > > > I have set up Livy and Spark in that, which is working perfectly fine but > I am not able to use my Local path files (suppose files in C:/ or D:/) in > SparkSession and SparkContext. Please help me out here. > > > > > > Thanks and Regards, > > > > Divya Arya > > > > [email protected] > > [email protected] > > > > >
