I created a Jira to start tracking the issue: https://issues.apache.org/jira/browse/LIVY-487
When you see the issue are you using the files parameter of sessions endpoint? It would be helpful for the ticket if we could construct SSCCE ( http://sscce.org/ ) to demonstrate the problem. I don't have an environment going where I can do that easily at the moment. If you do, can you please comment on the ticket? Thanks, Tim ________________________________ From: Divya Arya <[email protected]> Sent: Friday, July 27, 2018 11:17:30 PM To: [email protected] Subject: Re: Query: Local Path cannot read by Livy (Running with Docker) [External Email] Yes, that is the exact same scenario I have, just instead of jar files I am using CSV files stored in my client machine's file system and no protocol is supporting to access those files. On Sat, Jul 28, 2018 at 3:42 AM, Harsch, Tim <[email protected]<mailto:[email protected]>> wrote: When you say not able to use my files in Spark Session what do you mean? I think I’m seeing a similar problem. If I submit jars using a client machine that is not the same as the server (so file system is not shared) and I submit the jar it’s looks like it gets transmitted to the server but then I can’t reference any of the classes. If I look at the Spark UI that gets launched for the session then I can see references to paths in my client machine, which aren’t resolvable on the server. Is this the scenario you have? I think it might occur in a docker env if you are doing client stuff on the host os, and livy is running in a container. From: Divya Arya <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Thursday, July 26, 2018 at 11:45 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Query: Local Path cannot read by Livy (Running with Docker) [External Email] Please help me out. I am really looking forward. On Thu, Jul 26, 2018 at 5:49 PM, Divya Arya <[email protected]<mailto:[email protected]>> wrote: Hello Team, I need a little support from you, I am trying to bundle up my application which is working on Apache Livy, Spark (SCALA), so it can run on a Windows-based system, for that, I am using Apache Docker. I have set up Livy and Spark in that, which is working perfectly fine but I am not able to use my Local path files (suppose files in C:/ or D:/) in SparkSession and SparkContext. Please help me out here. Thanks and Regards, Divya Arya [email protected]<mailto:[email protected]> [email protected]<mailto:[email protected]>
