I am now so embarrased... Thanks Marcelo !!!
On Mon, Jan 29, 2018 at 3:37 PM, Marcelo Vanzin <[email protected]> wrote: > "jars" is a list so you should probably try: > > "jars": [ "hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar" ] > > On Mon, Jan 29, 2018 at 3:35 PM, Kyunam Kim <[email protected]> wrote: > > I get the following when I try "jars" in POST /sessions. > > > > I tried: > > - "jars": "hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar" > > - "jars": "/user/root/1x/my-lib-SNAPSHOT.jar" > > > > livy.spark.deploy-mode = cluster (client is also tried) > > > > Basically, I just want to load my custom library into LIVY. > > > > By the way, even if this works, do I need to let Spark know about my > custom > > library by using "conf: > > {"spark.jars":"hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar"}" > ?? > > > > > > curl -k -v -H 'Content-Type: application/json' -X POST -d > > '{"kind":"spark","driverMemory":"1000M","executorCores":2,"jars": > > "hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar"}' > > "http://myserver:8998/sessions"; > > Note: Unnecessary use of -X or --request, POST is already inferred. > > * Trying 10.49.51.117... > > * TCP_NODELAY set > > * Connected to ... port 8998 (#0) > >> POST /sessions HTTP/1.1 > >> Host: ...:8998 > >> User-Agent: curl/7.55.1 > >> Accept: */* > >> Content-Type: application/json > >> Content-Length: 139 > >> > > * upload completely sent off: 139 out of 139 bytes > > < HTTP/1.1 400 Bad Request > > < Date: Mon, 29 Jan 2018 23:21:04 GMT > > < Content-Type: application/json; charset=UTF-8 > > < Content-Length: 251 > > < Server: Jetty(9.2.16.v20160414) > > < > > * Connection #0 to host myserver left intact > > "Can not deserialize instance of scala.collection.immutable.List out of > > VALUE_STRING token\n at [Source: HttpInputOverHTTP@1f8a778f; line: 1, > > column: 57] (through reference chain: > > org.apache.livy.server.interactive.CreateInteractiveRequest[\"jars\"])" > > > > Thanks, > > Q > > > > -- > Marcelo > -- Thanks, Q
