"jars" is a list so you should probably try: "jars": [ "hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar" ]
On Mon, Jan 29, 2018 at 3:35 PM, Kyunam Kim <[email protected]> wrote: > I get the following when I try "jars" in POST /sessions. > > I tried: > - "jars": "hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar" > - "jars": "/user/root/1x/my-lib-SNAPSHOT.jar" > > livy.spark.deploy-mode = cluster (client is also tried) > > Basically, I just want to load my custom library into LIVY. > > By the way, even if this works, do I need to let Spark know about my custom > library by using "conf: > {"spark.jars":"hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar"}" ?? > > > curl -k -v -H 'Content-Type: application/json' -X POST -d > '{"kind":"spark","driverMemory":"1000M","executorCores":2,"jars": > "hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar"}' > "http://myserver:8998/sessions"; > Note: Unnecessary use of -X or --request, POST is already inferred. > * Trying 10.49.51.117... > * TCP_NODELAY set > * Connected to ... port 8998 (#0) >> POST /sessions HTTP/1.1 >> Host: ...:8998 >> User-Agent: curl/7.55.1 >> Accept: */* >> Content-Type: application/json >> Content-Length: 139 >> > * upload completely sent off: 139 out of 139 bytes > < HTTP/1.1 400 Bad Request > < Date: Mon, 29 Jan 2018 23:21:04 GMT > < Content-Type: application/json; charset=UTF-8 > < Content-Length: 251 > < Server: Jetty(9.2.16.v20160414) > < > * Connection #0 to host myserver left intact > "Can not deserialize instance of scala.collection.immutable.List out of > VALUE_STRING token\n at [Source: HttpInputOverHTTP@1f8a778f; line: 1, > column: 57] (through reference chain: > org.apache.livy.server.interactive.CreateInteractiveRequest[\"jars\"])" > > Thanks, > Q -- Marcelo
