Hi Andras Thanks for looking into it. Look like it happens in the following scenario: We have a Java API to execute statement and it does the following: 1. Create a session by callin Livy /sessions API 2. Then create a statement by passing the session id from prev step. /<sessionId>/statements In this case, Step 2 gives 500 error code. By looking at the response for session api call, noticed a 'state' flag in response(it was state='starting' when it failed). Look like I need to check the status of the session creation API before proceeding with Step 2.
- Is there a better way to handle the above approach(other than keep checking the 'state' flag by making a call to sessions/<sessionId>)? - How long a session is active by default? Again , thanks much for your time Jobinesh On Wed, Jun 24, 2020 at 10:22 AM Andras Beni <andrasb...@cloudera.com> wrote: > Hi Jobinesh, > > I'm not familiar with Jersey, but I find it strange how the request body > is put together. Are you sure you need to call .toString? My assumption is > that this way you don post the document itself but a json string that > contains the document. > If this is not the problem, can you please give us some Livy logs > to ease debugging? > > Thanks, > Andras > > On Wed, Jun 24, 2020 at 6:43 PM Jobinesh Purushothaman <jobin...@gmail.com> > wrote: > >> Hi >> I am new to Apache Livy. We are evaluating this product for a usecase to >> read data from object storage in a Spark cluster. Here are the detail of >> API and payload >> >> - Apache Livy API: POST http://server:port >> /sessions/{sessionId}/statements >> - Request Payload: >> {"code":" val df = spark.read.option(\"header\", >> true).json(\"xxx://somebucket@somenamespace/xyz\"); \n >> df.toJSON.collect.mkString(\"[\", \",\", \"]\")"} >> >> I see the above working with postman without issues. We need to call this >> API from Java. When we tried to call the above API from Java App using a >> Jersey Client(version 2.30.1 on Java 11), Apache Livy server returns >> 'Internal Server Error' (500). The Jersey client code looks like as shown >> below >> >> JSONObject payLoad = new JSONObject(); >> String codeBlock = "val df = spark.read.option(\"header\", >> true).json(\"xxx://somebucket@somenamespace/xyz\"); \n >> df.toJSON.collect.mkString(\"[\", \",\", \"]\")"; >> payLoad.put("code", codeBlock); >> >> // In this e.g, session id: '0' is created before calling the below API >> and it works >> String apacheLivyUrl = "http://localhost:8998/sessions/0/statements";; >> Response response = >> ClientBuilder.newBuilder().build().target(apacheLivyUrl) >> .request(MediaType.APPLICATION_JSON) >> .accept(MediaType.APPLICATION_JSON) >> .post(Entity.json(payLoad.toString())); >> >> Any idea what is going wrong here? >> Thanks for your time >> Jobinesh >> >
