Thanks for the pointers, Alex. Unfortunately we appear to be using Livy 0.3.0. I agree that it’s unlikely to be a Kerberos issue if the GET calls are working. Still debugging… but I’ll add an update if we get to the bottom of it…
Thanks again, Lucas. From: Alex Bozarth [mailto:[email protected]] Sent: 24 October 2017 23:26 To: [email protected] Subject: EXT: Re: Livy GET requests work but POST requests fail with "401 Authentication required" So it's been a few months since I played with a Kerberized Livy cluster but I'll see if I can help. First what version of Livy are you using? Authentication was updated in Livy 0.4 and it may change how to address this. ( https://github.com/apache/incubator-livy/pull/15 ) GET and POST calls can have different levels of auth depending on how it's set up and that may be why you're seeing the difference. In 0.4 GET calls check livy.server.access-control.view-users and POST calls check livy.server.access-control.modify-users. Also I'm a bit foggy on it, but IIRC Kerberos auth is used on all calls, so if you're getting responses on GETS its should not Kerberos causing the issue on the POSTs. Hopefully that give you a launching point at trouble shooting. Alex Bozarth Software Engineer Spark Technology Center ________________________________ E-mail: [email protected]<mailto:[email protected]> GitHub: github.com/ajbozarth<https://github.com/ajbozarth> 505 Howard Street San Francisco, CA 94105 United States [Inactive hide details for "Partridge, Lucas (GE Aviation)" ---10/23/2017 08:27:18 AM---I've written a Java client for parts of]"Partridge, Lucas (GE Aviation)" ---10/23/2017 08:27:18 AM---I've written a Java client for parts of Livy's REST API at https://urldefense.proofpoint.com/v2/url? From: "Partridge, Lucas (GE Aviation)" <[email protected]<mailto:[email protected]>> To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: 10/23/2017 08:27 AM Subject: Livy GET requests work but POST requests fail with "401 Authentication required" ________________________________ I’ve written a Java client for parts of Livy’s REST API at https://github.com/apache/incubator-livy/blob/master/docs/rest-api.md#response-body<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_incubator-2Dlivy_blob_master_docs_rest-2Dapi.md-23response-2Dbody&d=DwMFAg&c=jf_iaSHvJObTbx-siA1ZOg&r=S1_S7Dymu4ZL6g7L21O78VQZ53vEnAyZ-cx37DPYDyo&m=t80CtBH011YjL6EMsWz08NO-N15lAYv16fDRDxjPPXA&s=tFufJNt3BeUsmOQ5gNRd0Snak-_i_QtS7_gc7YHwWS4&e=> . The client uses Spring’s RestTemplate class and the Livy server is secured with Kerberos. Get /sessions and GET /batches requests work fine. But both POST /sessions and POST /batches requests fail with: org.springframework.web.client.HttpClientErrorException: 401 Authentication required Has anyone else seen something like this where the GET requests work but POST requests don’t? Does my username need adding to some Livy config setting perhaps, e.g., to livy.server.access_control.users as described at http://henning.kropponline.de/2016/11/06/connecting-livy-to-a-secured-kerberized-hdp-cluster/<https://urldefense.proofpoint.com/v2/url?u=http-3A__henning.kropponline.de_2016_11_06_connecting-2Dlivy-2Dto-2Da-2Dsecured-2Dkerberized-2Dhdp-2Dcluster_&d=DwMFAg&c=jf_iaSHvJObTbx-siA1ZOg&r=S1_S7Dymu4ZL6g7L21O78VQZ53vEnAyZ-cx37DPYDyo&m=t80CtBH011YjL6EMsWz08NO-N15lAYv16fDRDxjPPXA&s=96eelCDFXq8_SmS_kxRaLTcwLmxjIlyOqbtMF9mya6I&e=> ? Thanks, Lucas.
