[jira] [Commented] (KNOX-1148) Fix Livy Service Definition to align with Livy API (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689853#comment-16689853 ] ASF subversion and git services commented on KNOX-1148: --- Commit b5e73f184f88cc7ffbf3d874e5691689803f672c in knox's branch refs/heads/master from [~risdenk] [ https://git-wip-us.apache.org/repos/asf?p=knox.git;h=b5e73f1 ] KNOX-1148 - Fix Livy Service Definition to align with Livy API (Spark REST Service) Signed-off-by: Kevin Risden > Fix Livy Service Definition to align with Livy API (Spark REST Service) > > > Key: KNOX-1148 > URL: https://issues.apache.org/jira/browse/KNOX-1148 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Larry McCay >Assignee: Kevin Risden >Priority: Major > Fix For: 1.3.0 > > Attachments: KNOX-1148.patch, image-2018-11-16-09-59-21-519.png, > image-2018-11-16-10-05-25-088.png > > > [https://livy.incubator.apache.org/] > The initial contribution for Livy REST API includes a deviation from the > actual Livy server API in that it requires a "v1" where the actual API does > not. This may cause confusion for end users and possible incompatibilities > with existing Livy clients that don't expect that. > We need to remove the need for the "v1" for the 1.0.0 release as it implies > an extended level of backward compatibility commitment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KNOX-1148) Fix Livy Service Definition to align with Livy API (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689773#comment-16689773 ] Jesus Alvarez commented on KNOX-1148: - +1 LGTM , works with the old /v1 and with /livy. This definitely helps clear some of the confusion as to "What does v1" refer to for the livy endpoint. !image-2018-11-16-10-05-25-088.png! For some HDP clusters with multiple versions of spark, there still remains a bit of a version confusion - Currently the best approach for Multiple Spark Versions is to "cp -R data/services/livy data/services/livyspark2" , and change all service URL names from LIVYSERVER to LIVYSERVER2, such that the topology can then point to each Livy respectively. For HDP 3, maybe not as much concern as it was for HDP 2.6.x, which supported Spark 1 and spark 2.. But something to think about if HDP is going to support multiple versions of Spark( And thus multiple Livy servers, 1 per spark) +1 on this Jira to clear some of the confusion though !/jira/images/icons/emoticons/smile.png|width=16,height=16! > Fix Livy Service Definition to align with Livy API (Spark REST Service) > > > Key: KNOX-1148 > URL: https://issues.apache.org/jira/browse/KNOX-1148 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Larry McCay >Assignee: Kevin Risden >Priority: Major > Fix For: 1.3.0 > > Attachments: KNOX-1148.patch, image-2018-11-16-09-59-21-519.png, > image-2018-11-16-10-05-25-088.png > > > [https://livy.incubator.apache.org/] > The initial contribution for Livy REST API includes a deviation from the > actual Livy server API in that it requires a "v1" where the actual API does > not. This may cause confusion for end users and possible incompatibilities > with existing Livy clients that don't expect that. > We need to remove the need for the "v1" for the 1.0.0 release as it implies > an extended level of backward compatibility commitment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KNOX-1148) Fix Livy Service Definition to align with Livy API (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689778#comment-16689778 ] Kevin Risden commented on KNOX-1148: Funny you say multiple versions of Spark. That is exactly what I had in mind for LIVY-414 and https://github.com/apache/incubator-livy/pull/80. I think it needs a little refining but need someone from the Livy community to look at it closer. There is no reason to really run multiple versions of Livy for multiple versions of Spark. You should be able to set say SPARK_HOME environment variable and magic happens :) This would make it easy to switch spark version for /batch or /session. > Fix Livy Service Definition to align with Livy API (Spark REST Service) > > > Key: KNOX-1148 > URL: https://issues.apache.org/jira/browse/KNOX-1148 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Larry McCay >Assignee: Kevin Risden >Priority: Major > Fix For: 1.3.0 > > Attachments: KNOX-1148.patch, image-2018-11-16-09-59-21-519.png, > image-2018-11-16-10-05-25-088.png > > > [https://livy.incubator.apache.org/] > The initial contribution for Livy REST API includes a deviation from the > actual Livy server API in that it requires a "v1" where the actual API does > not. This may cause confusion for end users and possible incompatibilities > with existing Livy clients that don't expect that. > We need to remove the need for the "v1" for the 1.0.0 release as it implies > an extended level of backward compatibility commitment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KNOX-1148) Fix Livy Service Definition to align with Livy API (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689571#comment-16689571 ] Kevin Risden commented on KNOX-1148: [~jesus.alv] - Can you check out this patch as well? Not looking to get this into 1.2.0 but would be good to ensure what you are working on doesn't break too. :) > Fix Livy Service Definition to align with Livy API (Spark REST Service) > > > Key: KNOX-1148 > URL: https://issues.apache.org/jira/browse/KNOX-1148 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Larry McCay >Assignee: Kevin Risden >Priority: Major > Fix For: 1.3.0 > > Attachments: KNOX-1148.patch > > > [https://livy.incubator.apache.org/] > The initial contribution for Livy REST API includes a deviation from the > actual Livy server API in that it requires a "v1" where the actual API does > not. This may cause confusion for end users and possible incompatibilities > with existing Livy clients that don't expect that. > We need to remove the need for the "v1" for the 1.0.0 release as it implies > an extended level of backward compatibility commitment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KNOX-1148) Fix Livy Service Definition to align with Livy API (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689548#comment-16689548 ] Kevin Risden commented on KNOX-1148: Need to go through some testing but first stab at a patch for this. > Fix Livy Service Definition to align with Livy API (Spark REST Service) > > > Key: KNOX-1148 > URL: https://issues.apache.org/jira/browse/KNOX-1148 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Larry McCay >Assignee: Kevin Risden >Priority: Major > Fix For: 1.3.0 > > Attachments: KNOX-1148.patch > > > [https://livy.incubator.apache.org/] > The initial contribution for Livy REST API includes a deviation from the > actual Livy server API in that it requires a "v1" where the actual API does > not. This may cause confusion for end users and possible incompatibilities > with existing Livy clients that don't expect that. > We need to remove the need for the "v1" for the 1.0.0 release as it implies > an extended level of backward compatibility commitment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KNOX-1148) Fix Livy Service Definition to align with Livy API (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689477#comment-16689477 ] Kevin Risden commented on KNOX-1148: I can take a look at this since I just dug into KNOX-1098. It is probably easiest to just add the non /v1/ path as well. > Fix Livy Service Definition to align with Livy API (Spark REST Service) > > > Key: KNOX-1148 > URL: https://issues.apache.org/jira/browse/KNOX-1148 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Larry McCay >Assignee: Kevin Risden >Priority: Major > Fix For: 1.3.0 > > > http://livy.io/ > The initial contribution for Livy REST API includes a deviation from the > actual Livy server API in that it requires a "v1" where the actual API does > not. This may cause confusion for end users and possible incompatibilities > with existing Livy clients that don't expect that. > We need to remove the need for the "v1" for the 1.0.0 release as it implies > an extended level of backward compatibility commitment. > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KNOX-1148) Fix Livy Service Definition to align with Livy API (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316839#comment-16316839 ] Larry McCay commented on KNOX-1148: --- Moving this out to 1.1.0. We will need to ensure that whatever change is made here is done so with backward compatibility in mind. Either through versioning of the service definition or by allowing both paths within the same service def. Where the former is probably preferred though will break any topologies from 0.14.0 and 1.0.0. Perhaps, we make some statement of deprecation for the current service def in 1.0.0? > Fix Livy Service Definition to align with Livy API (Spark REST Service) > > > Key: KNOX-1148 > URL: https://issues.apache.org/jira/browse/KNOX-1148 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Larry McCay >Assignee: Larry McCay > Fix For: 1.1.0 > > > http://livy.io/ > The initial contribution for Livy REST API includes a deviation from the > actual Livy server API in that it requires a "v1" where the actual API does > not. This may cause confusion for end users and possible incompatibilities > with existing Livy clients that don't expect that. > We need to remove the need for the "v1" for the 1.0.0 release as it implies > an extended level of backward compatibility commitment. > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
