[
https://issues.apache.org/jira/browse/LIVY-616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16904900#comment-16904900
]
Oleksandr Shevchenko edited comment on LIVY-616 at 8/12/19 7:06 AM:
--------------------------------------------------------------------
[~jerryshao]
1. I considered two scenarios:
- getting Livy Server address directly from LivyDiscoveryManager:
{code:java}
val discoveryManager = LivyDiscoveryManager(livyConf)
val address = discoveryManager.getServerUri()
{code}
- and also getting Livy Server address from Java/Scala client. might be useful
when we get Livy client from somewhere, so I added a method for getting the
address. Not sure if it really needed. also notice that I added a new method to
LivyClient interface "Future<URI> getServerUri()", we use Future hare actually
only in RPC client, I left Future for HTTP Client to honor uniform access
principle:
{code:java}
val address = LivyDiscoveryManager(livyConf).getServerUri() // here we can use
it in REST requests
val client = LivyClientBuilder.setURI(s"http://$address";).build // or create
client
val futureUri = client.getServerUri // and get address from client if needed
val scalaClient = new LivyScalaClient(client)
val futureUri = scalaClient.getServerUri
{code}
I had a question about it before.
2. Livy Server discovery has a separate logic from the State Store, now these
are two independent features. Both features now use ZooKeeperManager to
communicate with ZK via Curator client (if we use ZK state store). The only
common thing is "livy.zookeeper.url" property which is used for both features.
FileSystem state store works as before. I don't see how Livy Discovery feature
relates to the FileSystem state store.
Correct me if missed something.
was (Author: oshevchenko):
[~jerryshao]
1. I considered two scenarios:
- getting Livy Server address directly from LivyDiscoveryManager:
{code}
val discoveryManager = LivyDiscoveryManager(livyConf)
val address = discoveryManager.getServerUri()
{code}
- and also getting Livy Server address from Java/Scala client. might be useful
when we get Livy client from somewhere, so I added a method for getting the
address. Not sure if it really needed. also notice that I added a new method to
LivyClient interface "Future<URI> getServerUri()", we use Future hare actually
only in RPC client, I left Future for HTTP Client to honor uniform access
principle:
{code}
val scalaClient = new LivyScalaClient(client)
val futureUri = scalaClient.getServerUri
{code}
I had a question about it before.
2. Livy Server discovery has a separate logic from the State Store, now these
are two independent features. Both features now use ZooKeeperManager to
communicate with ZK via Curator client (if we use ZK state store). The only
common thing is "livy.zookeeper.url" property which is used for both features.
FileSystem state store works as before. I don't see how Livy Discovery feature
relates to the FileSystem state store.
Correct me if missed something.
> Livy Server discovery
> ---------------------
>
> Key: LIVY-616
> URL: https://issues.apache.org/jira/browse/LIVY-616
> Project: Livy
> Issue Type: Improvement
> Components: Server
> Reporter: Oleksandr Shevchenko
> Priority: Major
> Attachments: Livy Server discovery spec.pdf
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, there isn't a way to get Livy Server URI by the client without
> setting Livy address explicitly to livy.conf. A client should set
> "livy.server.host" variable and then get it via LivyConf. The same behavior
> if you want to use Livy with Zeppelin, we need to set "zeppelin.livy.url". It
> very inconvenient when we install Livy packages on couple nodes and don't
> know where exactly Livy Server will be started e.g. by Ambari or Cloudera
> Manager. Also, in this case, we need to have Livy configuration files on a
> node where we want to get Livy address.
> It will be very helpful if we will add Livy Server address to Zookeeper and
> expose API for clients to get Livy URL to use it in client code for REST
> calls.
> Livy already supports state saving in Zookeeper but I don't see that we store
> Livy server address somewhere. Before starting investigating and
> implementation I want to ask here about this.
> Please, correct me if I missed something.
> Any comments will be highly appreciated!
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
