On Wed, Aug 16, 2017 at 12:27 PM, Nan Zhu <[email protected]> wrote: > I am using your words *current*. What's the definition of "current" in > livy? I think that's all application which still keep some records in the > livy's process's memory space
There are two views of what is current: Livy's and YARN's. They may not be the same. >From your reply below, you seem to want to query YARN for the state of applications that are current to Livy. There's no API for that, as you said. But that is not what I'm talking about. I'm saying that Livy should query YARN for YARN's current view of what applications exist, and then match those against its own view. Again, it's all a question about what is cheaper: a single request to YARN that results in a large reply, parts of which Livy will ignore because it's not interested in the data, or hundreds of small requests to YARN polling specific applications? > 1. How you express this "current" in a query to YARN? I think you have to > use ApplicationID (maybe there are some other ways) in a query > > 2. The problem is that I didn't see such an API to make such a "big call" > by passing in all applications's IDs -- Marcelo
