I think leveraging the REST application would work for this use case. Services already exist for most of the functions listed in your pseudocode (HDFS read/write, Zookeeper read/write). Asynchronous functions are also supported so no issues there.
On Thu, Apr 27, 2017 at 9:09 AM, Otto Fowler <[email protected]> wrote: > Also, if you want to help that would be great ;) > > > On April 27, 2017 at 09:30:59, Otto Fowler ([email protected]) > wrote: > > So, assuming ( I know I know ) that METRON-777 eventually lands, it will > have lain the framework for the extension system for parsers out, including > the capability to create parsers outside the metron tree. > > The next step is METRON-258, side loading of parsers. This will be the > effort to actually install 3rd party parser and other extensions ( stellar > libs for example ) into metron. > > My first inclination is to add new capabilities to the REST services to > accomplish this. We would use the current services, or orchestrate them to > accomplish this. > > The ‘process’ required would be the following: > > I’ll pseudo code it out: > > [rest endpoint] > installExtension(FILE extensionTgz, string extensionType) > > workingDir = unpackExtensionAssembly(tmpDir, extensionTgz) > > deployExtensionBundleToHdfs(workingDir) > > if(parser == extenstionType) > pushConfigurationToZK(workingDir) > pushESTemplate(workingDir) *possibly depending on 777 review > setupLogRotate(workingDIr) * possibly depending on 777 review > saveExtensionTgzSomewhere(extensionTgz) > > > Then, the configuration ui would be extended to front the new api. > > * there is still a question of how we get a parser to the ambari > configuration, such that when starting parsers it starts a parser - unless > that happens and I don’t see it > > > I would like some feedback on this approach. > > * Is rest the right way? Should we do an ambari view instead? > * Is this too much to do in a rest call? Will it timeout etc? > ??? > > > Any ideas would be appreciated. >
