Jeremy, I would be happy to tackle this with you and whoever else is willing…that being said, I have always had a love/hate relationship with Ambari…the stacks have always been intimidating to me….you mention revisiting the architecture….do you have any ideas for what might be better? In my opinion, I have always wanted to replace the guts of ambari with a system that is underpinned by Ansible…seems much more user-friendly and MUCH easier to configure and extend for those who are not comfortable with doing their own Java to Python and creating RPM’s and the like….What are your thoughts?
-- M. Aaron Bossert From: jeremy montgomery <alericmcke...@gmail.com> Reply-To: "user@ambari.apache.org" <user@ambari.apache.org> Date: Tuesday, May 12, 2020 at 9:14 AM To: "user@ambari.apache.org" <user@ambari.apache.org> Subject: Re: life after Hortonworks I make mpacks all the time, but tend to push them as extensions because the stack architecture is byzantine. * It is java that deploys python code. As such, the structure behaves like java, which makes it a pain in the rear to track down simple python functions. Sometimes the code ends up in common-services, sometimes it stays in the current version of stacks. Some stack components only use code in their latest version, some have code files strung through the folders of 8 previous versions and 3 stacks (looking at you LLAP). * The install code doesn't know where it wants to be. sometimes its in the stack, sometimes it is in ambari. This means that a version of ambari tends to be hard coded to 1-2 versions of a stack. * Changing javascript for a stack isn't possible. This means that stack components are hard coded into the ember.js with a bunch of if statements. * Making additions to a stack feature is a major process (like adding hbase thrift to an existing hbase installation) * Upgrades require a yum file regardless of installation method. so learning how to create dummy rpms is necessary. I'd be willing to chip in and maintain the stack python code but the stack architecture really needs to be revisited. On Mon, May 11, 2020 at 1:50 PM Aaron Bossert <aa...@punchcyber.com<mailto:aa...@punchcyber.com>> wrote: Matt, Yeah, my thought was to start with whatever the most recent HDP/HDF stack definition as a starting point. It just so happens that I have a backburner project to do this already. I have been using Hortonworks for a long time, but have found recently that I needed to install newer versions of Apache Druid and Apache Storm, which would require me to do a new stack…Full disclosure: I have NEVER mucked with stacks and am not a Python guy…I write in Scala/Java predominantly….that being said, I would be happy to collaborate on this if anyone feels that this would be worthwhile and useful to the broader community. -- M. Aaron Bossert From: Matt Andruff <matt.andr...@gmail.com<mailto:matt.andr...@gmail.com>> Reply-To: "user@ambari.apache.org<mailto:user@ambari.apache.org>" <user@ambari.apache.org<mailto:user@ambari.apache.org>> Date: Monday, May 11, 2020 at 1:46 PM To: "user@ambari.apache.org<mailto:user@ambari.apache.org>" <user@ambari.apache.org<mailto:user@ambari.apache.org>> Subject: Re: life after Hortonworks Cloudera no longer uses ambari. They stuck with Cloudera manager in their release of CDP.(CDH+HDP=CDP) https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.cloudera.com%2Fcdpdc%2F7.0%2Foverview%2Ftopics%2Fcdpdc-overview.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=7%2BMaRSRcCIjjKVT777pp18IYYsXEAGk3VPdNuOexV2w%3D&reserved=0> I don't think this means that Ambari is dead. I do think it means as stated that the community will need to take on packaging a stack and building rpms.(or at least packaging the stack) The legacy code for stack of HDP 2.6 stream is out there already in the repo so it's just some work to create rpms of whatever Ambari wants to release. I assume the stacks section is so poorly documented is because Hortonworks was doing the work of packaging. I'm not sure the level of effort needed to make a stack work but it seems like we could start with the last HDP build (3.1.4) and keep moving forward. On Mon., May 11, 2020, 13:08 Stephen Boesch, <java...@gmail.com<mailto:java...@gmail.com>> wrote: I am reading between the lines that ambari is no longer a strategic platform. Would someone please provide a link/reference to a Cloudera press release or blog describing this and maybe related decisions/roadmaps? thx! Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert <aa...@punchcyber.com<mailto:aa...@punchcyber.com>>: For what it is worth, I have written blueprints before, but never stacks. The documentation and tutorials for ambari stacks and blueprints are horribly out of date, incomplete, or flat out missing. Perhaps that could be an initial task for the community to undertake so that those of us who were using the Hortonworks suite of tools and were comfortable with Ambari can sever the cord, as it were....relying on commercial companies to support open source tools once their objectives have changed is rarely a good thing. Get Outlook for iOS<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=AqkexLKW6pLpkhG0auiGnGpIy5aZpcc%2B%2BG53O2qCNek%3D&reserved=0> ________________________________ From: Ganesh Raju <ganesh.r...@linaro.org<mailto:ganesh.r...@linaro.org>> Sent: Monday, May 11, 2020 12:57:50 PM To: user@ambari.apache.org<mailto:user@ambari.apache.org> <user@ambari.apache.org<mailto:user@ambari.apache.org>> Subject: Re: life after Hortonworks Here is more Apache Bigtop info releases<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=8xGs5j8Akzib7cRVlEgWq8ORcz8dWduDXqu8ev69mTE%3D&reserved=0> mailinglist<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=NgLr8gAPF2BXSr%2FXnUuaR2%2FdDU0c40RlyCoxiATR%2FWw%3D&reserved=0> list of components<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=vXMOynon6RZFabIVAo5PQ4D%2B6cB9rq%2BHSBbYBnH9uPI%3D&reserved=0> Thanks Ganesh Raju On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ganesh.r...@linaro.org<mailto:ganesh.r...@linaro.org>> wrote: Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30 other components. Ganesh On Mon, May 11, 2020 at 9:56 AM Matt Andruff <matt.andr...@gmail.com<mailto:matt.andr...@gmail.com>> wrote: I kept digging and I found that ambari supports stacks (that we have all seen before). You can write your own custom stack<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=3itBtwJy1mH3Eu2FbsVx0WurYZFlTlVY7VtV5D%2FbDjg%3D&reserved=0> meaning you could write the "glue" what apache projects hortonworks had written to roll out a set of services. I guess this is how they got around being open source. I guess I'll start looking into making my own stack from scratch. If anyone already has one or know of one, please let me know. On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ganesh.r...@linaro.org<mailto:ganesh.r...@linaro.org>> wrote: Apache Bigtop now has mpack and works with Ambari. You could try that. Thanks, Ganesh On Mon, May 11, 2020 at 9:15 AM Matt Andruff <matt.andr...@gmail.com<mailto:matt.andr...@gmail.com>> wrote: Hey I was just poking around today trying to figure out if ambari still works to install a cluster. I recently was able to install a Cloudera Manager installation. I was about to start an Ambari installation but when I poke the repos they all seem to need a user/password. Is there a non-hortonworks(cloudera) hadoop that can be still be installed with Amabari? If there is documentation somewhere just let me know, I couldn't find the answer googling as Hortonworks(cloudera) docs are all that come up. -- Live life Laugh Often -- IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>t -- Live life Laugh Often -- IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>t -- IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>t
