It would not be recommended to install kafka in its own cluster while installing the rest of metron somewhere else. In fact this cannot be done, as Metron requires Kafka and Storm to be colocated on at least one node. HDF/HDP are Hortonworks products and I cannot comment on them on this thread. I would recommend that you go to the Horton Community Connection and ask your question there. However, I will say that if you spin up our single node development environment you will see all Metron services running on the same node and are managed by a single Ambari.
28.06.2018, 23:34, "deepak kumar" <[email protected]>:
Thanks James.Isn't there any reference architecture to follow?I went through lot of metron wiki pages and they all mention HDF and HDP architecture.There is nothing specific to HCP.Even with HDF/HDP architecture , it mentioned everywhere to have HDF separately.So is it going to be separate cluster of HDF and HDP respectively?Somewhere i read ambari lacks the capability of managing multiple cluster from 1 Ambari instance.Is this true?So basically what i am looking for is hybrid architecture where kafka/nifi will reside in different cluster while hadoop and metron components should be on different cluster ideally.Any help would be appreciated.ThanksDeepakOn Fri, Jun 29, 2018 at 7:08 AM James Sirota <[email protected]> wrote:It is very highly dependent on the data you are consuming, how fast you are consuming it, what enrichments you have in place, and what your retention criteria are27.06.2018, 09:54, "deepak kumar" <[email protected]>:Thanks for reverting back.Lets keep it simple.Is there any generic recommendation , irrespective of hardware i use ?On Wed, Jun 27, 2018, 22:20 James Sirota <[email protected]> wrote:What kind of Cisco UCS hardware do you have? The recommendations would be vastly different based on what you have.27.06.2018, 09:46, "deepak kumar" <[email protected]>:Hi All,Is there any document around best practices for installing Metron in production environment?What i am looking at is given the number of events per minute or seconds , what could be the number of modes for master/hdf/hdp etc and how the cluster should be laid out?Should we consder having separate hdf cluster for nifi and kafka and separate cluster for hdp components including metron?Or should we colocate everything in 1 universal hdp cluster?Any input here will be really appreciated.Thanks in advance.-------------------Thank you,James SirotaPMC- Apache Metronjsirota AT apache DOT org-------------------Thank you,James SirotaPMC- Apache Metronjsirota AT apache DOT org
-------------------
Thank you,
James Sirota
PMC- Apache Metron
jsirota AT apache DOT org
