Good insight Anagha. I am now leaned toward rolling my own. It makes distributed Julia easier as well.
From: anagha khanolkar <[email protected]> Sent: Wednesday, December 23, 2020 1:45 PM To: [email protected] Subject: [External] Re: Accumulo on HDInsight I have experience with HDInsight (used to be Hortonworks Hadoop PaaS, now Azure has its own distro - same look and feel). Just a quick background. HDInsight offers workload based cluster offerings - Spark, Hadoop (MR) Kafka, Hive LLAP, HBase etc. Its disaggregated compute and storage (leverages configurable cloud native storage - Azure Blob Storage or Azure Data Lake Store), and virtual machines. Only for Kafka it uses network attached disks for storage, and Hbase WAL is on attached disks. The secure flavor (Kerberized and the only multi-user flavor) leverages a PaaS service called Azure Active Directory Domain Services (Active Directory under the hood). The product team - if you reached out to them would discourage running Accumulo on HDInsight and going ahead and installing it would affect support & SLAs. I would reach out to the customer assigned (by Microsoft) Azure cloud solution architect and get an email back from the product team to share with the customer if the customer needs it. I would go with VMs and if they are using the secure version (Enterprise Security Package), I would domain join those VMs in Azure Active Directory Domain Services. Anagha Khanolkar On Wed, Dec 23, 2020 at 12:23 PM Christopher <[email protected]<mailto:[email protected]>> wrote: I have not had experience with HDInsight. My first thoughts are that if it provides Hadoop and ZooKeeper for you, then that's a few less things to worry about from a maintenance perspective for your Accumulo cluster. On the other hand, if you can't run Accumulo nodes colocated with Hadoop DataNodes, then I wonder if you're losing some performance due to lack of data locality (on top of any performance hit from being in a virtual environment). On Wed, Dec 23, 2020 at 12:19 PM Roberts, Geoffry [USA] <[email protected]<mailto:[email protected]>> wrote: All, A quick question on something I’ve never tried before: Does anyone have any experience with setting up Accumulo with HDInsight? Can it be done? Or am I better off just using a few Linux VMs, which is my first inclination and definitely my comfort zone? The employer has me on MS Azure. I am setting up an Accumulo cluster there. I notice Az offers a Hadoop thing called HDInsight. I looked into possibly using it—it has Zookeeper—for the H & Z part of my installation but as yet I don’t see how to bring Accumulo into the picture. Any thoughts are appreciated.
