Hi Arvind, first, some housekeeping: I'm dropping user@ from the thread (at least temporarily), as most of the problems are centered around building Impala, which is mostly a developer activity.
As for your minicluster startup problem: HBase clearly seems unhappy about something. While I'm not an HBase expert (far from it!), there are a couple of ideas that come to mind. Please bear with me in case these seem trivial; I am trying to establish a common baseline so that we can rule out simple problems. 1. HBase starts pretty late in the startup sequence, so the failure might indicate a lack of (some) resources. I don't seem to recall any capacity data for your environment. As a guideline, our current automated precommit environment runs the build, the minicluster and all the tests on instances having 64 GB of memory, 16 logical cores and ~250 GB of disk space (this is a single volume). Some of this can be scaled back: you can run with fewer CPU cores of course; you can scale back memory to 32 GB if you reduce concurrency by limiting active cores to 4-8 (we ran into problems when running end-to-end tests on VMs with 16 logical cores and just 32 GB of memory). Storage requirements are the least flexible: you should really have at least 200 GBs of free space. 2. HBase requires passwordless ssh access to localhost, although this should have been set up by the machine preparation script bin/bootstrap_system.sh (this is also called by bin/bootstrap_build.sh and bin/bootstrap_development.sh). I assume you ran this script -- of no, please do it now. Also, could you please check SSH access verifying that $ ssh localhost does _not_ prompt you for a password? 3. Before launching the minicluster for the first time it is best for initialize the minicluster's (simulated) storage environment. You can request this by passing the "-format" arguments to buildall.sh, so in your current state $ ./buildall.sh -noclean -notests -format would be the shortest way to format data and metadata storage for the minicluster. 4. If all else fails, $ git clean -fdx returns you to a clean slate, although at the expense of wiping out everything in your build directory that did not come from Git. If you decide to do this, please stop the minicluster processes before you blow away everything, it is easier. To go back to square one and restart the build from a clean slate: - go to the Impala directory and stop all the processes making up the minicluster: $ testdata/bin/kill-all.sh - erase everything from the Impala working directory except for the project files from Git: $ git clean -fdx - close the terminal window and open a fresh one. This will ensure that there are no lingering environment variables that could interfere with your future clean state. - in the new terminal go to the Impala directory, and source impala-config.sh: $ source bin/impala-config.sh - Rebuild Impala, format the minicluster storage and launch the minicluster: $ ./buildall.sh -skiptests -format -start_minicluster This is expected to work -- but if it doesn't, it gives a firm baseline to work from. Good luck; let us know your results! - LaszloG PS: Answers for your other quesions will follow shortly. On Fri, Feb 3, 2023 at 2:07 PM Arvind Dige <arvind.d...@ellicium.com> wrote: > Hi Joe, > I have recreated the issue and sharing the logs for the same with you. > Please use below links for build_logs and hmaster logs from the path as you > mentioned in the above email. > > Build_logs => > > https://drive.google.com/file/d/126SfDWuMIkoY3QClhQzRhd3d1LfgeFwb/view?usp=sharing > hmaster_logs => > > https://drive.google.com/file/d/1nYsGqyZ2gpfawv7umAddmBuuOVGXvA7q/view?usp=sharing > > Also I have tried with different user on different machines and facing the > same issue there also. > With request, If possible please answer my questions from previous mails. > > Thanks & Regards, > Arvind Dige > > On Fri, Feb 3, 2023 at 11:26 AM Arvind Dige <arvind.d...@ellicium.com> > wrote: > > > HI Joe, > > > > Due to some work I lost progress on the same work so I am unable to > > provide the logs for the same. I am trying to recreate the same issue. > > Also as you mentioned about root user I will try to build the impala > using > > another user. > > But I have some questions like, > > > > 1) Is it needed to launch a mini cluster before starting impala? > > 2) Can we start impalad, catalogd or statestored roles individually to > > make impala works? > > 3) As I worked on CDH & CDP, is it possible to add one or more impala > > roles(components) on different nodes (How the role allocation works in > > impala)? > > 4) Also if i successfully build the impala, for node addition is it > > required to build impala separately on that node or how can we proceed > for > > the same? > > > > I will update you with the logs once I regenerate the issue. > > > > Thanks & Regards, > > Arvind Dige > > > > On Fri, Feb 3, 2023 at 12:52 AM Joe McDonnell <joemcdonn...@cloudera.com > > > > wrote: > > > >> Hi Arvind, > >> > >> Compilation succeeds, and now it is starting up the Hadoop components > >> that we use for testing. In this case, it is starting up HBase and > failing. > >> Additional logs will be in the logs/cluster/hbase/ directory. In this > >> case, /root/Impala/logs/cluster/hbase/hbase-master-master-hadoop.out > should > >> contain more info. > >> > >> We do regularly use Ubuntu 20 for development, so we expect Impala to > >> work. We do not test running as root user, so I don't know if that would > >> matter or not. > >> > >> Thanks, > >> Joe > >> > >> On Thu, Feb 2, 2023 at 9:17 AM Arvind Dige <arvind.d...@ellicium.com> > >> wrote: > >> > >>> Hi Joe, > >>> > >>> Thanks for your reply. > >>> Please find the below gdrive link which is publically accessible > >>> > >>> > https://drive.google.com/file/d/1rvJRWXephJJlV1Ij_QwkorFlUQx2fhdB/view?usp=sharing > >>> > >>> Thanks & Regards, > >>> Arvind Dige > >>> > >>> On Thu, Feb 2, 2023 at 10:38 PM Joe McDonnell < > joemcdonn...@cloudera.com> > >>> wrote: > >>> > >>>> Hi Arvind, > >>>> > >>>> Unfortunately, the email servers for these mailing lists strip > >>>> attachments. Can you upload the log to some public location and > provide a > >>>> link? > >>>> > >>>> Thanks, > >>>> Joe > >>>> > >>>> On Wed, Feb 1, 2023 at 10:07 PM Arvind Dige <arvind.d...@ellicium.com > > > >>>> wrote: > >>>> > >>>>> Hi Team, I am facing an issue while building impala, can you help me > >>>>> to troubleshoot it? > >>>>> I am trying to build impala on ubuntu-20.04 using > >>>>> https://cwiki.apache.org/confluence/display/IMPALA/Building+Impala > >>>>> . > >>>>> <https://stackoverflow.com/posts/75282210/timeline> > >>>>> > >>>>> I am able to see HDFS, HIVE, HBase, Ranger components are downloaded > >>>>> but the build was failing with "Could not get one or more nodes" > >>>>> > >>>>> I am attaching the log file for reference, please check and respond. > >>>>> > >>>>> Thanks & Regards, > >>>>> > >>>>> Arvind Dige > >>>>> > >>>>> Privileged/Confidential information may be contained in this message > >>>>> and may be subject to legal privilege. Access to this e-mail by > anyone > >>>>> other than the intended is unauthorized. If you are not the intended > >>>>> recipient (or responsible for delivery of the message to such > person), you > >>>>> may not use, copy, distribute or deliver to anyone this message (or > any > >>>>> part of its contents ) or take any action in reliance on it. In such > case, > >>>>> you should destroy this message, and notify us immediately. If you > have > >>>>> received this email in error, please notify us immediately by e-mail > or > >>>>> telephone and delete the e-mail from any computer. If you or your > employer > >>>>> does not consent to internet e-mail messages of this kind, please > notify us > >>>>> immediately by e-mail. All reasonable precautions have been taken to > ensure > >>>>> no viruses are present in this e-mail. As our company cannot accept > >>>>> responsibility for any loss or damage arising from the use of this > e-mail > >>>>> or attachments we recommend that you subject these to your virus > checking > >>>>> procedures prior to use. The views, opinions, conclusions and other > >>>>> information expressed in this electronic mail are not given or > endorsed by > >>>>> the company unless otherwise indicated by an authorized > representative > >>>>> independent of this message. > >>>> > >>>> > >>> Privileged/Confidential information may be contained in this message > and > >>> may be subject to legal privilege. Access to this e-mail by anyone > other > >>> than the intended is unauthorized. If you are not the intended > >>> recipient (or responsible for delivery of the message to such person), > you > >>> may not use, copy, distribute or deliver to anyone this message (or any > >>> part of its contents ) or take any action in reliance on it. In such > case, > >>> you should destroy this message, and notify us immediately. If you have > >>> received this email in error, please notify us immediately by e-mail or > >>> telephone and delete the e-mail from any computer. If you or your > employer > >>> does not consent to internet e-mail messages of this kind, please > notify us > >>> immediately by e-mail. All reasonable precautions have been taken to > ensure > >>> no viruses are present in this e-mail. As our company cannot accept > >>> responsibility for any loss or damage arising from the use of this > e-mail > >>> or attachments we recommend that you subject these to your virus > checking > >>> procedures prior to use. The views, opinions, conclusions and other > >>> information expressed in this electronic mail are not given or > endorsed by > >>> the company unless otherwise indicated by an authorized representative > >>> independent of this message. > >> > >> > > -- > Privileged/Confidential information may be contained in this message and > may be subject to legal privilege. Access to this e-mail by anyone other > than the intended is unauthorized. If you are not the intended recipient > (or responsible for delivery of the message to such person), you may not > use, copy, distribute or deliver to anyone this message (or any part of > its > contents ) or take any action in reliance on it. In such case, you should > destroy this message, and notify us immediately. If you have received this > email in error, please notify us immediately by e-mail or telephone and > delete the e-mail from any computer. If you or your employer does not > consent to internet e-mail messages of this kind, please notify us > immediately by e-mail. All reasonable precautions have been taken to > ensure > no viruses are present in this e-mail. As our company cannot accept > responsibility for any loss or damage arising from the use of this e-mail > or attachments we recommend that you subject these to your virus checking > procedures prior to use. The views, opinions, conclusions and other > information expressed in this electronic mail are not given or endorsed by > the company unless otherwise indicated by an authorized representative > independent of this message. >
