We generally run the full test suite on machines with at least 32GB of memory: it's pretty memory hungry because you have 3 Impalads running side-by-side. I believe we tend to run the full data load on machines with even more memory. You can start the test cluster with a single impalad before running tests (./bin/start-impala-cluster -s1 && ./tests/run-tests.py). Some tests will fail since they assume 3 Impalads but most should work ok.
Starting with the backend tests sounds like a good idea - they do exercise some of the codegen and other architecture-dependent parts that will likely be tricky. - Tim On Mon, Mar 21, 2016 at 5:09 AM, Jim Apple <[email protected]> wrote: > I think you should be able to run the backend tests without data loading: > > ./bin/run-backend-tests.sh > # or > ctest > > As in the frontend tests, you can specify which test you want to run: > > ctest --output-on-failure -R expr-test # also shows what broke, if anything > > To only build the backend test run: > > make be-test > > On Mon, Mar 21, 2016 at 4:12 AM, Nishidha Panpaliya <[email protected]> > wrote: > >> Thanks Jim and Tim for your replies. Really appreciate your co-operation >> and promptness. >> >> I've a few more queries - >> >> 1. What is the memory requirement of Impala to run all the tests? >> Currently, I see test data creation and loading is consuming almost 7GB of >> RAM. And after this, it gets stopped with bad_alloc exception. I've already >> requested to increase RAM of my VM. But just wanted to know if 16GB will >> suffice. >> >> 2. Can we skip load testing at this stage and simply run basic unit tests >> at first? Or is there any setting by means of which we can lower the volume >> of test data being generated/loaded? Once basic tests are working, we can >> focus on load testing. >> >> Also, we wish to have a call with you to discuss all this. We are located >> in India. >> >> Thanks, >> Nishidha >> >> >> [image: Inactive hide details for Sudarshan Jagadale---03/18/2016 >> 11:04:45 AM---Thanks and Regards Sudarshan Jagadale]Sudarshan >> Jagadale---03/18/2016 11:04:45 AM---Thanks and Regards Sudarshan Jagadale >> >> From: Sudarshan Jagadale/Austin/Contr/IBM >> To: Nishidha Panpaliya/Austin/Contr/IBM@IBMUS >> Cc: [email protected], Manish Patil/Austin/Contr/IBM@IBMUS >> Date: 03/18/2016 11:04 AM >> Subject: Fw: Debugging Impala code >> ------------------------------ >> >> >> >> Thanks and Regards >> Sudarshan Jagadale >> Power Open Source Solutions >> ----- Forwarded by Sudarshan Jagadale/Austin/Contr/IBM on 03/18/2016 >> 11:04 AM ----- >> >> From: Tim Armstrong <[email protected]> >> To: [email protected] >> Cc: Sudarshan Jagadale/Austin/Contr/IBM@IBMUS >> Date: 03/17/2016 10:39 PM >> Subject: Re: Debugging Impala code >> ------------------------------ >> >> >> >> Was it the impalad process that crashed? If so, there are a few places >> you can check: >> >> - Look in /tmp/impalad.ERROR, /tmp/impalad_node1.ERROR and >> /tmp/impalad_node2.ERROR for error messages. If it hit an assertion, >> you >> will get the message in there. >> - Look in the equivalent INFO logs for other error messages (for >> some crashes, there is info sent to INFO but not ERROR) >> - Look for hs_err_pid*.log files in the directory you ran Impala >> from. These are crash reports from the embedded JVM in the impalad >> process >> - Get impala to produce a core dump (make sure you have ulimit -c >> unlimited set when starting the cluster. I have it set in my .bashrc >> file) >> then debug with gdb. >> >> >> >> On Thu, Mar 17, 2016 at 8:59 AM, Jim Apple <*[email protected]* >> <[email protected]>> wrote: >> >> I believe Hive is sometimes used for data loading, though I'm not >> sure. >> >> I haven't debugged impala during data loading, but when I do need to >> debug >> the backend, I often do >> >> sudo gdb -p $(ps -C impalad -o pid | tail -1 | awk '{print $1}') >> >> >> On Thu, Mar 17, 2016 at 8:50 AM, Nishidha Panpaliya < >> *[email protected]* <[email protected]>> >> wrote: >> >> > >> > Hi All, >> > >> > I'm able to build Impala on Ubuntu ppc64le but getting crashes while >> > loading test data. >> > >> > I wanted to know how do you normally debug Impala code while >> loading test >> > data before running unit tests. Other than core dump, what are the >> other >> > ways to find out causes of crash in Impala? >> > >> > >> > Thanks, >> > Nishidha >> > >> >> >> >> >
