Thanks for your inputs, Jim!. The steps you have outlined in previous email, match with my findings too. I continued working further on the issue. Finally, I was able to resolve it by rebuilding hadoop with snappy-java support inclusion (Dsnappy.prefix) in the mvn command.
From: Jim Apple <[email protected]> To: Valencia Serrao/Austin/Contr/IBM@IBMUS Cc: "dev@impala" <[email protected]>, Manish Patil/Austin/Contr/IBM@IBMUS, Nishidha Panpaliya/Austin/Contr/IBM@IBMUS, Sudarshan Jagadale/Austin/Contr/IBM@IBMUS Date: 01/27/2017 03:28 AM Subject: Re: Issues while creating hbase data tables It looks to me like functional_schema_template.sql is called from testdata/bin/generate-schema-statements.py, which is called from bin/load-data.py, which is called from testdata/bin/create-load-data.sh, which is called from buildall.sh $ git grep schema_tem ... testdata/bin/generate-schema-statements.py:def parse_schema_template_file (file_name): testdata/bin/generate-schema-statements.py: schema_template_file = os.path.join(DATASET_DIR, target_dataset, testdata/bin/generate-schema-statements.py: '%s_schema_template.sql' % target_dataset) ... $ git grep generate-schema-statements bin/load-data.py:GENERATE_SCHEMA_CMD = "generate-schema-statements.py --exploration_strategy=%s "\ $ git grep load-data ... buildall.sh: "${IMPALA_HOME}/testdata/bin/create-load-data.sh" $ {CREATE_LOAD_DATA_ARGS} <<< Y ... testdata/bin/create-load-data.sh: if ! impala-python -u $ {IMPALA_HOME}/bin/load-data.py ${ARGS[@]} &> ${LOG_FILE}; then testdata/bin/create-load-data.sh: if ! impala-python -u $ {IMPALA_HOME}/bin/load-data.py --workloads all\ On Wed, Jan 4, 2017 at 10:32 PM, Valencia Serrao <[email protected]> wrote: Hi Jim, Thanks for the inputs. But our requirement is to get Impala 2.6 setup on Ubunutu16.04. I continued with debugging this issue, and i can confirm that the error is due to the table 'functional_hbase.hbasealltypeserror' not getting created. To ascertain this , I manually ran the hbase table creation query "create 'functional_hbase.hbasealltypeserror', 'd' " in hbase shell and then tried the 'put' statements and it worked fine in hbase shell. Later, I tried to inject the same create query in '$ {IMPALA_HOME}/testdata/HBaseAllTypesError/functional_hbase.hbasealltypeserror.hbase', used in dataloading sequence and that worked fine too. I now require to find where the hbase table creation is triggered. For this, I've tried to trace the script execution flow. However, I'm not able to clearly reach at the table creation point. Below is the script execution flow i was tracing. buildall.sh - > create-load-data.sh -> create-hbase.sh In create-hbase.sh, I see only the 'put' statements for the hbase table. I also found a table creation query for ' functional_hbase.hbasealltypeserror' in '$ {IMPALA_HOME}/testdata/datasets/functional/functional_schema_template.sql, but I could not clearly find where it is triggered. Am I missing out on something ? It would be great if you could guide me to find the point where the table 'functional_hbase.hbasealltypeserror' is created. Regards, Valencia Inactive hide details for Jim Apple ---01/03/2017 09:06:00 PM---I think 16.04 has not always worked well. IIRC, Impala 2.6 was Jim Apple ---01/03/2017 09:06:00 PM---I think 16.04 has not always worked well. IIRC, Impala 2.6 was released only a month or two after Ub From: Jim Apple <[email protected]> To: "dev@impala" <[email protected]> Cc: Valencia Serrao/Austin/Contr/IBM@IBMUS, Nishidha Panpaliya/Austin/Contr/IBM@IBMUS, Sudarshan Jagadale/Austin/Contr/IBM@IBMUS, Manish Patil/Austin/Contr/IBM@IBMUS Date: 01/03/2017 09:06 PM Subject: Re: Issues while creating hbase data tables I think 16.04 has not always worked well. IIRC, Impala 2.6 was released only a month or two after Ubuntu 16.04. Our pre-merge testing and some of our developers use 14.04 with success. On Thu, Dec 29, 2016 at 8:17 PM, Valencia Serrao <[email protected]> wrote: > Hi All, > > While working to get impala 2.6 setup on a Ubuntu16.04 on ppc64le we are > facing following issue while testdata loading: > > .................................. > 16/12/28 01:56:25 ERROR client.AsyncProcess: Failed to get region location > org.apache.hadoop.hbase.TableNotFoundException: > functional_hbase.hbasealltypeserror > ..................................... > > Here's the complete log. (See attached file: HBaseLog.zip) > > Environment info: > HBase: hbase-1.2.0-cdh5.7.0 > Hadoop: hadoop-2.6.0-cdh5.7.0 > Command used to load data: ${IMPALA_HOME}/buildall.sh -noclean -notests > -testdata > > I had also tried using default hbase-1.2.0-cdh5.8.0, however, it throws a > InvalidTargetException and also the HRegionServer service would not start. > Therefore, I switched back to hbase5.7.0. > > Checking into the create_hbase.sh file, I executed the following query on > hbase shell manually and got the same error as above: > Query: put 'functional_hbase.hbasealltypeserror', '20', 'd:bigint_col', '0' > > IMO the required functional_hbase.hbasealltypeserror table is not getting > created(?). I am further tracing the code to get to the point where the > hbase tables are created i.e. in generate_schema script. > > When tried earlier on Ubuntu15.10, data loading was working fine with the > same environment. > > Please guide me on this issue. > > Regards, > Valencia
