It looks to me like functional_schema_template.sql is called from
testdata/bin/generate-schema-statements.py, which is called from
bin/load-data.py, which is called from testdata/bin/create-load-data.sh,
which is called from buildall.sh
$ git grep schema_tem
...
testdata/bin/generate-schema-statements.py:def
parse_schema_template_file(file_name):
testdata/bin/generate-schema-statements.py: schema_template_file =
os.path.join(DATASET_DIR, target_dataset,
testdata/bin/generate-schema-statements.py:
'%s_schema_template.sql' % target_dataset)
...
$ git grep generate-schema-statements
bin/load-data.py:GENERATE_SCHEMA_CMD = "generate-schema-statements.py
--exploration_strategy=%s "\
$ git grep load-data
...
buildall.sh: "${IMPALA_HOME}/testdata/bin/create-load-data.sh"
${CREATE_LOAD_DATA_ARGS} <<< Y
...
testdata/bin/create-load-data.sh: if ! impala-python -u
${IMPALA_HOME}/bin/load-data.py ${ARGS[@]} &> ${LOG_FILE}; then
testdata/bin/create-load-data.sh: if ! impala-python -u
${IMPALA_HOME}/bin/load-data.py --workloads all\
On Wed, Jan 4, 2017 at 10:32 PM, Valencia Serrao <[email protected]> wrote:
> Hi Jim,
>
> Thanks for the inputs. But our requirement is to get Impala 2.6 setup on
> Ubunutu16.04.
>
> I continued with debugging this issue, and i can confirm that the error is
> due to the table 'functional_hbase.hbasealltypeserror' not getting
> created.
> To ascertain this , I manually ran the hbase table creation query *"create
> '**functional_hbase.hbasealltypeserror**', 'd' "* in hbase shell and then
> tried the 'put' statements and it worked fine in hbase shell. Later, I
> tried to inject the same create query in '${IMPALA_HOME}/testdata/
> HBaseAllTypesError/functional_hbase.hbasealltypeserror.hbase', used in
> dataloading sequence and that worked fine too.
>
> I now require to find where the hbase table creation is triggered. For
> this, I've tried to trace the script execution flow. However, I'm not able
> to clearly reach at the table creation point. Below is the script execution
> flow i was tracing.
> *buildall.sh - > create-load-data.sh -> create-hbase.sh*
> In create-hbase.sh, I see only the 'put' statements for the hbase table. I
> also found a table creation query for 'functional_hbase.hbasealltypeserror'
> in '${IMPALA_HOME}/testdata/datasets/functional/
> functional_schema_template.sql, but I could not clearly find where it is
> triggered.
> Am I missing out on something ?
>
> It would be great if you could guide me to find the point where the table '
> functional_hbase.hbasealltypeserror' is created.
>
> Regards,
> Valencia
>
> [image: Inactive hide details for Jim Apple ---01/03/2017 09:06:00 PM---I
> think 16.04 has not always worked well. IIRC, Impala 2.6 was]Jim Apple
> ---01/03/2017 09:06:00 PM---I think 16.04 has not always worked well. IIRC,
> Impala 2.6 was released only a month or two after Ub
>
> From: Jim Apple <[email protected]>
> To: "dev@impala" <[email protected]>
> Cc: Valencia Serrao/Austin/Contr/IBM@IBMUS, Nishidha
> Panpaliya/Austin/Contr/IBM@IBMUS, Sudarshan Jagadale/Austin/Contr/IBM@IBMUS,
> Manish Patil/Austin/Contr/IBM@IBMUS
> Date: 01/03/2017 09:06 PM
> Subject: Re: Issues while creating hbase data tables
> ------------------------------
>
>
>
> I think 16.04 has not always worked well. IIRC, Impala 2.6 was
> released only a month or two after Ubuntu 16.04. Our pre-merge testing
> and some of our developers use 14.04 with success.
>
> On Thu, Dec 29, 2016 at 8:17 PM, Valencia Serrao <[email protected]>
> wrote:
> > Hi All,
> >
> > While working to get impala 2.6 setup on a Ubuntu16.04 on ppc64le we are
> > facing following issue while testdata loading:
> >
> > ..................................
> > 16/12/28 01:56:25 ERROR client.AsyncProcess: Failed to get region
> location
> > org.apache.hadoop.hbase.TableNotFoundException:
> > functional_hbase.hbasealltypeserror
> > .....................................
> >
> > Here's the complete log. (See attached file: HBaseLog.zip)
> >
> > Environment info:
> > HBase: hbase-1.2.0-cdh5.7.0
> > Hadoop: hadoop-2.6.0-cdh5.7.0
> > Command used to load data: ${IMPALA_HOME}/buildall.sh -noclean -notests
> > -testdata
> >
> > I had also tried using default hbase-1.2.0-cdh5.8.0, however, it throws a
> > InvalidTargetException and also the HRegionServer service would not
> start.
> > Therefore, I switched back to hbase5.7.0.
> >
> > Checking into the create_hbase.sh file, I executed the following query on
> > hbase shell manually and got the same error as above:
> > Query: put 'functional_hbase.hbasealltypeserror', '20', 'd:bigint_col',
> '0'
> >
> > IMO the required functional_hbase.hbasealltypeserror table is not
> getting
> > created(?). I am further tracing the code to get to the point where the
> > hbase tables are created i.e. in generate_schema script.
> >
> > When tried earlier on Ubuntu15.10, data loading was working fine with the
> > same environment.
> >
> > Please guide me on this issue.
> >
> > Regards,
> > Valencia
>
>
>
>
>
