Hi Alex,
I was going more deeper through the logs. I have some findings and queries:
1. At the "Invalidating Metadata" step (as mentioned in below mail), i
noticed that, it is trying to use kerberos. Perhaps, this is preventing the
testdata generation from proceeding, as we are not using Kerberos.
I need to know how this can be done without involving Kerberos
support ?
2. I had executed the fe tests despite the incomplete testdata generation,
the tests started and surely have failed. Many of these (null pointer
exception in AuthorzationTests) have a common cause: "tpch database does
not exist."
e.g. as shown
in .Impala/cluster_logs/query_tests/test-run-workload.log.
Does the "tpch" database gets created after the current blocker step
"Invalidating Metadata" ?
3. In the fe test console output log, another error shown:
============================= test session starts
==============================
platform linux2 -- Python 2.7.5 -- py-1.4.30 -- pytest-2.7.2
rootdir: /work/, inifile:
plugins: random, xdist
ERROR: file not found:/work/I
mpala/../Impala-auxiliary-tests/tests/aux_custom_cluster_tests/
These are not present/created on my vm. May i know when these get
created ?
4. Could you also share the total number of fe tests ?
Looking forward to your reply.
Regards,
Valencia
From: Valencia Serrao/Austin/Contr/IBM
To: [email protected], Alex Behm
<[email protected]>
Cc: Sudarshan Jagadale/Austin/Contr/IBM@IBMUS, Nishidha
Panpaliya/Austin/Contr/IBM@IBMUS, Valencia
Serrao/Austin/Contr/IBM@IBMUS
Date: 04/30/2016 09:05 AM
Subject: Fw: Issues with generating testdata for Impala
Hi Alex,
I've been able to make some progress on testdata generation, however, i
still face the following issues:
*******************************************************************************************************************************************************************
Invalidating Metadata
(load-functional-query-exhaustive-impala-load-generated-parquet-none-none.sql):
INSERT OVERWRITE TABLE functional_parquet.alltypes partition (year, month)
SELECT id, bool_col, tinyint_col, smallint_col, int_col, bigint_col,
float_col, double_col, date_string_col, string_col, timestamp_col, year,
month
FROM functional.alltypes
Data Loading from Impala failed with error: ImpalaBeeswaxException:
INNER EXCEPTION: <class 'socket.error'>
MESSAGE: [Errno 104] Connection reset by peer
Error in /root/nishidha/Impala/testdata/bin/create-load-data.sh at line 41:
while [ -n "$*" ]
Error in /root/nishidha/Impala/buildall.sh at line 368: $
{IMPALA_HOME}/testdata/bin/create-load-data.sh ${CREATE_LOAD_DATA_ARGS} <<<
Y
*************************************************************************************************************************************************************************
i continued with fe tests as is. Here is the complete output log.
[attachment "fe_test_output.zip" deleted by Valencia
Serrao/Austin/Contr/IBM]
Cluster logs: [attachment "cluster_logs.7z" deleted by Valencia
Serrao/Austin/Contr/IBM]
Kindly guide me on the same.
Regards,
Valencia
----- Forwarded by Valencia Serrao/Austin/Contr/IBM on 04/29/2016 10:57 AM
-----
From: Sudarshan Jagadale/Austin/Contr/IBM
To: Valencia Serrao/Austin/Contr/IBM@IBMUS
Date: 04/29/2016 10:49 AM
Subject: Fw: Issues with generating testdata for Impala
FYI
Thanks and Regards
Sudarshan Jagadale
Power Open Source Solutions
----- Forwarded by Sudarshan Jagadale/Austin/Contr/IBM on 04/29/2016 10:48
AM -----
From: Alex Behm <[email protected]>
To: [email protected]
Cc: Sudarshan Jagadale/Austin/Contr/IBM@IBMUS, Nishidha
Panpaliya/Austin/Contr/IBM@IBMUS
Date: 04/28/2016 09:34 PM
Subject: Re: Issues with generating testdata for Impala
Hi Valencia,
sorry I did not get the attachment. Would you be able to tar.gz and attach
the whole cluster_logs directory?
Alex
On Thu, Apr 28, 2016 at 6:23 AM, Valencia Serrao <[email protected]>
wrote:
Hi Alex,
I tried building impala again with the following:
HDFS CDH 5.7.0 (
http://www.cloudera.com/documentation/enterprise/release-notes/topics/cdh_vd_cdh_package_tarball_57.html#topic_3
)
HBASE CDH 5.7.0 SNAPSHOT (
http://archive.cloudera.com/cdh5/cdh/5/hbase-1.2.0-cdh5.7.0.tar.gz )
- this required to patch in a fix (
https://issues.apache.org/jira/secure/attachment/12792536/HBASE-15322-branch-1.2.patch
)
HIVE CDH 5.8.0 SNAPSHOT
With the above combination, i'm able to move past the exception and also
have the RegionServer service up and running. However, it now gives error
as below:
********************************************************************************************************************
(load-functional-query-exhaustive-impala-generated-text-none-none.sql):
CREATE EXTERNAL TABLE IF NOT EXISTS functional.decimal_tbl (
d1 DECIMAL,
d2 DECIMAL(10, 0),
d3 DECIMAL(20, 10),
d4 DECIMAL(38, 38),
d5 DECIMAL(10, 5))
PARTITIONED BY (d6 DECIMAL(9, 0))
ROW FORMAT delimited fields terminated by ','
STORED AS TEXTFILE
LOCATION '/test-warehouse/decimal_tbl'
(load-functional-query-exhaustive-impala-generated-text-none-none.sql):
USE functional
(load-functional-query-exhaustive-impala-generated-text-none-none.sql):
ALTER TABLE decimal_tbl ADD IF NOT EXISTS PARTITION(d6=1)
Data Loading from Impala failed with error: ImpalaBeeswaxException:
INNER EXCEPTION: <class
'impala._thrift_gen.beeswax.ttypes.BeeswaxException'>
MESSAGE:
Error: null
******************************************************************************************************************
Here is the complete log for the same. (See attached file:
data-load-functional-exhaustive.log)
It would great if you could guide me on this issue, so i could proceed
with the fe tests.
Still awaiting link to the source code of HDFS CDH 5.8.0
Regards,
Valencia
