Joe McDonnell created IMPALA-7245:
-------------------------------------
Summary: test_kudu.py::test_kudu_insert() fails with "Error adding
columns to Kudu table tbl_with_defaults"
Key: IMPALA-7245
URL: https://issues.apache.org/jira/browse/IMPALA-7245
Project: IMPALA
Issue Type: Bug
Components: Catalog
Affects Versions: Impala 3.1.0
Reporter: Joe McDonnell
query_test.test_kudu.TestKuduOperations.test_kudu_insert hit the following
error:
{noformat}
query_test/test_kudu.py:93: in test_kudu_insert
self.run_test_case('QueryTest/kudu_insert', vector, use_db=unique_database)
common/impala_test_suite.py:405: in run_test_case
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:620: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:160: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:173: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:343: in __execute_query
handle = self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:339: in execute_query_async
return self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:483: in __do_rpc
raise ImpalaBeeswaxException(self.__build_error_message(b), b)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
E INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
E MESSAGE: ImpalaRuntimeException: Error adding columns to Kudu table
tbl_with_defaults
E CAUSED BY: NonRecoverableException: The column already exists: j{noformat}
This is very similar to what we saw in IMPALA-6107, which we couldn't reproduce.
It seems to be related to our interaction with the Kudu client. The statement
starts executing at 23:05:11.17. From impalad.INFO:
{noformat}
I0629 23:05:11.170009 20602 impala-beeswax-server.cc:54] query(): query=alter
table tbl_with_defaults add columns (j int null, k int not null default
10000){noformat}
This results in actions in catalogd, but catalogd encounters a timeout in the
AsyncKuduClient while processing this and then immediately hits the error:
{noformat}
I0629 23:05:11.212054 28109 AsyncKuduClient.java:1756] Invalidating location
master-127.0.0.1:7051(127.0.0.1:7051) for tablet Kudu Master: [peer
master-127.0.0.1:7051(127.0.0.1:7051)] encountered a read timeout; closing the
channel
I0629 23:05:11.288261 11518 jni-util.cc:230]
org.apache.impala.common.ImpalaRuntimeException: Error adding columns to Kudu
table tbl_with_defaults
at
org.apache.impala.service.KuduCatalogOpExecutor.alterKuduTable(KuduCatalogOpExecutor.java:499)
at
org.apache.impala.service.KuduCatalogOpExecutor.addColumn(KuduCatalogOpExecutor.java:412)
at
org.apache.impala.service.CatalogOpExecutor.alterKuduTable(CatalogOpExecutor.java:600)
at
org.apache.impala.service.CatalogOpExecutor.alterTable(CatalogOpExecutor.java:420)
at
org.apache.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:270)
at org.apache.impala.service.JniCatalog.execDdl(JniCatalog.java:146)
Caused by: org.apache.kudu.client.NonRecoverableException: The column already
exists: j
at
org.apache.kudu.client.KuduException.transformException(KuduException.java:110)
at org.apache.kudu.client.KuduClient.joinAndHandleException(KuduClient.java:351)
at org.apache.kudu.client.KuduClient.alterTable(KuduClient.java:141)
at
org.apache.impala.service.KuduCatalogOpExecutor.alterKuduTable(KuduCatalogOpExecutor.java:494)
... 5 more
{noformat}
The Kudu master sees the alter table request, then the connection is torn down,
then a duplicate alter table request:
{noformat}
I0629 23:05:11.197782 16157 catalog_manager.cc:2086] Servicing AlterTable
request from {username='jenkins'} at 127.0.0.1:34332:
table { table_name: "impala::test_kudu_insert_ca9324f5.tbl_with_defaults" }
alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "j" type:
INT32 is_key: false is_nullable: true cfile_block_size: 0 } } }
alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "k" type:
INT32 is_key: false is_nullable: false read_default_value: "\020\'\000\000"
cfile_block_size: 0 } } }
W0629 23:05:11.218549 15925 connection.cc:420] Connection torn down before Call
kudu.master.MasterService.AlterTable from 127.0.0.1:34332 (request call id 24)
could send its response
I0629 23:05:11.239550 16157 catalog_manager.cc:4132] T
00000000000000000000000000000000 P 32ff35a1134b48fbb17f9b6cdefd6b7b:
impala::test_kudu_insert_ca9324f5.tbl_with_defaults
[id=9f784287d5274dd9a56c434c977a3b4f] alter complete (version 1)
I0629 23:05:11.283748 16157 catalog_manager.cc:2086] Servicing AlterTable
request from {username='jenkins'} at 127.0.0.1:34380:
table { table_name: "impala::test_kudu_insert_ca9324f5.tbl_with_defaults" }
alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "j" type:
INT32 is_key: false is_nullable: true cfile_block_size: 0 } } }
alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "k" type:
INT32 is_key: false is_nullable: false read_default_value: "\020\'\000\000"
cfile_block_size: 0 } } }{noformat}
This indicates some problem where the column is added, but then there is a
communication problem and we retry anyway.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
