[
https://issues.apache.org/jira/browse/IMPALA-7238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532042#comment-16532042
]
Joe McDonnell commented on IMPALA-7238:
---------------------------------------
The create database doesn't show up at the metastore until 6:23:40:
{noformat}
2018-06-28T06:23:40,760 INFO [pool-7-thread-9] metastore.HiveMetaStore: 8:
source:127.0.0.1 create_database:
Database(name:testcreateexternaltable_23808_vu8cqo, description:null,
locationUri:null, parameters:null, ownerName:jenkins, ownerType:USER)
2018-06-28T06:23:40,760 INFO [pool-7-thread-9] HiveMetaStore.audit: ugi=jenkins
ip=127.0.0.1 cmd=source:127.0.0.1 create_database:
Database(name:testcreateexternaltable_23808_vu8cqo, description:null,
locationUri:null, parameters:null, ownerName:jenkins, ownerType:USER)
2018-06-28T06:23:40,760 WARN [pool-7-thread-9] metastore.ObjectStore: Failed to
get database testcreateexternaltable_23808_vu8cqo, returning
NoSuchObjectException
2018-06-28T06:23:41,081 INFO [pool-7-thread-9] common.FileUtils: Creating
directory if it doesn't exist:
s3a://impala-cdh5-s3-test/test-warehouse/testcreateexternaltable_23808_vu8cqo.db{noformat}
The customized connection timeout is not propagated to the connection that
executes the create database, so it uses the default of 45 seconds:
{code:java}
with __auto_closed_conn() as conn: # <- uses default timeout
with __auto_closed_cursor(conn) as cur:
cur.execute("CREATE DATABASE %s" % db_name)
with __auto_closed_conn(db_name=db_name, timeout=timeout) as conn:
...{code}
> test_kudu.TestCreateExternalTable sees unique database already exists
> ---------------------------------------------------------------------
>
> Key: IMPALA-7238
> URL: https://issues.apache.org/jira/browse/IMPALA-7238
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.1.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Critical
> Labels: broken-build, flaky
>
> All of the tests from query_test.test_kudu.TestCreateExternalTable fail with
> an error like:
> {noformat}
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/hiveserver2.py:704:
> in err_if_rpc_not_ok
> raise HiveServer2Error(resp.status.errorMessage)
> E HiveServer2Error: ImpalaRuntimeException: Error making 'createDatabase'
> RPC to Hive Metastore:
> E CAUSED BY: AlreadyExistsException: Database
> testcreateexternaltable_23808_vu8cqo already exists{noformat}
> It looks like the failures all happen at once in a single process. The first
> test to fail is test_kudu.TestCreateExternalTable.test_col_types. It takes 52
> seconds where all the other tests take no time. It also has an extra error on
> stderr:
> {noformat}
> -- connecting to: localhost:21000
> MainThread: Failed to open transport (tries_left=3)
> Traceback (most recent call last):
> File
> "/data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/hiveserver2.py",
> line 940, in _execute
> return func(request)
> File
> "/data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py",
> line 265, in ExecuteStatement
> return self.recv_ExecuteStatement()
> File
> "/data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py",
> line 276, in recv_ExecuteStatement
> (fname, mtype, rseqid) = self._iprot.readMessageBegin()
> File
> "/data/jenkins/workspace/impala-asf-master-core-s3/Impala-Toolchain/thrift-0.9.3-p4/python/lib64/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py",
> line 126, in readMessageBegin
> sz = self.readI32()
> File
> "/data/jenkins/workspace/impala-asf-master-core-s3/Impala-Toolchain/thrift-0.9.3-p4/python/lib64/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py",
> line 206, in readI32
> buff = self.trans.readAll(4)
> File
> "/data/jenkins/workspace/impala-asf-master-core-s3/Impala-Toolchain/thrift-0.9.3-p4/python/lib64/python2.7/site-packages/thrift/transport/TTransport.py",
> line 58, in readAll
> chunk = self.read(sz - have)
> File
> "/data/jenkins/workspace/impala-asf-master-core-s3/Impala-Toolchain/thrift-0.9.3-p4/python/lib64/python2.7/site-packages/thrift/transport/TTransport.py",
> line 159, in read
> self.__rbuf = StringIO(self.__trans.read(max(sz, self.__rbuf_size)))
> File
> "/data/jenkins/workspace/impala-asf-master-core-s3/Impala-Toolchain/thrift-0.9.3-p4/python/lib64/python2.7/site-packages/thrift/transport/TSocket.py",
> line 105, in read
> buff = self.handle.recv(sz)
> timeout: timed out
> MainThread: Error closing Impala cursor: Invalid session id:
> f54064f9a4604f23:fb686144269fc8b1{noformat}
> The other failures don't have this.
> This happened only once, so it is definitely intermittent. This has some
> similarity to IMPALA-6933, but this looks like a repeated failure in a single
> process, not a concurrency issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
