[
https://issues.apache.org/jira/browse/IMPALA-5968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-5968.
-----------------------------------
Fix Version/s: Not Applicable
Resolution: Later
There is no plan to fix this, closing.
> load_nested.py fails on secure cluster: no write access to /user on HDFS
> ------------------------------------------------------------------------
>
> Key: IMPALA-5968
> URL: https://issues.apache.org/jira/browse/IMPALA-5968
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 2.10.0
> Reporter: Matthew Mulder
> Priority: Minor
> Fix For: Not Applicable
>
>
> {code}testdata/bin/load_nested.py \
> -s tpch_100_parquet \
> -t tpch_100_nested_parquet \
> --cm-host=vc0332.test \
> -p 34 --use-kerberos --use-ssl
> Output: 2017-09-20 14:18:06,174 INFO:db_connection[234]:Creating database
> tpch_100_nested_parquet
> 2017-09-20 14:18:06,737 INFO:load_nested[98]:Creating temp orders (chunk 1 of
> 34)
> 2017-09-20 14:18:31,420 INFO:load_nested[98]:Creating temp orders (chunk 2 of
> 34)
> 2017-09-20 14:18:52,146 INFO:load_nested[98]:Creating temp orders (chunk 3 of
> 34)
> 2017-09-20 14:19:12,398 INFO:load_nested[98]:Creating temp orders (chunk 4 of
> 34)
> 2017-09-20 14:19:32,109 INFO:load_nested[98]:Creating temp orders (chunk 5 of
> 34)
> 2017-09-20 14:19:51,823 INFO:load_nested[98]:Creating temp orders (chunk 6 of
> 34)
> 2017-09-20 14:20:12,521 INFO:load_nested[98]:Creating temp orders (chunk 7 of
> 34)
> 2017-09-20 14:20:33,220 INFO:load_nested[98]:Creating temp orders (chunk 8 of
> 34)
> 2017-09-20 14:20:53,417 INFO:load_nested[98]:Creating temp orders (chunk 9 of
> 34)
> 2017-09-20 14:21:13,143 INFO:load_nested[98]:Creating temp orders (chunk 10
> of 34)
> 2017-09-20 14:21:34,361 INFO:load_nested[98]:Creating temp orders (chunk 11
> of 34)
> 2017-09-20 14:21:53,048 INFO:load_nested[98]:Creating temp orders (chunk 12
> of 34)
> 2017-09-20 14:22:13,743 INFO:load_nested[98]:Creating temp orders (chunk 13
> of 34)
> 2017-09-20 14:22:34,439 INFO:load_nested[98]:Creating temp orders (chunk 14
> of 34)
> 2017-09-20 14:22:55,160 INFO:load_nested[98]:Creating temp orders (chunk 15
> of 34)
> 2017-09-20 14:23:15,868 INFO:load_nested[98]:Creating temp orders (chunk 16
> of 34)
> 2017-09-20 14:23:37,075 INFO:load_nested[98]:Creating temp orders (chunk 17
> of 34)
> 2017-09-20 14:23:57,260 INFO:load_nested[98]:Creating temp orders (chunk 18
> of 34)
> 2017-09-20 14:24:18,967 INFO:load_nested[98]:Creating temp orders (chunk 19
> of 34)
> 2017-09-20 14:24:39,680 INFO:load_nested[98]:Creating temp orders (chunk 20
> of 34)
> 2017-09-20 14:24:59,364 INFO:load_nested[98]:Creating temp orders (chunk 21
> of 34)
> 2017-09-20 14:25:20,556 INFO:load_nested[98]:Creating temp orders (chunk 22
> of 34)
> 2017-09-20 14:25:40,240 INFO:load_nested[98]:Creating temp orders (chunk 23
> of 34)
> 2017-09-20 14:26:00,949 INFO:load_nested[98]:Creating temp orders (chunk 24
> of 34)
> 2017-09-20 14:26:21,146 INFO:load_nested[98]:Creating temp orders (chunk 25
> of 34)
> 2017-09-20 14:26:41,841 INFO:load_nested[98]:Creating temp orders (chunk 26
> of 34)
> 2017-09-20 14:27:01,530 INFO:load_nested[98]:Creating temp orders (chunk 27
> of 34)
> 2017-09-20 14:27:21,219 INFO:load_nested[98]:Creating temp orders (chunk 28
> of 34)
> 2017-09-20 14:27:42,921 INFO:load_nested[98]:Creating temp orders (chunk 29
> of 34)
> 2017-09-20 14:28:04,617 INFO:load_nested[98]:Creating temp orders (chunk 30
> of 34)
> 2017-09-20 14:28:25,808 INFO:load_nested[98]:Creating temp orders (chunk 31
> of 34)
> 2017-09-20 14:28:48,031 INFO:load_nested[98]:Creating temp orders (chunk 32
> of 34)
> 2017-09-20 14:29:08,213 INFO:load_nested[98]:Creating temp orders (chunk 33
> of 34)
> 2017-09-20 14:29:29,918 INFO:load_nested[98]:Creating temp orders (chunk 34
> of 34)
> 2017-09-20 14:29:50,610 INFO:load_nested[128]:Creating temp customers (chunk
> 1 of 34)
> 2017-09-20 14:30:18,747 INFO:load_nested[128]:Creating temp customers (chunk
> 2 of 34)
> 2017-09-20 14:30:44,058 INFO:load_nested[128]:Creating temp customers (chunk
> 3 of 34)
> 2017-09-20 14:31:07,818 INFO:load_nested[128]:Creating temp customers (chunk
> 4 of 34)
> 2017-09-20 14:31:31,565 INFO:load_nested[128]:Creating temp customers (chunk
> 5 of 34)
> 2017-09-20 14:31:54,799 INFO:load_nested[128]:Creating temp customers (chunk
> 6 of 34)
> 2017-09-20 14:32:18,529 INFO:load_nested[128]:Creating temp customers (chunk
> 7 of 34)
> 2017-09-20 14:32:41,762 INFO:load_nested[128]:Creating temp customers (chunk
> 8 of 34)
> 2017-09-20 14:33:04,974 INFO:load_nested[128]:Creating temp customers (chunk
> 9 of 34)
> 2017-09-20 14:33:28,713 INFO:load_nested[128]:Creating temp customers (chunk
> 10 of 34)
> 2017-09-20 14:33:51,923 INFO:load_nested[128]:Creating temp customers (chunk
> 11 of 34)
> 2017-09-20 14:34:15,656 INFO:load_nested[128]:Creating temp customers (chunk
> 12 of 34)
> 2017-09-20 14:34:38,900 INFO:load_nested[128]:Creating temp customers (chunk
> 13 of 34)
> 2017-09-20 14:35:02,628 INFO:load_nested[128]:Creating temp customers (chunk
> 14 of 34)
> 2017-09-20 14:35:26,347 INFO:load_nested[128]:Creating temp customers (chunk
> 15 of 34)
> 2017-09-20 14:35:51,079 INFO:load_nested[128]:Creating temp customers (chunk
> 16 of 34)
> 2017-09-20 14:36:14,804 INFO:load_nested[128]:Creating temp customers (chunk
> 17 of 34)
> 2017-09-20 14:36:38,535 INFO:load_nested[128]:Creating temp customers (chunk
> 18 of 34)
> 2017-09-20 14:37:01,765 INFO:load_nested[128]:Creating temp customers (chunk
> 19 of 34)
> 2017-09-20 14:37:25,512 INFO:load_nested[128]:Creating temp customers (chunk
> 20 of 34)
> 2017-09-20 14:37:49,745 INFO:load_nested[128]:Creating temp customers (chunk
> 21 of 34)
> 2017-09-20 14:38:13,980 INFO:load_nested[128]:Creating temp customers (chunk
> 22 of 34)
> 2017-09-20 14:38:37,706 INFO:load_nested[128]:Creating temp customers (chunk
> 23 of 34)
> 2017-09-20 14:39:01,428 INFO:load_nested[128]:Creating temp customers (chunk
> 24 of 34)
> 2017-09-20 14:39:25,652 INFO:load_nested[128]:Creating temp customers (chunk
> 25 of 34)
> 2017-09-20 14:39:49,368 INFO:load_nested[128]:Creating temp customers (chunk
> 26 of 34)
> 2017-09-20 14:40:13,600 INFO:load_nested[128]:Creating temp customers (chunk
> 27 of 34)
> 2017-09-20 14:40:38,902 INFO:load_nested[128]:Creating temp customers (chunk
> 28 of 34)
> 2017-09-20 14:41:03,117 INFO:load_nested[128]:Creating temp customers (chunk
> 29 of 34)
> 2017-09-20 14:41:26,847 INFO:load_nested[128]:Creating temp customers (chunk
> 30 of 34)
> 2017-09-20 14:41:51,068 INFO:load_nested[128]:Creating temp customers (chunk
> 31 of 34)
> 2017-09-20 14:42:15,300 INFO:load_nested[128]:Creating temp customers (chunk
> 32 of 34)
> 2017-09-20 14:42:39,527 INFO:load_nested[128]:Creating temp customers (chunk
> 33 of 34)
> 2017-09-20 14:43:03,753 INFO:load_nested[128]:Creating temp customers (chunk
> 34 of 34)
> 2017-09-20 14:43:28,187 INFO:load_nested[178]:Creating temp regions
> 2017-09-20 14:43:38,690 INFO:load_nested[208]:Creating temp suppliers
> 2017-09-20 14:44:51,084 INFO:load_nested[244]:Creating parts
> 2017-09-20 14:44:59,144 INFO:cluster[422]:kerberos module not found;
> attempting to install it...
> 2017-09-20 14:45:00,056 INFO:cluster[429]:kerberos installation complete.
> /var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90:
> InsecurePlatformWarning: A true SSLContext object is not available. This
> prevents urllib3 from configuring SSL appropriately and may cause certain SSL
> connections to fail. For more information, see
> https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
> InsecurePlatformWarning
> /var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:768:
> InsecureRequestWarning: Unverified HTTPS request is being made. Adding
> certificate verification is strongly advised. See:
> https://urllib3.readthedocs.org/en/latest/security.html
> InsecureRequestWarning)
> 2017-09-20 14:45:21,562 ERROR:kerberos_[226]:handle_other(): Mutual
> authentication unavailable on 403 response
> Traceback (most recent call last):
> File "testdata/bin/load_nested.py", line 320, in <module>
> load()
> File "testdata/bin/load_nested.py", line 253, in load
> cluster.hdfs.ensure_home_dir()
> File
> "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/tests/comparison/cluster.py",
> line 402, in ensure_home_dir
> client.makedirs(hdfs_dir)
> File
> "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/client.py",
> line 866, in makedirs
> self._mkdirs(hdfs_path, permission=permission)
> File
> "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/client.py",
> line 90, in api_handler
> **self.kwargs
> File
> "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/ext/kerberos.py",
> line 98, in _request
> return super(KerberosClient, self)._request(method, url, **kwargs)
> File
> "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/client.py",
> line 182, in _request
> return _on_error(response)
> File
> "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/client.py",
> line 44, in _on_error
> raise HdfsError(message)
> hdfs.util.HdfsError: Permission denied: user=systest, access=WRITE,
> inode="/user":hdfs:supergroup:drwxr-xr-x{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
