Glad you figured it out, and thanks for posting the solution!
On Mon, Jun 27, 2016 at 7:31 PM, Kidong Lee <[email protected]> wrote: > Hi Parth, > > thanks for your advice. > > I solved the problem. > > The problem was that the drill user which run drill did not know > HADOOP_HOME env, thus drill could not read my hadoop site.xml confs. > I have exported HADOOP_HOME explicitly with drill user and run drill, it > works fine!!! > > - Kidong. > > > > > 2016-06-28 9:58 GMT+09:00 Parth Chandra <[email protected]>: > > > You might need to make sure these files are in the Drill classpath. You > > could create a link to these files (or a copy) in your DRILL_HOME/conf > > directory and try. > > > > On Mon, Jun 27, 2016 at 4:45 PM, Kidong Lee <[email protected]> wrote: > > > > > Yes, I did. I have also tested my parquet file which reside in the > > > encryption zone and can be read with hive and parquet tool. > > > > > > - kidong > > > > > > 2016년 6월 28일 화요일, Parth Chandra<[email protected]>님이 작성한 메시지: > > > > > > > Hi Kidong, > > > > > > > > I haven't tried this myself, but my guess is that the KMS settings > > need > > > > to be provided at the HDFS layer not in the drill storage plugin. > > > > > > > > Specify hadoop.security.key.provider.path in core-site > > > > > > > > Specify dfs.encryption.key.provider.uri in hdfs-site > > > > > > > > Or did you already do that? > > > > > > > > Parth > > > > > > > > > > > > On Mon, Jun 27, 2016 at 1:11 AM, Kidong Lee <[email protected] > > > > <javascript:;>> wrote: > > > > > > > > > Hi, > > > > > > > > > > I got some problem using drill with HDFS Encryption. > > > > > > > > > > With Hive, DFS Storage, I got the errors like this: > > > > > Error: SYSTEM ERROR: IOException: No KeyProvider is configured, > > cannot > > > > > access an encrypted file > > > > > > > > > > Even if I have added some confs below to drill storage plugin, the > > > result > > > > > is the same: > > > > > > > > > > in dfs storage: > > > > > "config": { > > > > > "hadoop.security.key.provider.path": "kms://[email protected] > > > > <javascript:;>; > > > > > xxxx.com:16000/kms", > > > > > "dfs.encryption.key.provider.uri": "kms://[email protected] > > > > <javascript:;>; > > > > > xxxx.com:16000/kms" > > > > > } > > > > > > > > > > in hive storage: > > > > > "configProps": { > > > > > ... > > > > > "hadoop.security.key.provider.path": "kms://[email protected] > > > > <javascript:;>; > > > > > xxxx.com:16000/kms", > > > > > "dfs.encryption.key.provider.uri": "kms://[email protected] > > > > <javascript:;>; > > > > > xxxx.com:16000/kms" > > > > > } > > > > > > > > > > I have tested with hive for the tables of the encrypted files on > > hdfs, > > > it > > > > > works fine. > > > > > > > > > > Any idea. > > > > > > > > > > - Kidong Lee. > > > > > > > > > > > > > > >
