Related to this, might it be better to use the s3a protocol instead of s3n? https://wiki.apache.org/hadoop/AmazonS3
Additionally, can anyone advise when EMRFS is required when storing Hive tables in S3? http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-overview-arch.html#emr-arch-storage On 22 January 2016 at 12:52, Zsolt Tóth <toth.zsolt....@gmail.com> wrote: > Hi, > > I'd like to use S3 as the hive warehouse on my emr 4.x cluster. > I've set hive.metastore.warehouse.dir=s3n://testbucket/hive_warehouse and > fs.s3.impl=org.apache.hadoop.fs.s3native.NativeS3FileSystem (not sure if > this is needed) in the hive-site.xml on the master node. Double checked the > "set -v" output, the properties are correct. > > When I run a command like "create table test1 (x String);" in Hive CLI, it > is created in the default warehouse dir (/user/hive/warehouse/) instead of > s3n://... > > What am I missing here? > > Thanks! > Zsolt >
