[
https://issues.apache.org/jira/browse/SENTRY-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anne Yu updated SENTRY-1230:
----------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks [~haohao] for reviewing it.
commit b4a64f976dd3b98225bbbe8fcf4bf8d7ecddce90
Author: Anne Yu <[email protected]>
Date: Fri May 20 16:00:50 2016 -0700
SENTRY-1230: Add filesystem tests to test Sentry with user data storage on
S3. The tests can also be run on other types of fileSystems (ms azure, hdfs).
Just ensure pass correct parameters. (Anne Yu, reviewed by Haohao)
For example,
-Dsentry.e2etest.DFSType=S3DFS
-Dsentry.e2etest.storage.uri=s3a://sentry-s3
-Dfs.s3a.access.key=a
-Dfs.s3a.secret.key=s
export HIVE_CONF_DIR=/etc/hive/conf/hite-site.xml
> Add basic testing workflow to test Sentry with Hive storage on S3
> -----------------------------------------------------------------
>
> Key: SENTRY-1230
> URL: https://issues.apache.org/jira/browse/SENTRY-1230
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
> Affects Versions: 1.8.0
> Reporter: Anne Yu
> Assignee: Anne Yu
> Fix For: 1.8.0
>
> Attachments: SENTRY-1230.0.patch, SENTRY-1230.1.patch
>
>
> As Hive (and other Apache Engines) is capable of having storage in the
> AWS/S3, Sentry should be able to secure URIs with S3 schema. Basic workflow
> could be create an external table with location pointing to S3:
> 1. sudo -u hdfs hadoop fs ${S3} -mkdir -p s3a://sentry-s3/cdh-sentry/db/tbl
> 2. CREATE EXTERNAL TABLE my_s3_table (viewTime INT, userid BIGINT, page_url
> STRING, referrer_url STRING, ip STRING COMMENT 'IP Address of the User',
> country STRING COMMENT 'country of origination') COMMENT 'This is the staging
> page view table' ROW FORMAT DELIMITED FIELDS TERMINATED BY '\054' STORED AS
> TEXTFILE LOCATION 's3a://sentry-s3/cdh-sentry/db/tbl'; (show create table
> page_view shows the location points to s3a)
> 3. insert into table my_s3_table values (1, 1001, 'test_url',
> 'reference_url', '201.245.14.5', 'us'); (sudo -u hdfs hadoop fs ${S3} -cat
> s3a://sentry-s3/cdh-sentry/db/tbl/000000_0 shows the data is successfully
> created in s3a file 000000: 1,1001,test_url,reference_url,201.245.14.5,us);
> 4. Sentry is able to grant URI privilege on
> ‘s3a://sentry-s3/cdh-sentry/db/tbl’, Without URI privilege, Sentry throws
> exception such as, “User systest does not have privileges for CREATETABLE;
> The required privileges:
> Server=server1->URI=s3a://sentry-s3/cdh-sentry/db/tbl->action=*;
> (state=42000,code=40000)”
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
