[
https://issues.apache.org/jira/browse/SENTRY-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sravya Tirukkovalur updated SENTRY-1295:
----------------------------------------
Priority: Critical (was: Major)
> Investigate malformed paths in HMS db
> -------------------------------------
>
> Key: SENTRY-1295
> URL: https://issues.apache.org/jira/browse/SENTRY-1295
> Project: Sentry
> Issue Type: Bug
> Reporter: Sravya Tirukkovalur
> Priority: Critical
>
> Paths in HMS are expected to be in one of these forms:
> * hdfs://hostname:port/path
> * hdfs:///path
> * /path, in which case, scheme will be constructed from
> FileSystem.getDefaultURI
> * URIs with non hdfs scheme will just be ignored
> I came across atleast 2 sentry users where HMS did have paths which do not
> comply with above rules and hence HMS plugin initialization for pathupdates
> failed. See sentry-1260 and sentry-1270 for details on how these errors
> surface.
> With 1260 and 1270 we should have more information on what these malformed
> paths were. But we should continue to investigate and fix the root cause, It
> would most likely be in HMS code base. Until then, here is how you can
> diagnose and fix it manually:
> *Look for malformed paths in HMS* : Look in DBS as well as SDS tables.
> {code}
> SELECT "NAME", "DB_LOCATION_URI" FROM "DBS" WHERE NOT "DB_LOCATION_URI" LIKE
> 'hdfs://%/%';
> NAME | DB_LOCATION_URI
> -----------+--------------------
> db_name | hdfs://nameservice1
> (1 row)
> {code}
> *Fix it manually updating the HMS location*
> {code}
> UPDATE DBS
> SET DB_LOCATION_URI='hdfs://nameservice1/user/hive/warehouse/db_name.db'
> WHERE DB_ID=12345;
> {code}
> Lets track occurrences of these malformed paths here:
> * hdfs://nameservice1 : Not sure why would any one create a db/table in root
> directory? Should we accept this in Sentry?
> What does SKEWED_COL_VALUE_LOC_MAP.location in HMS correspond to? Double
> check if there are any malformed paths here?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
