[jira] [Created] (HIVE-17606) Improve security for DB notification related APIs
Tao Li created HIVE-17606: - Summary: Improve security for DB notification related APIs Key: HIVE-17606 URL: https://issues.apache.org/jira/browse/HIVE-17606 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Tao Li Assignee: Tao Li The purpose is to make sure only the superusers which are specified in the proxyuser settings can make the db notification related API calls, since this is supposed to be called by superuser/admin instead of any end user. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17587) Remove unnecessary filter from getPartitionsFromPartitionIds call
Tao Li created HIVE-17587: - Summary: Remove unnecessary filter from getPartitionsFromPartitionIds call Key: HIVE-17587 URL: https://issues.apache.org/jira/browse/HIVE-17587 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Tao Li Assignee: Tao Li Priority: Minor -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17585) Improve thread safety when loading dynamic partitions in parallel
Tao Li created HIVE-17585: - Summary: Improve thread safety when loading dynamic partitions in parallel Key: HIVE-17585 URL: https://issues.apache.org/jira/browse/HIVE-17585 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17570) Fix view deletion related test failures (create_view.q etc)
Tao Li created HIVE-17570: - Summary: Fix view deletion related test failures (create_view.q etc) Key: HIVE-17570 URL: https://issues.apache.org/jira/browse/HIVE-17570 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li Fixing the bug introduced by HIVE-17459. Sorry that did not capture that in a timely fashion. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17558) Skip non-native/temporary tables for constraint related scenarios
Tao Li created HIVE-17558: - Summary: Skip non-native/temporary tables for constraint related scenarios Key: HIVE-17558 URL: https://issues.apache.org/jira/browse/HIVE-17558 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li The change would be similar to HIVE-17422. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17522) cleanup old 'repl dump' dirs
Tao Li created HIVE-17522: - Summary: cleanup old 'repl dump' dirs Key: HIVE-17522 URL: https://issues.apache.org/jira/browse/HIVE-17522 Project: Hive Issue Type: Bug Components: repl Reporter: Tao Li Assignee: Tao Li We want to clean up the old dump dirs to save space and reduce scan time when needed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17515) Use SHA-256 for GenericUDFMaskHash to improve security
Tao Li created HIVE-17515: - Summary: Use SHA-256 for GenericUDFMaskHash to improve security Key: HIVE-17515 URL: https://issues.apache.org/jira/browse/HIVE-17515 Project: Hive Issue Type: Sub-task Components: UDF Reporter: Tao Li Assignee: Tao Li -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17514) Use SHA-256 for cookie signer to improve security
Tao Li created HIVE-17514: - Summary: Use SHA-256 for cookie signer to improve security Key: HIVE-17514 URL: https://issues.apache.org/jira/browse/HIVE-17514 Project: Hive Issue Type: Sub-task Components: HiveServer2 Reporter: Tao Li Assignee: Tao Li -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17513) Refactor PathUtils to not contain state (instance fields)
Tao Li created HIVE-17513: - Summary: Refactor PathUtils to not contain state (instance fields) Key: HIVE-17513 URL: https://issues.apache.org/jira/browse/HIVE-17513 Project: Hive Issue Type: Improvement Components: repl Reporter: Tao Li Assignee: Tao Li Priority: Minor This util class should just provide the static helper methods. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17504) Skip ACID table for replication
Tao Li created HIVE-17504: - Summary: Skip ACID table for replication Key: HIVE-17504 URL: https://issues.apache.org/jira/browse/HIVE-17504 Project: Hive Issue Type: Bug Components: repl Reporter: Tao Li Assignee: Tao Li Currently we are not supporting replicate ACID tables (which will be future work). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17496) Bootstrap repl is not cleaning up staging dirs
Tao Li created HIVE-17496: - Summary: Bootstrap repl is not cleaning up staging dirs Key: HIVE-17496 URL: https://issues.apache.org/jira/browse/HIVE-17496 Project: Hive Issue Type: Bug Components: repl Reporter: Tao Li Assignee: Tao Li This will put more pressure on the HDFS file limit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17480) repl dump sub dir should use UUID instead of timestamp
Tao Li created HIVE-17480: - Summary: repl dump sub dir should use UUID instead of timestamp Key: HIVE-17480 URL: https://issues.apache.org/jira/browse/HIVE-17480 Project: Hive Issue Type: Task Components: repl Reporter: Tao Li Assignee: Tao Li -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17459) View deletion operation failed to replicate on target cluster
Tao Li created HIVE-17459: - Summary: View deletion operation failed to replicate on target cluster Key: HIVE-17459 URL: https://issues.apache.org/jira/browse/HIVE-17459 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li View dropping is not replicated during incremental repl. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17422) Don't dump non-native/temporary tables during incremental dump
Tao Li created HIVE-17422: - Summary: Don't dump non-native/temporary tables during incremental dump Key: HIVE-17422 URL: https://issues.apache.org/jira/browse/HIVE-17422 Project: Hive Issue Type: Improvement Components: repl Reporter: Tao Li Currently during incremental dump, the non-native/temporary table info is partially dumped in metadata file and will be ignored later by the repl load. We can optimize it by moving the check (whether the table should be exported or not) earlier so that we don't save any info to dump file for such types of tables. CreateTableHandler already has this optimization, so we just need to apply similar logic to other scenarios. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17385) Fix repl load error for non-native tables
Tao Li created HIVE-17385: - Summary: Fix repl load error for non-native tables Key: HIVE-17385 URL: https://issues.apache.org/jira/browse/HIVE-17385 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17364) Add unit test to "alter view" replication
Tao Li created HIVE-17364: - Summary: Add unit test to "alter view" replication Key: HIVE-17364 URL: https://issues.apache.org/jira/browse/HIVE-17364 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li Priority: Minor -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17354) Fix "alter view" for incremental replication
Tao Li created HIVE-17354: - Summary: Fix "alter view" for incremental replication Key: HIVE-17354 URL: https://issues.apache.org/jira/browse/HIVE-17354 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17301) Make JSONMessageFactory.getTObj method thread safe
Tao Li created HIVE-17301: - Summary: Make JSONMessageFactory.getTObj method thread safe Key: HIVE-17301 URL: https://issues.apache.org/jira/browse/HIVE-17301 Project: Hive Issue Type: Bug Reporter: Tao Li This static method is using a singleton instance of TDeserializer, which is not thread safe. Instead we want to create a new instance per method call. This class is lightweight, so it should be fine from perf perspective. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17226) Use strong hashing as security improvement
Tao Li created HIVE-17226:
-
Summary: Use strong hashing as security improvement
Key: HIVE-17226
URL: https://issues.apache.org/jira/browse/HIVE-17226
Project: Hive
Issue Type: Improvement
Reporter: Tao Li
Assignee: Tao Li
There have been 2 places identified where weak hashing needs to be replaced by
SHA256.
1. CookieSigner.java uses MessageDigest.getInstance("SHA"). Mostly SHA is
mapped to SHA-1, which is not secure enough according to today's standards. We
should use SHA-256 instead.
2. GenericUDFMaskHash.java uses DigestUtils.md5Hex. MD5 is considered weak and
should be replaced by DigestUtils.sha256Hex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HIVE-17152) Improve security of random generator for HS2 cookies
Tao Li created HIVE-17152: - Summary: Improve security of random generator for HS2 cookies Key: HIVE-17152 URL: https://issues.apache.org/jira/browse/HIVE-17152 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li The random number generated is used as a secret to append to a sequence and SHA to implement a CookieSigner. If this is attackable, then it's possible for an attacker to sign a cookie as if we had. We should fix this and use SecureRandom as a stronger random function . HTTPAuthUtils has a similar issue. If that is attackable, an attacker might be able to create a similar cookie. Paired with the above issue with the CookieSigner, it could reasonably spoof a HS2 cookie. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-16979) Cache UGI for metastore
Tao Li created HIVE-16979: - Summary: Cache UGI for metastore Key: HIVE-16979 URL: https://issues.apache.org/jira/browse/HIVE-16979 Project: Hive Issue Type: Improvement Reporter: Tao Li FileSystem.closeAllForUGI is called per request against metastore to dispose UGI, which involves talking to HDFS name node and is time consuming. So the perf improvement would be caching and reusing the UGI. Per FileSystem.closeAllForUG call could take up to 20 ms as E2E latency against HDFS. Usually a Hive query could result in several calls against metastore, so we can save up to 50-100 ms per hive query. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-16419) Exclude hadoop related classes for JDBC stabdalone jar
Tao Li created HIVE-16419: - Summary: Exclude hadoop related classes for JDBC stabdalone jar Key: HIVE-16419 URL: https://issues.apache.org/jira/browse/HIVE-16419 Project: Hive Issue Type: Bug Components: Hive Reporter: Tao Li Priority: Blocker -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16172) Switch to a fairness lock to synchronize HS2 thrift client
Tao Li created HIVE-16172: - Summary: Switch to a fairness lock to synchronize HS2 thrift client Key: HIVE-16172 URL: https://issues.apache.org/jira/browse/HIVE-16172 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16170) Exclude relocation of org.apache.hadoop.security.* in the JDBC standalone jar
Tao Li created HIVE-16170: - Summary: Exclude relocation of org.apache.hadoop.security.* in the JDBC standalone jar Key: HIVE-16170 URL: https://issues.apache.org/jira/browse/HIVE-16170 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li There has been a use case that core-site.xml file is used along with the JDBC jar, which sets "hadoop.security.group.mapping" using the class names such as "org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback". This will cause CNF errors due to the renaming. So we need to exclude those security related classes in the relocation part. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16161) Disable "packaging.minimizeJar" for JDBC build
Tao Li created HIVE-16161: - Summary: Disable "packaging.minimizeJar" for JDBC build Key: HIVE-16161 URL: https://issues.apache.org/jira/browse/HIVE-16161 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li Priority: Critical -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16130) Remove jackson classes from hive-jdbc standalone jar
Tao Li created HIVE-16130: - Summary: Remove jackson classes from hive-jdbc standalone jar Key: HIVE-16130 URL: https://issues.apache.org/jira/browse/HIVE-16130 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15846) Relocate more dependencies (e.g. org.apache.zookeeper) for JDBC uber jar
Tao Li created HIVE-15846: - Summary: Relocate more dependencies (e.g. org.apache.zookeeper) for JDBC uber jar Key: HIVE-15846 URL: https://issues.apache.org/jira/browse/HIVE-15846 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
Tao Li created HIVE-15717:
-
Summary: Class "org.apache.hive.beeline.Rows.Row" constructor is
CPU consuming due to exception handling
Key: HIVE-15717
URL: https://issues.apache.org/jira/browse/HIVE-15717
Project: Hive
Issue Type: Improvement
Reporter: Tao Li
Assignee: Tao Li
The exception handling from the 3 methods calls (rowDeleted, rowDeleted and
rowUpdated). The implementation of these methods in
org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-15080) Remove unnecessary jars from jdbc standalone jar
Tao Li created HIVE-15080: - Summary: Remove unnecessary jars from jdbc standalone jar Key: HIVE-15080 URL: https://issues.apache.org/jira/browse/HIVE-15080 Project: Hive Issue Type: Bug Reporter: Tao Li Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15060) Unsupported warning from beeline with autoCommit=false
Tao Li created HIVE-15060: - Summary: Unsupported warning from beeline with autoCommit=false Key: HIVE-15060 URL: https://issues.apache.org/jira/browse/HIVE-15060 Project: Hive Issue Type: Bug Reporter: Tao Li -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14992) Relocate several common libraries in hive jdbc uber jar
Tao Li created HIVE-14992: - Summary: Relocate several common libraries in hive jdbc uber jar Key: HIVE-14992 URL: https://issues.apache.org/jira/browse/HIVE-14992 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14942) HS2 UI: Canceled queries show up in "Open Queries"
Tao Li created HIVE-14942: - Summary: HS2 UI: Canceled queries show up in "Open Queries" Key: HIVE-14942 URL: https://issues.apache.org/jira/browse/HIVE-14942 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14618) beeline fetch logging delays before query completion
Tao Li created HIVE-14618: - Summary: beeline fetch logging delays before query completion Key: HIVE-14618 URL: https://issues.apache.org/jira/browse/HIVE-14618 Project: Hive Issue Type: Bug Reporter: Tao Li -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14591) HS2 is shut down unexpectedly during the startup time
Tao Li created HIVE-14591: - Summary: HS2 is shut down unexpectedly during the startup time Key: HIVE-14591 URL: https://issues.apache.org/jira/browse/HIVE-14591 Project: Hive Issue Type: Bug Reporter: Tao Li If there is issue with Zookeeper (e.g. connection issues), then it takes HS2 some time to connect. During this time, Ambari could issue health checks against HS2 and the CloseSession call will trigger the shutdown of HS2, which is not expected. That triggering should happen only when the HS2 has been deregistered with Zookeeper, not during the startup time when HS2 is not registered with ZK yet. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14476) Fix logging issue for branch-1
Tao Li created HIVE-14476: - Summary: Fix logging issue for branch-1 Key: HIVE-14476 URL: https://issues.apache.org/jira/browse/HIVE-14476 Project: Hive Issue Type: Bug Reporter: Tao Li Assignee: Tao Li -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14441) TestOperationLogging fails
Tao Li created HIVE-14441: - Summary: TestOperationLogging fails Key: HIVE-14441 URL: https://issues.apache.org/jira/browse/HIVE-14441 Project: Hive Issue Type: Test Reporter: Tao Li -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14254) Correct the hive version by changing "svn" to "git"
Tao Li created HIVE-14254: - Summary: Correct the hive version by changing "svn" to "git" Key: HIVE-14254 URL: https://issues.apache.org/jira/browse/HIVE-14254 Project: Hive Issue Type: Bug Components: CLI Reporter: Tao Li Assignee: Tao Li Priority: Minor When running "hive --version", "subversion" is displayed below, which should be "git". $ hive --version Hive 2.1.0-SNAPSHOT Subversion git:// -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13714) How to config hive metastore mysql connection pool size?
Tao Li created HIVE-13714: - Summary: How to config hive metastore mysql connection pool size? Key: HIVE-13714 URL: https://issues.apache.org/jira/browse/HIVE-13714 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.13.1 Reporter: Tao Li How to config hive metastore mysql connection pool size? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
