[jira] [Created] (HIVE-6598) Importing the project into eclipse as maven project have some issues
Chinna Rao Lalam created HIVE-6598: -- Summary: Importing the project into eclipse as maven project have some issues Key: HIVE-6598 URL: https://issues.apache.org/jira/browse/HIVE-6598 Project: Hive Issue Type: Bug Environment: Windows 8 ,Eclipse Kepler and Maven 3.1.1 Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Importing the project into eclipse as maven project throwing these problems. Plugin execution not covered by lifecycle configuration: org.apache.maven.plugins:maven-antrun-plugin:1.7:run (execution: setup-test-dirs, phase: process-test-resources) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6594) UnsignedInt128 addition does not increase internal int array count resulting in corrupted values during serialization
[
https://issues.apache.org/jira/browse/HIVE-6594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925477#comment-13925477
]
Hive QA commented on HIVE-6594:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633542/HIVE-6594.2.patch
{color:green}SUCCESS:{color} +1 5375 tests passed
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1685/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1685/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633542
UnsignedInt128 addition does not increase internal int array count resulting
in corrupted values during serialization
-
Key: HIVE-6594
URL: https://issues.apache.org/jira/browse/HIVE-6594
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.13.0
Reporter: Remus Rusanu
Assignee: Remus Rusanu
Attachments: HIVE-6594.1.patch, HIVE-6594.2.patch
Discovered this while investigating why my fix for HIVE-6222 produced diffs.
I discovered that Decimal128.addDestructive does not adjust the internal
count when an the number of relevant ints increases. Since this count is used
in the fast HiveDecimalWriter conversion code, the results are off.
The root cause is UnsignedDecimal128.differenceInternal does not do an
updateCount() on the result.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6598) Importing the project into eclipse as maven project have some issues
[ https://issues.apache.org/jira/browse/HIVE-6598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-6598: --- Attachment: HIVE-6598.patch Added plugin's configuration. Importing the project into eclipse as maven project have some issues Key: HIVE-6598 URL: https://issues.apache.org/jira/browse/HIVE-6598 Project: Hive Issue Type: Bug Environment: Windows 8 ,Eclipse Kepler and Maven 3.1.1 Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-6598.patch Importing the project into eclipse as maven project throwing these problems. Plugin execution not covered by lifecycle configuration: org.apache.maven.plugins:maven-antrun-plugin:1.7:run (execution: setup-test-dirs, phase: process-test-resources) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6598) Importing the project into eclipse as maven project have some issues
[ https://issues.apache.org/jira/browse/HIVE-6598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-6598: --- Fix Version/s: 0.13.0 Affects Version/s: 0.13.0 Status: Patch Available (was: Open) Importing the project into eclipse as maven project have some issues Key: HIVE-6598 URL: https://issues.apache.org/jira/browse/HIVE-6598 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Environment: Windows 8 ,Eclipse Kepler and Maven 3.1.1 Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Fix For: 0.13.0 Attachments: HIVE-6598.patch Importing the project into eclipse as maven project throwing these problems. Plugin execution not covered by lifecycle configuration: org.apache.maven.plugins:maven-antrun-plugin:1.7:run (execution: setup-test-dirs, phase: process-test-resources) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6486) Support secure Subject.doAs() in HiveServer2 JDBC client.
[ https://issues.apache.org/jira/browse/HIVE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925486#comment-13925486 ] Thejas M Nair commented on HIVE-6486: - [~rhbutani] I think it will be very valuable to have this patch committed to 0.13 as well. Support secure Subject.doAs() in HiveServer2 JDBC client. - Key: HIVE-6486 URL: https://issues.apache.org/jira/browse/HIVE-6486 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Shivaraju Gowda Assignee: Shivaraju Gowda Fix For: 0.13.0 Attachments: HIVE-6486.1.patch, HIVE-6486.2.patch, HIVE-6486.3.patch, Hive_011_Support-Subject_doAS.patch, TestHive_SujectDoAs.java HIVE-5155 addresses the problem of kerberos authentication in multi-user middleware server using proxy user. In this mode the principal used by the middle ware server has privileges to impersonate selected users in Hive/Hadoop. This enhancement is to support Subject.doAs() authentication in Hive JDBC layer so that the end users Kerberos Subject is passed through in the middle ware server. With this improvement there won't be any additional setup in the server to grant proxy privileges to some users and there won't be need to specify a proxy user in the JDBC client. This version should also be more secure since it won't require principals with the privileges to impersonate other users in Hive/Hadoop setup. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (HIVE-6485) Downgrade to httpclient-4.2.5 in JDBC from httpclient-4.3.2
[ https://issues.apache.org/jira/browse/HIVE-6485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair reopened HIVE-6485: - Lets mark it as fixed only after HIVE-4764 goes in. (alternatively, maybe mark it as duplicate ). Downgrade to httpclient-4.2.5 in JDBC from httpclient-4.3.2 --- Key: HIVE-6485 URL: https://issues.apache.org/jira/browse/HIVE-6485 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-6485.1.patch Had upgraded to the new version while adding SSL over Http mode support for HiveServer2. But that conflicts with httpclient-4.2.5 which is in hadoop classpath. I don't have a good reason to use httpclient-4.3.2, so it's better to match hadoop. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-4629) HS2 should support an API to retrieve query logs
[ https://issues.apache.org/jira/browse/HIVE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925491#comment-13925491 ] Carl Steinbach commented on HIVE-4629: -- Does the new version of the patch address any of the API design issues I mentioned earlier? HS2 should support an API to retrieve query logs Key: HIVE-4629 URL: https://issues.apache.org/jira/browse/HIVE-4629 Project: Hive Issue Type: Sub-task Components: HiveServer2 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Attachments: HIVE-4629-no_thrift.1.patch, HIVE-4629.1.patch, HIVE-4629.2.patch HiveServer2 should support an API to retrieve query logs. This is particularly relevant because HiveServer2 supports async execution but doesn't provide a way to report progress. Providing an API to retrieve query logs will help report progress to the client. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6486) Support secure Subject.doAs() in HiveServer2 JDBC client.
[ https://issues.apache.org/jira/browse/HIVE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925504#comment-13925504 ] Lefty Leverenz commented on HIVE-6486: -- [~shivshi], the release note looks good to me. I'll make a few minor edits when I put it in the JDBC section of the HS2 Clients wiki (which will probably happen tomorrow), then you can review to make sure your meaning didn't get changed. Support secure Subject.doAs() in HiveServer2 JDBC client. - Key: HIVE-6486 URL: https://issues.apache.org/jira/browse/HIVE-6486 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Shivaraju Gowda Assignee: Shivaraju Gowda Fix For: 0.13.0 Attachments: HIVE-6486.1.patch, HIVE-6486.2.patch, HIVE-6486.3.patch, Hive_011_Support-Subject_doAS.patch, TestHive_SujectDoAs.java HIVE-5155 addresses the problem of kerberos authentication in multi-user middleware server using proxy user. In this mode the principal used by the middle ware server has privileges to impersonate selected users in Hive/Hadoop. This enhancement is to support Subject.doAs() authentication in Hive JDBC layer so that the end users Kerberos Subject is passed through in the middle ware server. With this improvement there won't be any additional setup in the server to grant proxy privileges to some users and there won't be need to specify a proxy user in the JDBC client. This version should also be more secure since it won't require principals with the privileges to impersonate other users in Hive/Hadoop setup. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6222) Make Vector Group By operator abandon grouping if too many distinct keys
[
https://issues.apache.org/jira/browse/HIVE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925526#comment-13925526
]
Hive QA commented on HIVE-6222:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633560/HIVE-6222.1.patch
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1687/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1687/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1687/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target
shims/0.20S/target shims/0.23/target shims/aggregator/target
shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target
itests/target itests/hcatalog-unit/target itests/test-serde/target
itests/qtest/target itests/hive-unit/target itests/custom-serde/target
itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target
hcatalog/server-extensions/target hcatalog/core/target
hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target
hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen
contrib/target service/target serde/target beeline/target odbc/target
cli/target ql/dependency-reduced-pom.xml ql/target
+ svn update
Ujdbc/src/java/org/apache/hive/jdbc/HiveConnection.java
Uservice/src/java/org/apache/hive/service/auth/KerberosSaslHelper.java
A
service/src/java/org/apache/hive/service/auth/TSubjectAssumingTransport.java
Fetching external item into 'hcatalog/src/test/e2e/harness'
Updated external to revision 1575861.
Updated to revision 1575861.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633560
Make Vector Group By operator abandon grouping if too many distinct keys
Key: HIVE-6222
URL: https://issues.apache.org/jira/browse/HIVE-6222
Project: Hive
Issue Type: Sub-task
Reporter: Remus Rusanu
Assignee: Remus Rusanu
Priority: Minor
Attachments: HIVE-6222.1.patch
Row mode GBY is becoming a pass-through if not enough aggregation occurs on
the map side, relying on the shuffle+reduce side to do the work. Have VGBY do
the same.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6222) Make Vector Group By operator abandon grouping if too many distinct keys
[ https://issues.apache.org/jira/browse/HIVE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925529#comment-13925529 ] Remus Rusanu commented on HIVE-6222: Conflict with HIVE-6531, I'll upload a new patch Make Vector Group By operator abandon grouping if too many distinct keys Key: HIVE-6222 URL: https://issues.apache.org/jira/browse/HIVE-6222 Project: Hive Issue Type: Sub-task Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Minor Attachments: HIVE-6222.1.patch Row mode GBY is becoming a pass-through if not enough aggregation occurs on the map side, relying on the shuffle+reduce side to do the work. Have VGBY do the same. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925532#comment-13925532 ] Thejas M Nair commented on HIVE-5155: - [~prasadm] Can you please check if the test failures are caused by the patch ? Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, HIVE-5155.4.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6486) Support secure Subject.doAs() in HiveServer2 JDBC client.
[ https://issues.apache.org/jira/browse/HIVE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925534#comment-13925534 ] Thejas M Nair commented on HIVE-6486: - I think this should go under JDBC Client Setup for a Secure Cluster in https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients . This feature not something most end users would want to learn about. This is something users who are implementing a middleware system, where the middleware system needs to run the query as the user running the client. HIVE-5155 is another feature that middleware systems would be interested in. Mabye we should create a sub section under JDBC Client Setup for a Secure Cluster like Using jdbc from custom middleware application. Support secure Subject.doAs() in HiveServer2 JDBC client. - Key: HIVE-6486 URL: https://issues.apache.org/jira/browse/HIVE-6486 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Shivaraju Gowda Assignee: Shivaraju Gowda Fix For: 0.13.0 Attachments: HIVE-6486.1.patch, HIVE-6486.2.patch, HIVE-6486.3.patch, Hive_011_Support-Subject_doAS.patch, TestHive_SujectDoAs.java HIVE-5155 addresses the problem of kerberos authentication in multi-user middleware server using proxy user. In this mode the principal used by the middle ware server has privileges to impersonate selected users in Hive/Hadoop. This enhancement is to support Subject.doAs() authentication in Hive JDBC layer so that the end users Kerberos Subject is passed through in the middle ware server. With this improvement there won't be any additional setup in the server to grant proxy privileges to some users and there won't be need to specify a proxy user in the JDBC client. This version should also be more secure since it won't require principals with the privileges to impersonate other users in Hive/Hadoop setup. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6599) Document new jdbc url connection parameters for HiveServer2
Vaibhav Gumashta created HIVE-6599: -- Summary: Document new jdbc url connection parameters for HiveServer2 Key: HIVE-6599 URL: https://issues.apache.org/jira/browse/HIVE-6599 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Need to document the parameters for http mode, ssl mode and secure proxy access. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6486) Support secure Subject.doAs() in HiveServer2 JDBC client.
[ https://issues.apache.org/jira/browse/HIVE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925536#comment-13925536 ] Thejas M Nair commented on HIVE-6486: - I just meant to give out some more context on the change, and suggest possible ways of doing it. [~leftylev] , please make the decisions on how to present it using your doc expertise ! :) Support secure Subject.doAs() in HiveServer2 JDBC client. - Key: HIVE-6486 URL: https://issues.apache.org/jira/browse/HIVE-6486 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Shivaraju Gowda Assignee: Shivaraju Gowda Fix For: 0.13.0 Attachments: HIVE-6486.1.patch, HIVE-6486.2.patch, HIVE-6486.3.patch, Hive_011_Support-Subject_doAS.patch, TestHive_SujectDoAs.java HIVE-5155 addresses the problem of kerberos authentication in multi-user middleware server using proxy user. In this mode the principal used by the middle ware server has privileges to impersonate selected users in Hive/Hadoop. This enhancement is to support Subject.doAs() authentication in Hive JDBC layer so that the end users Kerberos Subject is passed through in the middle ware server. With this improvement there won't be any additional setup in the server to grant proxy privileges to some users and there won't be need to specify a proxy user in the JDBC client. This version should also be more secure since it won't require principals with the privileges to impersonate other users in Hive/Hadoop setup. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6222) Make Vector Group By operator abandon grouping if too many distinct keys
[ https://issues.apache.org/jira/browse/HIVE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-6222: --- Attachment: HIVE-6222.2.patch Rebased after HIVE-6531 Make Vector Group By operator abandon grouping if too many distinct keys Key: HIVE-6222 URL: https://issues.apache.org/jira/browse/HIVE-6222 Project: Hive Issue Type: Sub-task Components: Query Processor Affects Versions: 0.13.0 Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Minor Labels: vectorization Attachments: HIVE-6222.1.patch, HIVE-6222.2.patch Row mode GBY is becoming a pass-through if not enough aggregation occurs on the map side, relying on the shuffle+reduce side to do the work. Have VGBY do the same. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6222) Make Vector Group By operator abandon grouping if too many distinct keys
[ https://issues.apache.org/jira/browse/HIVE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-6222: --- Status: Patch Available (was: Open) Make Vector Group By operator abandon grouping if too many distinct keys Key: HIVE-6222 URL: https://issues.apache.org/jira/browse/HIVE-6222 Project: Hive Issue Type: Sub-task Components: Query Processor Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Minor Attachments: HIVE-6222.1.patch, HIVE-6222.2.patch Row mode GBY is becoming a pass-through if not enough aggregation occurs on the map side, relying on the shuffle+reduce side to do the work. Have VGBY do the same. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6222) Make Vector Group By operator abandon grouping if too many distinct keys
[ https://issues.apache.org/jira/browse/HIVE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-6222: --- Status: Open (was: Patch Available) Make Vector Group By operator abandon grouping if too many distinct keys Key: HIVE-6222 URL: https://issues.apache.org/jira/browse/HIVE-6222 Project: Hive Issue Type: Sub-task Components: Query Processor Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Minor Labels: vectorization Attachments: HIVE-6222.1.patch, HIVE-6222.2.patch Row mode GBY is becoming a pass-through if not enough aggregation occurs on the map side, relying on the shuffle+reduce side to do the work. Have VGBY do the same. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6222) Make Vector Group By operator abandon grouping if too many distinct keys
[ https://issues.apache.org/jira/browse/HIVE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-6222: --- Affects Version/s: 0.13.0 Make Vector Group By operator abandon grouping if too many distinct keys Key: HIVE-6222 URL: https://issues.apache.org/jira/browse/HIVE-6222 Project: Hive Issue Type: Sub-task Components: Query Processor Affects Versions: 0.13.0 Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Minor Labels: vectorization Attachments: HIVE-6222.1.patch, HIVE-6222.2.patch Row mode GBY is becoming a pass-through if not enough aggregation occurs on the map side, relying on the shuffle+reduce side to do the work. Have VGBY do the same. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6222) Make Vector Group By operator abandon grouping if too many distinct keys
[ https://issues.apache.org/jira/browse/HIVE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-6222: --- Component/s: Query Processor Make Vector Group By operator abandon grouping if too many distinct keys Key: HIVE-6222 URL: https://issues.apache.org/jira/browse/HIVE-6222 Project: Hive Issue Type: Sub-task Components: Query Processor Affects Versions: 0.13.0 Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Minor Labels: vectorization Attachments: HIVE-6222.1.patch, HIVE-6222.2.patch Row mode GBY is becoming a pass-through if not enough aggregation occurs on the map side, relying on the shuffle+reduce side to do the work. Have VGBY do the same. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6222) Make Vector Group By operator abandon grouping if too many distinct keys
[ https://issues.apache.org/jira/browse/HIVE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-6222: --- Labels: vectorization (was: ) Make Vector Group By operator abandon grouping if too many distinct keys Key: HIVE-6222 URL: https://issues.apache.org/jira/browse/HIVE-6222 Project: Hive Issue Type: Sub-task Components: Query Processor Affects Versions: 0.13.0 Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Minor Labels: vectorization Attachments: HIVE-6222.1.patch, HIVE-6222.2.patch Row mode GBY is becoming a pass-through if not enough aggregation occurs on the map side, relying on the shuffle+reduce side to do the work. Have VGBY do the same. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6503) document pluggable authentication modules (PAM) in template config, wiki
[ https://issues.apache.org/jira/browse/HIVE-6503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-6503: --- Component/s: HiveServer2 document pluggable authentication modules (PAM) in template config, wiki Key: HIVE-6503 URL: https://issues.apache.org/jira/browse/HIVE-6503 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Vaibhav Gumashta Priority: Blocker Fix For: 0.13.0 HIVE-6466 adds support for PAM as a supported value for hive.server2.authentication. It also adds a config parameter hive.server2.authentication.pam.services. The default template file needs to be updated to document these. The wiki docs should also document the support for pluggable authentication modules. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6599) Document new jdbc url connection parameters for HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-6599: --- Priority: Critical (was: Major) Document new jdbc url connection parameters for HiveServer2 --- Key: HIVE-6599 URL: https://issues.apache.org/jira/browse/HIVE-6599 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Priority: Critical Fix For: 0.13.0 Need to document the parameters for http mode, ssl mode and secure proxy access. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6600) Add Remus to Hive people list on credits page
Remus Rusanu created HIVE-6600: -- Summary: Add Remus to Hive people list on credits page Key: HIVE-6600 URL: https://issues.apache.org/jira/browse/HIVE-6600 Project: Hive Issue Type: Task Components: Website Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Trivial -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6597) WebHCat E2E tests doAsTests_6 and doAsTests_7 need to be updated
[
https://issues.apache.org/jira/browse/HIVE-6597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925588#comment-13925588
]
Hive QA commented on HIVE-6597:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633581/HIVE-6597.patch
{color:green}SUCCESS:{color} +1 5374 tests passed
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1689/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1689/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633581
WebHCat E2E tests doAsTests_6 and doAsTests_7 need to be updated
Key: HIVE-6597
URL: https://issues.apache.org/jira/browse/HIVE-6597
Project: Hive
Issue Type: Bug
Components: Tests, WebHCat
Affects Versions: 0.13.0
Reporter: Deepesh Khandelwal
Assignee: Deepesh Khandelwal
Fix For: 0.13.0
Attachments: HIVE-6597.patch
Currently the following WebHCat doAsTests need to be fixed:
In doAsTests_6 REST request url needs to be updated and corresponding
expected output to reflect the correct intent.
doAsTests_7 fails because of the strict error message checking.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-4764) Support Kerberos HTTP authentication for HiveServer2 running in http mode
[
https://issues.apache.org/jira/browse/HIVE-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925590#comment-13925590
]
Hive QA commented on HIVE-4764:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633588/HIVE-4764.4.patch
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1690/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1690/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1690/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'hcatalog/src/test/e2e/templeton/tests/doas.conf'
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target
shims/0.20S/target shims/0.23/target shims/aggregator/target
shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target
itests/target itests/hcatalog-unit/target itests/test-serde/target
itests/qtest/target itests/hive-unit/target itests/custom-serde/target
itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target
hcatalog/server-extensions/target hcatalog/core/target
hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target
hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen
contrib/target service/target serde/target beeline/target odbc/target
cli/target ql/dependency-reduced-pom.xml ql/target
+ svn update
Fetching external item into 'hcatalog/src/test/e2e/harness'
External at revision 1575875.
At revision 1575875.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633588
Support Kerberos HTTP authentication for HiveServer2 running in http mode
-
Key: HIVE-4764
URL: https://issues.apache.org/jira/browse/HIVE-4764
Project: Hive
Issue Type: Sub-task
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Vaibhav Gumashta
Fix For: 0.13.0
Attachments: HIVE-4764.1.patch, HIVE-4764.2.patch, HIVE-4764.3.patch,
HIVE-4764.4.patch
Support Kerberos authentication for HiveServer2 running in http mode.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-4764) Support Kerberos HTTP authentication for HiveServer2 running in http mode
[ https://issues.apache.org/jira/browse/HIVE-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-4764: --- Status: Patch Available (was: Open) Support Kerberos HTTP authentication for HiveServer2 running in http mode - Key: HIVE-4764 URL: https://issues.apache.org/jira/browse/HIVE-4764 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-4764.1.patch, HIVE-4764.2.patch, HIVE-4764.3.patch, HIVE-4764.4.patch Support Kerberos authentication for HiveServer2 running in http mode. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-4764) Support Kerberos HTTP authentication for HiveServer2 running in http mode
[ https://issues.apache.org/jira/browse/HIVE-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-4764: --- Status: Open (was: Patch Available) Support Kerberos HTTP authentication for HiveServer2 running in http mode - Key: HIVE-4764 URL: https://issues.apache.org/jira/browse/HIVE-4764 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-4764.1.patch, HIVE-4764.2.patch, HIVE-4764.3.patch, HIVE-4764.4.patch Support Kerberos authentication for HiveServer2 running in http mode. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6558) HiveServer2 Plain SASL authentication broken after hadoop 2.3 upgrade
[ https://issues.apache.org/jira/browse/HIVE-6558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925635#comment-13925635 ] Thejas M Nair commented on HIVE-6558: - +1 HiveServer2 Plain SASL authentication broken after hadoop 2.3 upgrade - Key: HIVE-6558 URL: https://issues.apache.org/jira/browse/HIVE-6558 Project: Hive Issue Type: Bug Components: Authentication, HiveServer2 Affects Versions: 0.13.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Priority: Blocker Attachments: HIVE-6558.2.patch Java only includes Plain SASL client and not server. Hence HiveServer2 includes a Plain SASL server implementation. Now Hadoop has its own Plain SASL server [HADOOP-9020|https://issues.apache.org/jira/browse/HADOOP-9020] which is part of Hadoop 2.3 [release|http://hadoop.apache.org/docs/r2.3.0/hadoop-project-dist/hadoop-common/releasenotes.html]. The two servers use different Sasl callbacks and the servers are registered in java.security.Provider via static code. As a result the HiveServer2 instance could be using Hadoop's Plain SASL server which breaks the authentication. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18185: Support Kerberos HTTP authentication for HiveServer2 running in http mode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18185/ --- (Updated March 10, 2014, 10:52 a.m.) Review request for hive and Thejas Nair. Changes --- Rebased on trunk Bugs: HIVE-4764 https://issues.apache.org/jira/browse/HIVE-4764 Repository: hive-git Description --- Support Kerberos HTTP authentication for HiveServer2 running in http mode Diffs (updated) - itests/hive-unit/src/test/java/org/apache/hive/service/cli/thrift/TestThriftHttpCLIService.java 57fda94 jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java 607fc7a jdbc/src/java/org/apache/hive/jdbc/HttpBasicAuthInterceptor.java 66eba1b jdbc/src/java/org/apache/hive/jdbc/HttpKerberosRequestInterceptor.java PRE-CREATION pom.xml cb901ad service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java d8ba3aa service/src/java/org/apache/hive/service/auth/HttpAuthUtils.java PRE-CREATION service/src/java/org/apache/hive/service/auth/HttpAuthenticationException.java PRE-CREATION service/src/java/org/apache/hive/service/auth/HttpCLIServiceUGIProcessor.java PRE-CREATION service/src/java/org/apache/hive/service/cli/CLIService.java 2b1e712 service/src/java/org/apache/hive/service/cli/session/SessionManager.java cec3b04 service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java 6fbc847 service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 26bda5a service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java a6ff6ce service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java e77f043 shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java dc89de1 shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java e15ab4e shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 03f4e51 Diff: https://reviews.apache.org/r/18185/diff/ Testing --- Using beeline in a kerberos setup. Thanks, Vaibhav Gumashta
[jira] [Updated] (HIVE-4764) Support Kerberos HTTP authentication for HiveServer2 running in http mode
[ https://issues.apache.org/jira/browse/HIVE-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-4764: --- Attachment: HIVE-4764.4.patch Rebased on trunk Support Kerberos HTTP authentication for HiveServer2 running in http mode - Key: HIVE-4764 URL: https://issues.apache.org/jira/browse/HIVE-4764 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-4764.1.patch, HIVE-4764.2.patch, HIVE-4764.3.patch, HIVE-4764.4.patch Support Kerberos authentication for HiveServer2 running in http mode. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-4764) Support Kerberos HTTP authentication for HiveServer2 running in http mode
[ https://issues.apache.org/jira/browse/HIVE-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-4764: --- Status: Open (was: Patch Available) Support Kerberos HTTP authentication for HiveServer2 running in http mode - Key: HIVE-4764 URL: https://issues.apache.org/jira/browse/HIVE-4764 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-4764.1.patch, HIVE-4764.2.patch, HIVE-4764.3.patch, HIVE-4764.4.patch Support Kerberos authentication for HiveServer2 running in http mode. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-4764) Support Kerberos HTTP authentication for HiveServer2 running in http mode
[ https://issues.apache.org/jira/browse/HIVE-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-4764: --- Attachment: (was: HIVE-4764.4.patch) Support Kerberos HTTP authentication for HiveServer2 running in http mode - Key: HIVE-4764 URL: https://issues.apache.org/jira/browse/HIVE-4764 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-4764.1.patch, HIVE-4764.2.patch, HIVE-4764.3.patch, HIVE-4764.4.patch Support Kerberos authentication for HiveServer2 running in http mode. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-4764) Support Kerberos HTTP authentication for HiveServer2 running in http mode
[ https://issues.apache.org/jira/browse/HIVE-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-4764: --- Status: Patch Available (was: Open) Support Kerberos HTTP authentication for HiveServer2 running in http mode - Key: HIVE-4764 URL: https://issues.apache.org/jira/browse/HIVE-4764 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-4764.1.patch, HIVE-4764.2.patch, HIVE-4764.3.patch, HIVE-4764.4.patch Support Kerberos authentication for HiveServer2 running in http mode. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6147) Support avro data stored in HBase columns
[
https://issues.apache.org/jira/browse/HIVE-6147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925658#comment-13925658
]
Hive QA commented on HIVE-6147:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633602/HIVE-6147.5.patch.txt
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5381 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_hadoop20
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1691/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1691/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633602
Support avro data stored in HBase columns
-
Key: HIVE-6147
URL: https://issues.apache.org/jira/browse/HIVE-6147
Project: Hive
Issue Type: Bug
Components: HBase Handler
Affects Versions: 0.12.0
Reporter: Swarnim Kulkarni
Assignee: Swarnim Kulkarni
Attachments: HIVE-6147.1.patch.txt, HIVE-6147.2.patch.txt,
HIVE-6147.3.patch.txt, HIVE-6147.3.patch.txt, HIVE-6147.4.patch.txt,
HIVE-6147.5.patch.txt
Presently, the HBase Hive integration supports querying only primitive data
types in columns. It would be nice to be able to store and query Avro objects
in HBase columns by making them visible as structs to Hive. This will allow
Hive to perform ad hoc analysis of HBase data which can be deeply structured.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6466) Add support for pluggable authentication modules (PAM) in Hive
[ https://issues.apache.org/jira/browse/HIVE-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-6466: --- Description: More on PAM in these articles: http://www.tuxradar.com/content/how-pam-works https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Pluggable_Authentication_Modules.html Usage from JPAM api: http://jpam.sourceforge.net/JPamUserGuide.html#id.s7.1 *Release notes:* PAM support allows Hive to use existing PAM services for authentication. PAM requires the native [JPAM|http://sourceforge.net/projects/jpam/files/jpam/jpam-1.1/] library. To enable PAM usage: 1. Download the JPAM native library [JPAM|http://sourceforge.net/projects/jpam/files/jpam/jpam-1.1/] for the relevant architecture. 2. Unzip and copy libjpam.so to a directory (libjmap-directory) on the system. 3. Add the directory to the LD_LIBRARY_PATH environment variable like so: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:libjmap-directory 4. Set the hive.server2.authentication to PAM in hive-site.xml 5. Set the hive.server2.authentication.pam.services to a list of comma separated PAM services that will be used. Note that a file with the same name as the PAM service must exist in /etc/pam.d 6. Start HiveServer2 was: More on PAM in these articles: http://www.tuxradar.com/content/how-pam-works https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Pluggable_Authentication_Modules.html Usage from JPAM api: http://jpam.sourceforge.net/JPamUserGuide.html#id.s7.1 Add support for pluggable authentication modules (PAM) in Hive -- Key: HIVE-6466 URL: https://issues.apache.org/jira/browse/HIVE-6466 Project: Hive Issue Type: New Feature Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-6466.1.patch, HIVE-6466.2.patch More on PAM in these articles: http://www.tuxradar.com/content/how-pam-works https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Pluggable_Authentication_Modules.html Usage from JPAM api: http://jpam.sourceforge.net/JPamUserGuide.html#id.s7.1 *Release notes:* PAM support allows Hive to use existing PAM services for authentication. PAM requires the native [JPAM|http://sourceforge.net/projects/jpam/files/jpam/jpam-1.1/] library. To enable PAM usage: 1. Download the JPAM native library [JPAM|http://sourceforge.net/projects/jpam/files/jpam/jpam-1.1/] for the relevant architecture. 2. Unzip and copy libjpam.so to a directory (libjmap-directory) on the system. 3. Add the directory to the LD_LIBRARY_PATH environment variable like so: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:libjmap-directory 4. Set the hive.server2.authentication to PAM in hive-site.xml 5. Set the hive.server2.authentication.pam.services to a list of comma separated PAM services that will be used. Note that a file with the same name as the PAM service must exist in /etc/pam.d 6. Start HiveServer2 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6600) Add Remus to Hive people list on credits page
[ https://issues.apache.org/jira/browse/HIVE-6600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-6600: --- Attachment: HIVE-6600.1.patch NO PRECOMMIT TESTS Add Remus to Hive people list on credits page - Key: HIVE-6600 URL: https://issues.apache.org/jira/browse/HIVE-6600 Project: Hive Issue Type: Task Components: Website Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Trivial Attachments: HIVE-6600.1.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6503) document pluggable authentication modules (PAM) in template config, wiki
[ https://issues.apache.org/jira/browse/HIVE-6503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-6503: --- Attachment: HIVE-6503.1.patch document pluggable authentication modules (PAM) in template config, wiki Key: HIVE-6503 URL: https://issues.apache.org/jira/browse/HIVE-6503 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Vaibhav Gumashta Priority: Blocker Fix For: 0.13.0 Attachments: HIVE-6503.1.patch HIVE-6466 adds support for PAM as a supported value for hive.server2.authentication. It also adds a config parameter hive.server2.authentication.pam.services. The default template file needs to be updated to document these. The wiki docs should also document the support for pluggable authentication modules. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6503) document pluggable authentication modules (PAM) in template config, wiki
[ https://issues.apache.org/jira/browse/HIVE-6503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925668#comment-13925668 ] Vaibhav Gumashta commented on HIVE-6503: [~thejas] [~leftylev] I've added release notes to HIVE-6466 and attached a patch to modify hive-default. I don't have wiki privileges yet :( document pluggable authentication modules (PAM) in template config, wiki Key: HIVE-6503 URL: https://issues.apache.org/jira/browse/HIVE-6503 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Vaibhav Gumashta Priority: Blocker Fix For: 0.13.0 Attachments: HIVE-6503.1.patch HIVE-6466 adds support for PAM as a supported value for hive.server2.authentication. It also adds a config parameter hive.server2.authentication.pam.services. The default template file needs to be updated to document these. The wiki docs should also document the support for pluggable authentication modules. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6601) alter database commands should support schema synonym keyword
Thejas M Nair created HIVE-6601:
---
Summary: alter database commands should support schema synonym
keyword
Key: HIVE-6601
URL: https://issues.apache.org/jira/browse/HIVE-6601
Project: Hive
Issue Type: Bug
Reporter: Thejas M Nair
It should be possible to use alter schema as an alternative to alter
database. But the syntax is not currently supported.
{code}
alter schema db1 set owner user x;
NoViableAltException(215@[])
FAILED: ParseException line 1:6 cannot recognize input near 'schema' 'db1'
'set' in alter statement
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6440) sql std auth - add command to change owner of database
[ https://issues.apache.org/jira/browse/HIVE-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925673#comment-13925673 ] Thejas M Nair commented on HIVE-6440: - bq. But why is username in brackets – can it really be omitted? No, it can't be omitted. It should not be in brackets. bq. does this also work for alter schema? I just checked the code, the existing ALTER (DATABASE|SCHEMA) database_name SET DBPROPERTIES as well as this one don't work with schema keyword. Created HIVE-6601 to track that. Thanks for bringing up these issues! Please go ahead and update the wiki if you can. sql std auth - add command to change owner of database -- Key: HIVE-6440 URL: https://issues.apache.org/jira/browse/HIVE-6440 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Fix For: 0.13.0 Attachments: HIVE-6440.1.patch, HIVE-6440.2.patch, HIVE-6440.3.patch It should be possible to change the owner of a database once it is created. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6466) Add support for pluggable authentication modules (PAM) in Hive
[ https://issues.apache.org/jira/browse/HIVE-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6466: Description: More on PAM in these articles: http://www.tuxradar.com/content/how-pam-works https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Pluggable_Authentication_Modules.html Usage from JPAM api: http://jpam.sourceforge.net/JPamUserGuide.html#id.s7.1 was: More on PAM in these articles: http://www.tuxradar.com/content/how-pam-works https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Pluggable_Authentication_Modules.html Usage from JPAM api: http://jpam.sourceforge.net/JPamUserGuide.html#id.s7.1 *Release notes:* PAM support allows Hive to use existing PAM services for authentication. PAM requires the native [JPAM|http://sourceforge.net/projects/jpam/files/jpam/jpam-1.1/] library. To enable PAM usage: 1. Download the JPAM native library [JPAM|http://sourceforge.net/projects/jpam/files/jpam/jpam-1.1/] for the relevant architecture. 2. Unzip and copy libjpam.so to a directory (libjmap-directory) on the system. 3. Add the directory to the LD_LIBRARY_PATH environment variable like so: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:libjmap-directory 4. Set the hive.server2.authentication to PAM in hive-site.xml 5. Set the hive.server2.authentication.pam.services to a list of comma separated PAM services that will be used. Note that a file with the same name as the PAM service must exist in /etc/pam.d 6. Start HiveServer2 Add support for pluggable authentication modules (PAM) in Hive -- Key: HIVE-6466 URL: https://issues.apache.org/jira/browse/HIVE-6466 Project: Hive Issue Type: New Feature Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-6466.1.patch, HIVE-6466.2.patch More on PAM in these articles: http://www.tuxradar.com/content/how-pam-works https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Pluggable_Authentication_Modules.html Usage from JPAM api: http://jpam.sourceforge.net/JPamUserGuide.html#id.s7.1 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6466) Add support for pluggable authentication modules (PAM) in Hive
[ https://issues.apache.org/jira/browse/HIVE-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6466: Release Note: *Release notes:* PAM support allows Hive to use existing PAM services for authentication. PAM requires the native [JPAM|http://sourceforge.net/projects/jpam/files/jpam/jpam-1.1/] library. To enable PAM usage: 1. Download the JPAM native library [JPAM|http://sourceforge.net/projects/jpam/files/jpam/jpam-1.1/] for the relevant architecture. 2. Unzip and copy libjpam.so to a directory (libjmap-directory) on the system. 3. Add the directory to the LD_LIBRARY_PATH environment variable like so: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:libjmap-directory 4. Set the hive.server2.authentication to PAM in hive-site.xml 5. Set the hive.server2.authentication.pam.services to a list of comma separated PAM services that will be used. Note that a file with the same name as the PAM service must exist in /etc/pam.d 6. Start HiveServer2 Add support for pluggable authentication modules (PAM) in Hive -- Key: HIVE-6466 URL: https://issues.apache.org/jira/browse/HIVE-6466 Project: Hive Issue Type: New Feature Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-6466.1.patch, HIVE-6466.2.patch More on PAM in these articles: http://www.tuxradar.com/content/how-pam-works https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Pluggable_Authentication_Modules.html Usage from JPAM api: http://jpam.sourceforge.net/JPamUserGuide.html#id.s7.1 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6109) Support customized location for EXTERNAL tables created by Dynamic Partitioning
[ https://issues.apache.org/jira/browse/HIVE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925683#comment-13925683 ] Satish Mittal commented on HIVE-6109: - [~leftylev], can you please update the HCatalog wikidoc section on 'external tables' in 'dynamic partitioning' with release notes of this ticket? Support customized location for EXTERNAL tables created by Dynamic Partitioning --- Key: HIVE-6109 URL: https://issues.apache.org/jira/browse/HIVE-6109 Project: Hive Issue Type: Improvement Components: HCatalog Reporter: Satish Mittal Assignee: Satish Mittal Fix For: 0.13.0 Attachments: HIVE-6109.1.patch.txt, HIVE-6109.2.patch.txt, HIVE-6109.3.patch.txt, HIVE-6109.pdf Currently when dynamic partitions are created by HCatalog, the underlying directories for the partitions are created in a fixed 'Hive-style' format, i.e. root_dir/key1=value1/key2=value2/ and so on. However in case of external table, user should be able to control the format of directories created for dynamic partitions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6068) HiveServer2 client on windows does not handle the non-ascii characters properly
[
https://issues.apache.org/jira/browse/HIVE-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-6068:
---
Attachment: HIVE-6068.1.patch
cc [~thejas]
HiveServer2 client on windows does not handle the non-ascii characters
properly
---
Key: HIVE-6068
URL: https://issues.apache.org/jira/browse/HIVE-6068
Project: Hive
Issue Type: Bug
Components: HiveServer2, JDBC
Affects Versions: 0.13.0
Environment: Windows
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
Fix For: 0.13.0
Attachments: HIVE-6068.1.patch
When running a select query against a table which contains rows with
non-ascii characters HiveServer2 Beeline client returns them wrong. Example:
{noformat}
738;Garçu, Le (1995);Drama
741;Ghost in the Shell (Kôkaku kidôtai) (1995);Animation|Sci-Fi
{noformat}
come out from a HiveServer2 beeline client as:
{noformat}
'738' 'Gar?u, Le (1995)' 'Drama'
'741' 'Ghost in the Shell (K?kaku kid?tai) (1995)''Animation|Sci-Fi'
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6600) Add Remus to Hive people list on credits page
[ https://issues.apache.org/jira/browse/HIVE-6600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925686#comment-13925686 ] Thejas M Nair commented on HIVE-6600: - +1 Add Remus to Hive people list on credits page - Key: HIVE-6600 URL: https://issues.apache.org/jira/browse/HIVE-6600 Project: Hive Issue Type: Task Components: Website Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Trivial Attachments: HIVE-6600.1.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5958) SQL std auth - authorize statements that work with paths
[ https://issues.apache.org/jira/browse/HIVE-5958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925684#comment-13925684 ] Thejas M Nair commented on HIVE-5958: - I will document this along with the parent jira. SQL std auth - authorize statements that work with paths Key: HIVE-5958 URL: https://issues.apache.org/jira/browse/HIVE-5958 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Fix For: 0.13.0 Attachments: HIVE-5958.1.patch, HIVE-5958.2.patch, HIVE-5958.3.patch, HIVE-5958.4.patch, HIVE-5958.5.patch, HIVE-5958.6.patch, HIVE-5958.7.patch Original Estimate: 72h Remaining Estimate: 72h Statement such as create table, alter table that specify an path uri should be allowed under the new authorization scheme only if URI(Path) specified has permissions including read/write and ownership of the file/dir and its children. Also, fix issue of database not getting set as output for create-table. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6068) HiveServer2 client on windows does not handle the non-ascii characters properly
[
https://issues.apache.org/jira/browse/HIVE-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-6068:
---
Status: Patch Available (was: Open)
HiveServer2 client on windows does not handle the non-ascii characters
properly
---
Key: HIVE-6068
URL: https://issues.apache.org/jira/browse/HIVE-6068
Project: Hive
Issue Type: Bug
Components: HiveServer2, JDBC
Affects Versions: 0.13.0
Environment: Windows
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
Fix For: 0.13.0
Attachments: HIVE-6068.1.patch
When running a select query against a table which contains rows with
non-ascii characters HiveServer2 Beeline client returns them wrong. Example:
{noformat}
738;Garçu, Le (1995);Drama
741;Ghost in the Shell (Kôkaku kidôtai) (1995);Animation|Sci-Fi
{noformat}
come out from a HiveServer2 beeline client as:
{noformat}
'738' 'Gar?u, Le (1995)' 'Drama'
'741' 'Ghost in the Shell (K?kaku kid?tai) (1995)''Animation|Sci-Fi'
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-5931) SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5931: Summary: SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE (was: SQL std auth - add metastore get_role_participants api - to support DESCRIBE ROLE) SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE -- Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Attachments: HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h This is necessary for DESCRIBE ROLE role statement. This will list all users and roles that participate in a role. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5931) SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5931: Attachment: (was: HIVE-5931.nothrifgen.1.patch) SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE -- Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Attachments: HIVE-5931.1.patch, HIVE-5931.nothrifgen.1.patch, HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h This is necessary for DESCRIBE ROLE role statement. This will list all users and roles that participate in a role. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5931) SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5931: Attachment: HIVE-5931.nothrifgen.1.patch SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE -- Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Attachments: HIVE-5931.1.patch, HIVE-5931.nothrifgen.1.patch, HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h This is necessary for DESCRIBE ROLE role statement. This will list all users and roles that participate in a role. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5931) SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5931: Attachment: HIVE-5931.nothrifgen.1.patch SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE -- Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Attachments: HIVE-5931.1.patch, HIVE-5931.nothrifgen.1.patch, HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h This is necessary for DESCRIBE ROLE role statement. This will list all users and roles that participate in a role. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5931) SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5931: Attachment: HIVE-5931.1.patch SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE -- Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Attachments: HIVE-5931.1.patch, HIVE-5931.nothrifgen.1.patch, HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h This is necessary for DESCRIBE ROLE role statement. This will list all users and roles that participate in a role. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5931) SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925702#comment-13925702 ] Thejas M Nair commented on HIVE-5931: - Instead of describe-role, I have added support for the command: SHOW ROLE PRINCIPALS role_name; Attaching a patch file without the thrift generated files as well, for ease of review ( HIVE-5931.nothrifgen.1.patch is no thrift-gen version of HIVE-5931.1.patch ) SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE -- Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Attachments: HIVE-5931.1.patch, HIVE-5931.nothrifgen.1.patch, HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h This is necessary for DESCRIBE ROLE role statement. This will list all users and roles that participate in a role. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6602) Multi-user HiveServer2 throws error
Vaibhav Gumashta created HIVE-6602: -- Summary: Multi-user HiveServer2 throws error Key: HIVE-6602 URL: https://issues.apache.org/jira/browse/HIVE-6602 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Error thrown: Error while processing statement: FAILED: RuntimeException org.apache.hadoop.security.AccessControlException: Permission denied: user=user_1, access=WRITE, inode=/tmp/hive-hive:hdfs:drwxr-xr-x For hive query execution, a scratch directory specified by hive.exec.scratchdir is created with default permission 700. In HiveServer2, during the CLIService startup, we check for the presence of scratch directories (local + dfs) and if they don't exist, create them with permission 777. However, we should also change the permission from the default 700 to 777 in case the dfs scratch directory already exists. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6602) Multi-user HiveServer2 throws error
[ https://issues.apache.org/jira/browse/HIVE-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-6602: --- Status: Patch Available (was: Open) Multi-user HiveServer2 throws error --- Key: HIVE-6602 URL: https://issues.apache.org/jira/browse/HIVE-6602 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-6602.1.patch Error thrown: Error while processing statement: FAILED: RuntimeException org.apache.hadoop.security.AccessControlException: Permission denied: user=user_1, access=WRITE, inode=/tmp/hive-hive:hdfs:drwxr-xr-x For hive query execution, a scratch directory specified by hive.exec.scratchdir is created with default permission 700. In HiveServer2, during the CLIService startup, we check for the presence of scratch directories (local + dfs) and if they don't exist, create them with permission 777. However, we should also change the permission from the default 700 to 777 in case the dfs scratch directory already exists. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6602) Multi-user HiveServer2 throws error
[ https://issues.apache.org/jira/browse/HIVE-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-6602: --- Attachment: HIVE-6602.1.patch cc [~thejas] Multi-user HiveServer2 throws error --- Key: HIVE-6602 URL: https://issues.apache.org/jira/browse/HIVE-6602 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-6602.1.patch Error thrown: Error while processing statement: FAILED: RuntimeException org.apache.hadoop.security.AccessControlException: Permission denied: user=user_1, access=WRITE, inode=/tmp/hive-hive:hdfs:drwxr-xr-x For hive query execution, a scratch directory specified by hive.exec.scratchdir is created with default permission 700. In HiveServer2, during the CLIService startup, we check for the presence of scratch directories (local + dfs) and if they don't exist, create them with permission 777. However, we should also change the permission from the default 700 to 777 in case the dfs scratch directory already exists. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5931) SQL std auth - add metastore get_principals_in_role api, support SHOW ROLE PRINCIPALS
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5931: Summary: SQL std auth - add metastore get_principals_in_role api, support SHOW ROLE PRINCIPALS (was: SQL std auth - add metastore get_principals_in_role api, support DESCRIBE ROLE) SQL std auth - add metastore get_principals_in_role api, support SHOW ROLE PRINCIPALS - Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Attachments: HIVE-5931.1.patch, HIVE-5931.nothrifgen.1.patch, HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h This is necessary for DESCRIBE ROLE role statement. This will list all users and roles that participate in a role. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6568) Vectorized cast of decimal to string and timestamp produces incorrect result.
[
https://issues.apache.org/jira/browse/HIVE-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925707#comment-13925707
]
Hive QA commented on HIVE-6568:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633603/HIVE-6568.2.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5375 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorTypeCasts.testCastDecimalToString
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testExecuteStatementAsync
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1692/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1692/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633603
Vectorized cast of decimal to string and timestamp produces incorrect result.
-
Key: HIVE-6568
URL: https://issues.apache.org/jira/browse/HIVE-6568
Project: Hive
Issue Type: Bug
Components: Vectorization
Affects Versions: 0.13.0
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey
Attachments: HIVE-6568.1.patch, HIVE-6568.2.patch
A decimal value 1.23 with scale 5 is represented in string as 1.23000. This
behavior is different from HiveDecimal behavior.
The difference in cast to timestamp is due to more aggressive rounding in
vectorized expression.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-5931) SQL std auth - add metastore get_principals_in_role api, support SHOW ROLE PRINCIPALS
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5931: Description: Support command for listing all members of a role. A new metastore api call also needs to be added for this. was: This is necessary for DESCRIBE ROLE role statement. This will list all users and roles that participate in a role. SQL std auth - add metastore get_principals_in_role api, support SHOW ROLE PRINCIPALS - Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Attachments: HIVE-5931.1.patch, HIVE-5931.nothrifgen.1.patch, HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h Support command for listing all members of a role. A new metastore api call also needs to be added for this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5931) SQL std auth - add metastore get_principals_in_role api, support SHOW ROLE PRINCIPALS
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5931: Status: Patch Available (was: Open) SQL std auth - add metastore get_principals_in_role api, support SHOW ROLE PRINCIPALS - Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Attachments: HIVE-5931.1.patch, HIVE-5931.nothrifgen.1.patch, HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h Support command for listing all members of a role. A new metastore api call also needs to be added for this. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Proposal to switch to pull requests
Agree with Carl about git merge commits, they make the changes hard to
follow. But it should be OK, if there is no way to disable it in the
main git repo, it is a small set of active committers, we can make a
policy and expect people to follow it.
But we should certainly disable 'git push -f' (and anything as distruptive).
On Sun, Mar 9, 2014 at 3:51 PM, Carl Steinbach cwsteinb...@gmail.com wrote:
I'm +1 on switching to git, but only if we can find a way to disable merge
commits to trunk and feature branches. I'm -1 on switching to Github since,
as far as I know, it only supports merge based workflows.
On Sun, Mar 9, 2014 at 12:25 PM, Edward Capriolo edlinuxg...@gmail.comwrote:
I do not think we want Pull Requests coming at us. Better way is let
someone open a git branch for the changes, then we review and merge the
branch.
On Sat, Mar 8, 2014 at 4:25 PM, Brock Noland br...@cloudera.com wrote:
In my read of the Apache git - github integration blog post we cannot use
pull requests as patches. Just that we'll be notified of them and could
perhaps use them as code review.
One additional item I think we should investigate is disabling merge
commits on trunk and feature branches.
On Mar 7, 2014 7:57 PM, Edward Capriolo edlinuxg...@gmail.com wrote:
We need to keep patches in Jira I feel. We have gotten better on the
documentation front but having a patch in the jira is critical I feel.
We
must at least have a perma link to the changes.
On Fri, Mar 7, 2014 at 8:40 PM, Sergey Shelukhin
ser...@hortonworks.com
wrote:
+1 to git!
On Fri, Mar 7, 2014 at 12:46 PM, Xuefu Zhang xzh...@cloudera.com
wrote:
Switching to git from svn seems to be a proposal slightly different
from
that of switching to pull request from the head of the thread.
Personally
I'm +1 to git, but I think patches are very portable and widely
adopted
in
Hadoop ecosystem and we should keep the practice. Thus, +1 to that
also.
--Xuefu
On Fri, Mar 7, 2014 at 12:27 PM, Gunther Hagleitner
ghagleit...@hortonworks.com wrote:
Once Prasad's loop finishes I'd like to add my +1 too.
On Fri, Mar 7, 2014 at 11:44 AM, Vaibhav Gumashta
vgumas...@hortonworks.com
wrote:
+1 for moving to git!
Thanks,
--Vaibhav
On Fri, Mar 7, 2014 at 9:46 AM, Prasad Mujumdar
pras...@cloudera.com
wrote:
while (true) {
+1
}
+1 // another, just in case ;)
thanks
Prasad
On Fri, Mar 7, 2014 at 6:47 AM, kulkarni.swar...@gmail.com
kulkarni.swar...@gmail.com wrote:
+1
On Fri, Mar 7, 2014 at 1:05 AM, Thejas Nair
the...@hortonworks.com
wrote:
Should we start with moving our primary source code
repository
from
svn to git ? I feel git is more powerful and easy to use
(once
you
go
past the learning curve!).
On Wed, Mar 5, 2014 at 7:39 AM, Brock Noland
br...@cloudera.com
wrote:
Personally I prefer the Github workflow, but I believe
there
have
been
some challenges with that since the source for apache
projects
must
be
stored in apache source control (git or svn).
Relevent:
https://blogs.apache.org/infra/entry/improved_integration_between_apache_and
On Wed, Mar 5, 2014 at 9:19 AM,
kulkarni.swar...@gmail.com
kulkarni.swar...@gmail.com wrote:
Hello,
Since we have a nice mirrored git repository for
hive[1],
any
specific
reason why we can't switch to doing pull requests
instead
of
patches?
IMHO
pull requests are awesome for peer review plus it is
also
very
easy
to
keep
track of JIRAs with open pull requests instead of
looking
for
JIRAs
in a
Patch Available state. Also since they get updated
automatically,
it
is
also very easy to see if a review comment made by a
reviewer
was
addressed
properly or not.
Thoughts?
Thanks,
[1] https://github.com/apache/hive
--
Swarnim
--
Apache MRUnit - Unit testing MapReduce -
http://mrunit.apache.org
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the
individual
or
entity
to
[jira] [Resolved] (HIVE-3033) Loading data from a file in hdfs to hive table is failing if we try to load the same file into the same table second time
[
https://issues.apache.org/jira/browse/HIVE-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinna Rao Lalam resolved HIVE-3033.
Resolution: Not A Problem
This is not an issue with current current code base..
Loading data from a file in hdfs to hive table is failing if we try to load
the same file into the same table second time
-
Key: HIVE-3033
URL: https://issues.apache.org/jira/browse/HIVE-3033
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.8.1, 0.9.0, 0.9.1
Reporter: Kristam Subba Swathi
Assignee: Chinna Rao Lalam
Attachments: HIVE-3033.patch
Steps to reproduce
-
1)create table in hive
create table emp(IP STRING,showtime double) ROW FORMAT DELIMITED FIELDS
TERMINATED BY '\040'
2)load data into the table
LOAD DATA INPATH '/hive/input/data2.txt' OVERWRITE INTO TABLE emp
LOAD DATA INPATH '/hive/input/data2.txt' OVERWRITE INTO TABLE emp
Loading the same file into the same table is failing
{noformat}
2012-05-11 19:28:54,415 DEBUG metadata.Hive (Hive.java:checkPaths(1937)) -
Successfully renamed
hdfs://10.18.40.25:54310/HiveNFT_testLoadDataShouldOverWriteIfSameFileAlreadyExistsInTableByGivingTheRooTPath/data2.txt
to
hdfs://10.18.40.25:54310/HiveNFT_testLoadDataShouldOverWriteIfSameFileAlreadyExistsInTableByGivingTheRooTPath/data2_copy_3.txt
2012-05-11 19:28:54,416 DEBUG ipc.Client (Client.java:sendParam(786)) - IPC
Client (32955489) connection to HOST-10-18-40-25/10.18.40.25:54310 from root
sending #5749
2012-05-11 19:28:54,416 DEBUG ipc.Client (Client.java:receiveResponse(821)) -
IPC Client (32955489) connection to HOST-10-18-40-25/10.18.40.25:54310 from
root got value #5749
2012-05-11 19:28:54,417 DEBUG ipc.RPC (WritableRpcEngine.java:invoke(197)) -
Call: getFileInfo 2
2012-05-11 19:28:54,417 DEBUG ipc.Client (Client.java:sendParam(786)) - IPC
Client (32955489) connection to HOST-10-18-40-25/10.18.40.25:54310 from root
sending #5750
2012-05-11 19:28:54,419 DEBUG ipc.Client (Client.java:receiveResponse(821)) -
IPC Client (32955489) connection to HOST-10-18-40-25/10.18.40.25:54310 from
root got value #5750
2012-05-11 19:28:54,419 DEBUG ipc.RPC (WritableRpcEngine.java:invoke(197)) -
Call: getListing 2
2012-05-11 19:28:54,420 ERROR exec.Task (SessionState.java:printError(380)) -
Failed with exception copyFiles: error while moving files!!!
org.apache.hadoop.hive.ql.metadata.HiveException: copyFiles: error while
moving files!!!
at org.apache.hadoop.hive.ql.metadata.Hive.copyFiles(Hive.java:1989)
at org.apache.hadoop.hive.ql.metadata.Table.copyFiles(Table.java:547)
at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:1283)
at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:234)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at
org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:191)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:629)
at
org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:617)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.FileNotFoundException: File
hdfs://10.18.40.25:54310/HiveNFT_testLoadDataShouldOverWriteIfSameFileAlreadyExistsInTableByGivingTheRooTPath/data2.txt
does not exist.
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:353)
at org.apache.hadoop.hive.ql.metadata.Hive.copyFiles(Hive.java:1979)
... 17 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Review Request 17566: Support to query avro data stored in HBase
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17566/ --- (Updated March 10, 2014, 2:17 p.m.) Review request for hive. Bugs: HIVE-6147 https://issues.apache.org/jira/browse/HIVE-6147 Repository: hive-git Description --- Support to query avro data stored in HBase. Diffs (updated) - hbase-handler/pom.xml 132af43 hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/Address.java PRE-CREATION hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/ContactInfo.java PRE-CREATION hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/Employee.java PRE-CREATION hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/EmployeeAvro.java PRE-CREATION hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/Gender.java PRE-CREATION hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/HomePhone.java PRE-CREATION hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/Magic.java PRE-CREATION hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/OfficePhone.java PRE-CREATION hbase-handler/src/if/avro/avro_test.avpr PRE-CREATION hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseCompositeKey.java 5008f15 hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java 2cd65cb hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDeHelper.java PRE-CREATION hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseCellMap.java cedef10 hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestAvroSchemaRetriever.java PRE-CREATION hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestCompositeKey.java 13c344b hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestHBaseSerDe.java 089a31a serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroGenericRecordWritable.java 8beffd7 serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroLazyObjectInspector.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroObjectInspectorException.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroObjectInspectorGenerator.java 46cdb4f serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSchemaRetriever.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 9d58d13 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java d9a38ba serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazySimpleSerDe.java 82c1263 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyStruct.java 8a1ea46 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUnion.java 9f6bc3f serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/LazyObjectInspectorFactory.java ed670b0 serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/LazySimpleStructObjectInspector.java 8a5386a serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java 9a226b3 Diff: https://reviews.apache.org/r/17566/diff/ Testing --- New unit tests added to demonstrate the functionality. Thanks, Swarnim Kulkarni
[jira] [Commented] (HIVE-4764) Support Kerberos HTTP authentication for HiveServer2 running in http mode
[ https://issues.apache.org/jira/browse/HIVE-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925753#comment-13925753 ] Thejas M Nair commented on HIVE-4764: - +1 Support Kerberos HTTP authentication for HiveServer2 running in http mode - Key: HIVE-4764 URL: https://issues.apache.org/jira/browse/HIVE-4764 Project: Hive Issue Type: Sub-task Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-4764.1.patch, HIVE-4764.2.patch, HIVE-4764.3.patch, HIVE-4764.4.patch Support Kerberos authentication for HiveServer2 running in http mode. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6147) Support avro data stored in HBase columns
[ https://issues.apache.org/jira/browse/HIVE-6147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925754#comment-13925754 ] Swarnim Kulkarni commented on HIVE-6147: [~xuefuz] As the previously failing tests now pass, I have updated the RB with the latest patch for review. Support avro data stored in HBase columns - Key: HIVE-6147 URL: https://issues.apache.org/jira/browse/HIVE-6147 Project: Hive Issue Type: Bug Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Attachments: HIVE-6147.1.patch.txt, HIVE-6147.2.patch.txt, HIVE-6147.3.patch.txt, HIVE-6147.3.patch.txt, HIVE-6147.4.patch.txt, HIVE-6147.5.patch.txt Presently, the HBase Hive integration supports querying only primitive data types in columns. It would be nice to be able to store and query Avro objects in HBase columns by making them visible as structs to Hive. This will allow Hive to perform ad hoc analysis of HBase data which can be deeply structured. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6598) Importing the project into eclipse as maven project have some issues
[
https://issues.apache.org/jira/browse/HIVE-6598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925762#comment-13925762
]
Hive QA commented on HIVE-6598:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633644/HIVE-6598.patch
{color:green}SUCCESS:{color} +1 5374 tests passed
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1695/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1695/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633644
Importing the project into eclipse as maven project have some issues
Key: HIVE-6598
URL: https://issues.apache.org/jira/browse/HIVE-6598
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Environment: Windows 8 ,Eclipse Kepler and Maven 3.1.1
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
Fix For: 0.13.0
Attachments: HIVE-6598.patch
Importing the project into eclipse as maven project throwing these problems.
Plugin execution not covered by lifecycle configuration:
org.apache.maven.plugins:maven-antrun-plugin:1.7:run (execution:
setup-test-dirs, phase: process-test-resources)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6359) beeline -f fails on scripts with tabs in them.
[
https://issues.apache.org/jira/browse/HIVE-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-6359:
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch committed to trunk and 0.13 branch (It was already in the list maintained
by Harish).
cc [~rhbutani]
beeline -f fails on scripts with tabs in them.
--
Key: HIVE-6359
URL: https://issues.apache.org/jira/browse/HIVE-6359
Project: Hive
Issue Type: Bug
Reporter: Carter Shanklin
Assignee: Navis
Priority: Minor
Fix For: 0.13.0
Attachments: HIVE-6359.1.patch.txt, HIVE-6359.2.patch.txt
NO PRECOMMIT TESTS
On a recent trunk build I used beeline -f on a script with tabs in it.
Beeline rather unhelpfully attempts to perform tab expansion on the tabs and
the query fails. Here's a screendump.
{code}
Connecting to jdbc:hive2://mymachine:1/mydb
Connected to: Apache Hive (version 0.13.0-SNAPSHOT)
Driver: Hive JDBC (version 0.13.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 0.13.0-SNAPSHOT by Apache Hive
0: jdbc:hive2://mymachine:1/mydb select i_brand_id as brand_id, i_brand
as brand,
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . ager_id=36
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_moy=12
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_year=2001
. . . . . . . . . . . . . . . . . . . . . . . and ss_sold_date
between '2001-12-01' and '2001-12-31'
. . . . . . . . . . . . . . . . . . . . . . . group by i_brand, i_brand_id
. . . . . . . . . . . . . . . . . . . . . . . order by ext_price desc,
brand_id
. . . . . . . . . . . . . . . . . . . . . . . limit 100 ;
Error: Error while compiling statement: FAILED: ParseException line 1:65
missing FROM at 'd_moy' near 'd' in from source (state=42000,code=4)
Closing: org.apache.hive.jdbc.HiveConnection
{code}
The same query works fine if I replace tabs with some spaces.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6359) beeline -f fails on scripts with tabs in them.
[
https://issues.apache.org/jira/browse/HIVE-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-6359:
Fix Version/s: 0.13.0
beeline -f fails on scripts with tabs in them.
--
Key: HIVE-6359
URL: https://issues.apache.org/jira/browse/HIVE-6359
Project: Hive
Issue Type: Bug
Reporter: Carter Shanklin
Assignee: Navis
Priority: Minor
Fix For: 0.13.0
Attachments: HIVE-6359.1.patch.txt, HIVE-6359.2.patch.txt
NO PRECOMMIT TESTS
On a recent trunk build I used beeline -f on a script with tabs in it.
Beeline rather unhelpfully attempts to perform tab expansion on the tabs and
the query fails. Here's a screendump.
{code}
Connecting to jdbc:hive2://mymachine:1/mydb
Connected to: Apache Hive (version 0.13.0-SNAPSHOT)
Driver: Hive JDBC (version 0.13.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 0.13.0-SNAPSHOT by Apache Hive
0: jdbc:hive2://mymachine:1/mydb select i_brand_id as brand_id, i_brand
as brand,
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . ager_id=36
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_moy=12
. . . . . . . . . . . . . . . . . . . . . . .
Display all 560 possibilities? (y or n)
. . . . . . . . . . . . . . . . . . . . . . . d d_year=2001
. . . . . . . . . . . . . . . . . . . . . . . and ss_sold_date
between '2001-12-01' and '2001-12-31'
. . . . . . . . . . . . . . . . . . . . . . . group by i_brand, i_brand_id
. . . . . . . . . . . . . . . . . . . . . . . order by ext_price desc,
brand_id
. . . . . . . . . . . . . . . . . . . . . . . limit 100 ;
Error: Error while compiling statement: FAILED: ParseException line 1:65
missing FROM at 'd_moy' near 'd' in from source (state=42000,code=4)
Closing: org.apache.hive.jdbc.HiveConnection
{code}
The same query works fine if I replace tabs with some spaces.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6068) HiveServer2 client on windows does not handle the non-ascii characters properly
[
https://issues.apache.org/jira/browse/HIVE-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925766#comment-13925766
]
Thejas M Nair commented on HIVE-6068:
-
Looks good. Can you also include a test case ?
HiveServer2 client on windows does not handle the non-ascii characters
properly
---
Key: HIVE-6068
URL: https://issues.apache.org/jira/browse/HIVE-6068
Project: Hive
Issue Type: Bug
Components: HiveServer2, JDBC
Affects Versions: 0.13.0
Environment: Windows
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
Fix For: 0.13.0
Attachments: HIVE-6068.1.patch
When running a select query against a table which contains rows with
non-ascii characters HiveServer2 Beeline client returns them wrong. Example:
{noformat}
738;Garçu, Le (1995);Drama
741;Ghost in the Shell (Kôkaku kidôtai) (1995);Animation|Sci-Fi
{noformat}
come out from a HiveServer2 beeline client as:
{noformat}
'738' 'Gar?u, Le (1995)' 'Drama'
'741' 'Ghost in the Shell (K?kaku kid?tai) (1995)''Animation|Sci-Fi'
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6602) Multi-user HiveServer2 throws error
[
https://issues.apache.org/jira/browse/HIVE-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925770#comment-13925770
]
Thejas M Nair commented on HIVE-6602:
-
This looks like a race condition between threads, where one thread has created
the dir but not set the permissions, and at that instance another thread finds
that dir has been created but later finds permissions are not what it expects.
Should we just synchronize this section of code ?
{code}
if (!fs.exists(scratchDir)) {
fs.mkdirs(scratchDir);
FsPermission fsPermission = new FsPermission((short)0777);
fs.setPermission(scratchDir, fsPermission);
}
{code}
Multi-user HiveServer2 throws error
---
Key: HIVE-6602
URL: https://issues.apache.org/jira/browse/HIVE-6602
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
Fix For: 0.13.0
Attachments: HIVE-6602.1.patch
Error thrown:
Error while processing statement: FAILED: RuntimeException
org.apache.hadoop.security.AccessControlException: Permission denied:
user=user_1, access=WRITE, inode=/tmp/hive-hive:hdfs:drwxr-xr-x
For hive query execution, a scratch directory specified by
hive.exec.scratchdir is created with default permission 700. In HiveServer2,
during the CLIService startup, we check for the presence of scratch
directories (local + dfs) and if they don't exist, create them with
permission 777. However, we should also change the permission from the
default 700 to 777 in case the dfs scratch directory already exists.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (HIVE-6602) Multi-user HiveServer2 throws error
[
https://issues.apache.org/jira/browse/HIVE-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925770#comment-13925770
]
Thejas M Nair edited comment on HIVE-6602 at 3/10/14 2:47 PM:
--
Deleting the earlier comment, as I had misread the code.
was (Author: thejas):
This looks like a race condition between threads, where one thread has created
the dir but not set the permissions, and at that instance another thread finds
that dir has been created but later finds permissions are not what it expects.
Should we just synchronize this section of code ?
{code}
if (!fs.exists(scratchDir)) {
fs.mkdirs(scratchDir);
FsPermission fsPermission = new FsPermission((short)0777);
fs.setPermission(scratchDir, fsPermission);
}
{code}
Multi-user HiveServer2 throws error
---
Key: HIVE-6602
URL: https://issues.apache.org/jira/browse/HIVE-6602
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.13.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
Fix For: 0.13.0
Attachments: HIVE-6602.1.patch
Error thrown:
Error while processing statement: FAILED: RuntimeException
org.apache.hadoop.security.AccessControlException: Permission denied:
user=user_1, access=WRITE, inode=/tmp/hive-hive:hdfs:drwxr-xr-x
For hive query execution, a scratch directory specified by
hive.exec.scratchdir is created with default permission 700. In HiveServer2,
during the CLIService startup, we check for the presence of scratch
directories (local + dfs) and if they don't exist, create them with
permission 777. However, we should also change the permission from the
default 700 to 777 in case the dfs scratch directory already exists.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6602) Multi-user HiveServer2 throws error
[ https://issues.apache.org/jira/browse/HIVE-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925774#comment-13925774 ] Thejas M Nair commented on HIVE-6602: - +1 Multi-user HiveServer2 throws error --- Key: HIVE-6602 URL: https://issues.apache.org/jira/browse/HIVE-6602 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.13.0 Attachments: HIVE-6602.1.patch Error thrown: Error while processing statement: FAILED: RuntimeException org.apache.hadoop.security.AccessControlException: Permission denied: user=user_1, access=WRITE, inode=/tmp/hive-hive:hdfs:drwxr-xr-x For hive query execution, a scratch directory specified by hive.exec.scratchdir is created with default permission 700. In HiveServer2, during the CLIService startup, we check for the presence of scratch directories (local + dfs) and if they don't exist, create them with permission 777. However, we should also change the permission from the default 700 to 777 in case the dfs scratch directory already exists. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18925: HIVE-6575 select * fails on parquet table with map datatype
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18925/#review36644
---
Ship it!
go for r3 with the getClass (and no instanceof) check and {} formatting.
- justin coffey
On March 8, 2014, 12:01 a.m., Szehon Ho wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18925/
---
(Updated March 8, 2014, 12:01 a.m.)
Review request for hive, Brock Noland, justin coffey, and Xuefu Zhang.
Repository: hive-git
Description
---
The issue is, as part of select * query, a DeepParquetHiveMapInspector is
used for one column of an overall parquet-table struct object inspector.
The problem lies in the ObjectInspectorFactory's cache for struct object
inspector. For performance, there is a cache keyed on an array list, of all
object inspectors of columns. The second time the query is run, it attempts
to lookup cached struct inspector. But when the hashmap looks up the part of
the key consisting of the DeepParquetHiveMapInspector, java calls .equals
against the existing DeepParquetHivemapInspector. This fails, as the .equals
method casted the other to a StandardParquetHiveInspector.
Regenerating the .equals and .hashcode from eclipse.
Also adding one more check in .equals before casting, to handle the case if
another class of object inspector gets hashed to the same hashcode in the
cache. Then java would call .equals against the other, which in this case is
not of the same class.
Diffs
-
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/AbstractParquetMapInspector.java
1d72747
Diff: https://reviews.apache.org/r/18925/diff/
Testing
---
Manual testing.
Thanks,
Szehon Ho
[jira] [Commented] (HIVE-6558) HiveServer2 Plain SASL authentication broken after hadoop 2.3 upgrade
[ https://issues.apache.org/jira/browse/HIVE-6558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925790#comment-13925790 ] Ashutosh Chauhan commented on HIVE-6558: [~prasadm] I guess you need to reupload the patch for Hive QA to pick it up. HiveServer2 Plain SASL authentication broken after hadoop 2.3 upgrade - Key: HIVE-6558 URL: https://issues.apache.org/jira/browse/HIVE-6558 Project: Hive Issue Type: Bug Components: Authentication, HiveServer2 Affects Versions: 0.13.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Priority: Blocker Attachments: HIVE-6558.2.patch Java only includes Plain SASL client and not server. Hence HiveServer2 includes a Plain SASL server implementation. Now Hadoop has its own Plain SASL server [HADOOP-9020|https://issues.apache.org/jira/browse/HADOOP-9020] which is part of Hadoop 2.3 [release|http://hadoop.apache.org/docs/r2.3.0/hadoop-project-dist/hadoop-common/releasenotes.html]. The two servers use different Sasl callbacks and the servers are registered in java.security.Provider via static code. As a result the HiveServer2 instance could be using Hadoop's Plain SASL server which breaks the authentication. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6599) Document new jdbc url connection parameters for HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925826#comment-13925826 ] Kevin Minder commented on HIVE-6599: Should there be a different jira filed for setting up the server side or will that be covered by this jira as well? Document new jdbc url connection parameters for HiveServer2 --- Key: HIVE-6599 URL: https://issues.apache.org/jira/browse/HIVE-6599 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Priority: Critical Fix For: 0.13.0 Need to document the parameters for http mode, ssl mode and secure proxy access. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6222) Make Vector Group By operator abandon grouping if too many distinct keys
[
https://issues.apache.org/jira/browse/HIVE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925836#comment-13925836
]
Hive QA commented on HIVE-6222:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633651/HIVE-6222.2.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5374 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_short_regress
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1696/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1696/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633651
Make Vector Group By operator abandon grouping if too many distinct keys
Key: HIVE-6222
URL: https://issues.apache.org/jira/browse/HIVE-6222
Project: Hive
Issue Type: Sub-task
Components: Query Processor
Affects Versions: 0.13.0
Reporter: Remus Rusanu
Assignee: Remus Rusanu
Priority: Minor
Labels: vectorization
Attachments: HIVE-6222.1.patch, HIVE-6222.2.patch
Row mode GBY is becoming a pass-through if not enough aggregation occurs on
the map side, relying on the shuffle+reduce side to do the work. Have VGBY do
the same.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925875#comment-13925875 ] Prasad Mujumdar commented on HIVE-5155: --- [~thejas] will take a look shortly. Thanks! Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, HIVE-5155.4.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6558) HiveServer2 Plain SASL authentication broken after hadoop 2.3 upgrade
[ https://issues.apache.org/jira/browse/HIVE-6558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-6558: -- Attachment: HIVE-6558.2.patch Re-attaching the patch for pre-commit test run HiveServer2 Plain SASL authentication broken after hadoop 2.3 upgrade - Key: HIVE-6558 URL: https://issues.apache.org/jira/browse/HIVE-6558 Project: Hive Issue Type: Bug Components: Authentication, HiveServer2 Affects Versions: 0.13.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Priority: Blocker Attachments: HIVE-6558.2.patch, HIVE-6558.2.patch Java only includes Plain SASL client and not server. Hence HiveServer2 includes a Plain SASL server implementation. Now Hadoop has its own Plain SASL server [HADOOP-9020|https://issues.apache.org/jira/browse/HADOOP-9020] which is part of Hadoop 2.3 [release|http://hadoop.apache.org/docs/r2.3.0/hadoop-project-dist/hadoop-common/releasenotes.html]. The two servers use different Sasl callbacks and the servers are registered in java.security.Provider via static code. As a result the HiveServer2 instance could be using Hadoop's Plain SASL server which breaks the authentication. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6603) Separate out Hiveserver2 service and JDBC packages from old HiveServer
Prasad Mujumdar created HIVE-6603: - Summary: Separate out Hiveserver2 service and JDBC packages from old HiveServer Key: HIVE-6603 URL: https://issues.apache.org/jira/browse/HIVE-6603 Project: Hive Issue Type: Improvement Components: Build Infrastructure, HiveServer2, JDBC Affects Versions: 0.13.0 Reporter: Prasad Mujumdar Currently both HiveServer2 and old Thrift HiveServer are part of same jar, for both service and JDBC client. The old HiveServer, especially on the client side has many dependencies that are not required for HiveServer2. It would be a good idea to separate out these two components. In the last Hive contributor meetup, there was a general consensus on deprecating old HiveServer. The code separation would help if/when community decides to stop supporting it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6603) Separate out Hiveserver2 service and JDBC packages from old HiveServer
[ https://issues.apache.org/jira/browse/HIVE-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-6603: -- Description: Currently both HiveServer2 and old Thrift HiveServer are part of same jar, for both service and JDBC client. The old HiveServer, especially on the client side has many dependencies that are not required for HiveServer2. It would be a good idea to separate out these two components. In the last Hive contributor meetup (http://www.meetup.com/Hive-Contributors-Group/events/150001922/), there was a general consensus on deprecating old HiveServer. The code separation would help if/when community decides to stop supporting it. was: Currently both HiveServer2 and old Thrift HiveServer are part of same jar, for both service and JDBC client. The old HiveServer, especially on the client side has many dependencies that are not required for HiveServer2. It would be a good idea to separate out these two components. In the last Hive contributor meetup, there was a general consensus on deprecating old HiveServer. The code separation would help if/when community decides to stop supporting it. Separate out Hiveserver2 service and JDBC packages from old HiveServer -- Key: HIVE-6603 URL: https://issues.apache.org/jira/browse/HIVE-6603 Project: Hive Issue Type: Improvement Components: Build Infrastructure, HiveServer2, JDBC Affects Versions: 0.13.0 Reporter: Prasad Mujumdar Currently both HiveServer2 and old Thrift HiveServer are part of same jar, for both service and JDBC client. The old HiveServer, especially on the client side has many dependencies that are not required for HiveServer2. It would be a good idea to separate out these two components. In the last Hive contributor meetup (http://www.meetup.com/Hive-Contributors-Group/events/150001922/), there was a general consensus on deprecating old HiveServer. The code separation would help if/when community decides to stop supporting it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6568) Vectorized cast of decimal to string and timestamp produces incorrect result.
[ https://issues.apache.org/jira/browse/HIVE-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-6568: --- Status: Open (was: Patch Available) Vectorized cast of decimal to string and timestamp produces incorrect result. - Key: HIVE-6568 URL: https://issues.apache.org/jira/browse/HIVE-6568 Project: Hive Issue Type: Bug Components: Vectorization Affects Versions: 0.13.0 Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-6568.1.patch, HIVE-6568.2.patch A decimal value 1.23 with scale 5 is represented in string as 1.23000. This behavior is different from HiveDecimal behavior. The difference in cast to timestamp is due to more aggressive rounding in vectorized expression. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6568) Vectorized cast of decimal to string and timestamp produces incorrect result.
[ https://issues.apache.org/jira/browse/HIVE-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-6568: --- Attachment: HIVE-6568.3.patch Vectorized cast of decimal to string and timestamp produces incorrect result. - Key: HIVE-6568 URL: https://issues.apache.org/jira/browse/HIVE-6568 Project: Hive Issue Type: Bug Components: Vectorization Affects Versions: 0.13.0 Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-6568.1.patch, HIVE-6568.2.patch, HIVE-6568.3.patch A decimal value 1.23 with scale 5 is represented in string as 1.23000. This behavior is different from HiveDecimal behavior. The difference in cast to timestamp is due to more aggressive rounding in vectorized expression. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6568) Vectorized cast of decimal to string and timestamp produces incorrect result.
[ https://issues.apache.org/jira/browse/HIVE-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-6568: --- Status: Patch Available (was: Open) Vectorized cast of decimal to string and timestamp produces incorrect result. - Key: HIVE-6568 URL: https://issues.apache.org/jira/browse/HIVE-6568 Project: Hive Issue Type: Bug Components: Vectorization Affects Versions: 0.13.0 Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-6568.1.patch, HIVE-6568.2.patch, HIVE-6568.3.patch A decimal value 1.23 with scale 5 is represented in string as 1.23000. This behavior is different from HiveDecimal behavior. The difference in cast to timestamp is due to more aggressive rounding in vectorized expression. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6568) Vectorized cast of decimal to string and timestamp produces incorrect result.
[ https://issues.apache.org/jira/browse/HIVE-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925892#comment-13925892 ] Jitendra Nath Pandey commented on HIVE-6568: Updated patch fixes the test TestVectorTypeCasts.testCastDecimalToString. testExecuteStatementAsync is not related to the patch. Vectorized cast of decimal to string and timestamp produces incorrect result. - Key: HIVE-6568 URL: https://issues.apache.org/jira/browse/HIVE-6568 Project: Hive Issue Type: Bug Components: Vectorization Affects Versions: 0.13.0 Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-6568.1.patch, HIVE-6568.2.patch, HIVE-6568.3.patch A decimal value 1.23 with scale 5 is represented in string as 1.23000. This behavior is different from HiveDecimal behavior. The difference in cast to timestamp is due to more aggressive rounding in vectorized expression. -- This message was sent by Atlassian JIRA (v6.2#6252)
Review Request 18972: Vectorized cast of decimal to string and timestamp produces incorrect result.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18972/ --- Review request for hive and Eric Hanson. Repository: hive-git Description --- Vectorized cast of decimal to string and timestamp produces incorrect result. Diffs - common/src/java/org/apache/hadoop/hive/common/type/Decimal128.java 9d25620 common/src/java/org/apache/hadoop/hive/common/type/UnsignedInt128.java 34bd9d0 common/src/test/org/apache/hadoop/hive/common/type/TestDecimal128.java debc270 common/src/test/org/apache/hadoop/hive/common/type/TestUnsignedInt128.java 9ac68fe ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/CastDecimalToString.java 2e8c3a4 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/CastDecimalToTimestamp.java df7e1ee ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorTypeCasts.java 832463d ql/src/test/queries/clientpositive/vector_decimal_expressions.q 38934d2 ql/src/test/results/clientpositive/vector_decimal_expressions.q.out 629f5d5 Diff: https://reviews.apache.org/r/18972/diff/ Testing --- Thanks, Jitendra Pandey
[jira] [Commented] (HIVE-6486) Support secure Subject.doAs() in HiveServer2 JDBC client.
[ https://issues.apache.org/jira/browse/HIVE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925894#comment-13925894 ] Shivaraju Gowda commented on HIVE-6486: --- Lefty Leverenz : OK sounds good. Thejas M Nair: While my main intention to add this improvement was for middle-ware server, I think it can be helpful even for regular Kerberos use case, the advantage being no dependency on hadoop-core*.jar and more flexibility on Kerberos login. The disadvantage being to know the details of how to login to Kerberos programmatically(an example of how to do it will ease this though). Support secure Subject.doAs() in HiveServer2 JDBC client. - Key: HIVE-6486 URL: https://issues.apache.org/jira/browse/HIVE-6486 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Shivaraju Gowda Assignee: Shivaraju Gowda Fix For: 0.13.0 Attachments: HIVE-6486.1.patch, HIVE-6486.2.patch, HIVE-6486.3.patch, Hive_011_Support-Subject_doAS.patch, TestHive_SujectDoAs.java HIVE-5155 addresses the problem of kerberos authentication in multi-user middleware server using proxy user. In this mode the principal used by the middle ware server has privileges to impersonate selected users in Hive/Hadoop. This enhancement is to support Subject.doAs() authentication in Hive JDBC layer so that the end users Kerberos Subject is passed through in the middle ware server. With this improvement there won't be any additional setup in the server to grant proxy privileges to some users and there won't be need to specify a proxy user in the JDBC client. This version should also be more secure since it won't require principals with the privileges to impersonate other users in Hive/Hadoop setup. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6568) Vectorized cast of decimal to string and timestamp produces incorrect result.
[ https://issues.apache.org/jira/browse/HIVE-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925895#comment-13925895 ] Jitendra Nath Pandey commented on HIVE-6568: Review board: https://reviews.apache.org/r/18972/ Vectorized cast of decimal to string and timestamp produces incorrect result. - Key: HIVE-6568 URL: https://issues.apache.org/jira/browse/HIVE-6568 Project: Hive Issue Type: Bug Components: Vectorization Affects Versions: 0.13.0 Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-6568.1.patch, HIVE-6568.2.patch, HIVE-6568.3.patch A decimal value 1.23 with scale 5 is represented in string as 1.23000. This behavior is different from HiveDecimal behavior. The difference in cast to timestamp is due to more aggressive rounding in vectorized expression. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5607) Hive fails to parse the % (mod) sign after brackets.
[ https://issues.apache.org/jira/browse/HIVE-5607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5607: -- Resolution: Fixed Fix Version/s: 0.14.0 Release Note: From 0.10.0 to 0.13.0, the following syntax for order by is allowed (though the doc doesn't specify): select * from table order by (expr1, exp2); From 0.14, the above syntax is illegal. Instead, the following should be used: select * from table order by expr1, exp2; Status: Resolved (was: Patch Available) Patch committed to trunk. Thanks Prasad for the review. Hive fails to parse the % (mod) sign after brackets. -- Key: HIVE-5607 URL: https://issues.apache.org/jira/browse/HIVE-5607 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: dima machlin Assignee: Xuefu Zhang Priority: Minor Fix For: 0.14.0 Attachments: HIVE-5607.1.patch, HIVE-5607.patch the scenario : create table t(a int); select * from t order by (a)%7; will fail with the following exception : FAILED: ParseException line 1:28 mismatched input '%' expecting EOF near ')' I must mention that this *does* work in 0.7.1 and doesn't work in 0.10 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6604) Fix vectorized input to work with ACID
Owen O'Malley created HIVE-6604: --- Summary: Fix vectorized input to work with ACID Key: HIVE-6604 URL: https://issues.apache.org/jira/browse/HIVE-6604 Project: Hive Issue Type: Sub-task Reporter: Owen O'Malley Assignee: Owen O'Malley Fix the VectorizedOrcInputFormat to work with the ACID directories. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6551) group by after join with skew join optimization references invalid task sometimes
[
https://issues.apache.org/jira/browse/HIVE-6551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harish Butani updated HIVE-6551:
Fix Version/s: (was: 0.14.0)
0.13.0
group by after join with skew join optimization references invalid task
sometimes
-
Key: HIVE-6551
URL: https://issues.apache.org/jira/browse/HIVE-6551
Project: Hive
Issue Type: Bug
Reporter: Navis
Assignee: Navis
Priority: Trivial
Fix For: 0.13.0
Attachments: HIVE-6551.1.patch.txt
For example,
{noformat}
hive set hive.auto.convert.join = true;
hive set hive.optimize.skewjoin = true;
hive set hive.skewjoin.key = 3;
hive
EXPLAIN FROM
(SELECT src.* FROM src) x
JOIN
(SELECT src.* FROM src) Y
ON (x.key = Y.key)
SELECT sum(hash(Y.key)), sum(hash(Y.value));
OK
STAGE DEPENDENCIES:
Stage-8 is a root stage
Stage-6 depends on stages: Stage-8
Stage-5 depends on stages: Stage-6 , consists of Stage-4, Stage-2
Stage-4
Stage-2 depends on stages: Stage-4, Stage-1
Stage-0 is a root stage
...
{noformat}
Stage-2 references not-existing Stage-1
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6551) group by after join with skew join optimization references invalid task sometimes
[
https://issues.apache.org/jira/browse/HIVE-6551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925956#comment-13925956
]
Harish Butani commented on HIVE-6551:
-
ported to 0.13
group by after join with skew join optimization references invalid task
sometimes
-
Key: HIVE-6551
URL: https://issues.apache.org/jira/browse/HIVE-6551
Project: Hive
Issue Type: Bug
Reporter: Navis
Assignee: Navis
Priority: Trivial
Fix For: 0.13.0
Attachments: HIVE-6551.1.patch.txt
For example,
{noformat}
hive set hive.auto.convert.join = true;
hive set hive.optimize.skewjoin = true;
hive set hive.skewjoin.key = 3;
hive
EXPLAIN FROM
(SELECT src.* FROM src) x
JOIN
(SELECT src.* FROM src) Y
ON (x.key = Y.key)
SELECT sum(hash(Y.key)), sum(hash(Y.value));
OK
STAGE DEPENDENCIES:
Stage-8 is a root stage
Stage-6 depends on stages: Stage-8
Stage-5 depends on stages: Stage-6 , consists of Stage-4, Stage-2
Stage-4
Stage-2 depends on stages: Stage-4, Stage-1
Stage-0 is a root stage
...
{noformat}
Stage-2 references not-existing Stage-1
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6559) sourcing txn-script from schema script results in failure for mysql oracle
[ https://issues.apache.org/jira/browse/HIVE-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-6559: - Attachment: HIVE-6559.patch I fixed the issue by removing the invocations from the hive-schema-0.13 scripts and inlining the table creation. sourcing txn-script from schema script results in failure for mysql oracle Key: HIVE-6559 URL: https://issues.apache.org/jira/browse/HIVE-6559 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.13.0 Reporter: Ashutosh Chauhan Assignee: Alan Gates Fix For: 0.13.0 Attachments: HIVE-6559.patch On mysql, I got: ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' SOURCE hive-txn-schem' at line 1 On Oracle, I got: SP2-0310: unable to open file hive-txn-schema-0.13.0.oracle.sql -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6574) Type in ql/pom.xml prevents jobs from parquet queries from running on a cluster
[
https://issues.apache.org/jira/browse/HIVE-6574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925968#comment-13925968
]
Harish Butani commented on HIVE-6574:
-
ported to 0.13
Type in ql/pom.xml prevents jobs from parquet queries from running on a
cluster
---
Key: HIVE-6574
URL: https://issues.apache.org/jira/browse/HIVE-6574
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Blocker
Fix For: 0.13.0
Attachments: HIVE-6574.patch
MR job complains about not found class:
{code}
Caused by: java.lang.ClassNotFoundException: parquet.hadoop.ParquetInputFormat
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
... 64 more
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6574) Type in ql/pom.xml prevents jobs from parquet queries from running on a cluster
[
https://issues.apache.org/jira/browse/HIVE-6574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harish Butani updated HIVE-6574:
Fix Version/s: (was: 0.14.0)
0.13.0
Type in ql/pom.xml prevents jobs from parquet queries from running on a
cluster
---
Key: HIVE-6574
URL: https://issues.apache.org/jira/browse/HIVE-6574
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Blocker
Fix For: 0.13.0
Attachments: HIVE-6574.patch
MR job complains about not found class:
{code}
Caused by: java.lang.ClassNotFoundException: parquet.hadoop.ParquetInputFormat
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
... 64 more
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6486) Support secure Subject.doAs() in HiveServer2 JDBC client.
[ https://issues.apache.org/jira/browse/HIVE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925973#comment-13925973 ] Thejas M Nair commented on HIVE-6486: - [~shivshi] I would still consider it an advance user scenario, as it involves understanding of non jdbc classes and API's . Since it is on the same page users can go further and read how the client library can be used this way. Support secure Subject.doAs() in HiveServer2 JDBC client. - Key: HIVE-6486 URL: https://issues.apache.org/jira/browse/HIVE-6486 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Shivaraju Gowda Assignee: Shivaraju Gowda Fix For: 0.13.0 Attachments: HIVE-6486.1.patch, HIVE-6486.2.patch, HIVE-6486.3.patch, Hive_011_Support-Subject_doAS.patch, TestHive_SujectDoAs.java HIVE-5155 addresses the problem of kerberos authentication in multi-user middleware server using proxy user. In this mode the principal used by the middle ware server has privileges to impersonate selected users in Hive/Hadoop. This enhancement is to support Subject.doAs() authentication in Hive JDBC layer so that the end users Kerberos Subject is passed through in the middle ware server. With this improvement there won't be any additional setup in the server to grant proxy privileges to some users and there won't be need to specify a proxy user in the JDBC client. This version should also be more secure since it won't require principals with the privileges to impersonate other users in Hive/Hadoop setup. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6574) Type in ql/pom.xml prevents jobs from parquet queries from running on a cluster
[
https://issues.apache.org/jira/browse/HIVE-6574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925972#comment-13925972
]
Xuefu Zhang commented on HIVE-6574:
---
Thanks, Harish!
Type in ql/pom.xml prevents jobs from parquet queries from running on a
cluster
---
Key: HIVE-6574
URL: https://issues.apache.org/jira/browse/HIVE-6574
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Blocker
Fix For: 0.13.0
Attachments: HIVE-6574.patch
MR job complains about not found class:
{code}
Caused by: java.lang.ClassNotFoundException: parquet.hadoop.ParquetInputFormat
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
... 64 more
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6559) sourcing txn-script from schema script results in failure for mysql oracle
[ https://issues.apache.org/jira/browse/HIVE-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-6559: - Status: Patch Available (was: Open) sourcing txn-script from schema script results in failure for mysql oracle Key: HIVE-6559 URL: https://issues.apache.org/jira/browse/HIVE-6559 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.13.0 Reporter: Ashutosh Chauhan Assignee: Alan Gates Fix For: 0.13.0 Attachments: HIVE-6559.patch On mysql, I got: ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' SOURCE hive-txn-schem' at line 1 On Oracle, I got: SP2-0310: unable to open file hive-txn-schema-0.13.0.oracle.sql -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6486) Support secure Subject.doAs() in HiveServer2 JDBC client.
[ https://issues.apache.org/jira/browse/HIVE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6486: Resolution: Fixed Status: Resolved (was: Patch Available) Patch committed to trunk and 0.13 branch (this is a patch included by Harish in the list for 0.13 in the cwiki page). Thanks for the contribution [~shivshi] ! Support secure Subject.doAs() in HiveServer2 JDBC client. - Key: HIVE-6486 URL: https://issues.apache.org/jira/browse/HIVE-6486 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Shivaraju Gowda Assignee: Shivaraju Gowda Fix For: 0.13.0 Attachments: HIVE-6486.1.patch, HIVE-6486.2.patch, HIVE-6486.3.patch, Hive_011_Support-Subject_doAS.patch, TestHive_SujectDoAs.java HIVE-5155 addresses the problem of kerberos authentication in multi-user middleware server using proxy user. In this mode the principal used by the middle ware server has privileges to impersonate selected users in Hive/Hadoop. This enhancement is to support Subject.doAs() authentication in Hive JDBC layer so that the end users Kerberos Subject is passed through in the middle ware server. With this improvement there won't be any additional setup in the server to grant proxy privileges to some users and there won't be need to specify a proxy user in the JDBC client. This version should also be more secure since it won't require principals with the privileges to impersonate other users in Hive/Hadoop setup. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6414) ParquetInputFormat provides data values that do not match the object inspectors
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925998#comment-13925998
]
Harish Butani commented on HIVE-6414:
-
ported to 0.13 branch
ParquetInputFormat provides data values that do not match the object
inspectors
---
Key: HIVE-6414
URL: https://issues.apache.org/jira/browse/HIVE-6414
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Remus Rusanu
Assignee: Justin Coffey
Labels: Parquet
Fix For: 0.13.0
Attachments: HIVE-6414.2.patch, HIVE-6414.3.patch, HIVE-6414.3.patch,
HIVE-6414.3.patch, HIVE-6414.patch
While working on HIVE-5998 I noticed that the ParquetRecordReader returns
IntWritable for all 'int like' types, in disaccord with the row object
inspectors. I though fine, and I worked my way around it. But I see now that
the issue trigger failuers in other places, eg. in aggregates:
{noformat}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
{cint:528534767,ctinyint:31,csmallint:4963,cfloat:31.0,cdouble:4963.0,cstring1:cvLH6Eat2yFsyy7p}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
... 8 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to java.lang.Short
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:808)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:87)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:92)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:524)
... 9 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable
cannot be cast to java.lang.Short
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaShortObjectInspector.get(JavaShortObjectInspector.java:41)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(ObjectInspectorUtils.java:671)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(ObjectInspectorUtils.java:631)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMin$GenericUDAFMinEvaluator.merge(GenericUDAFMin.java:109)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMin$GenericUDAFMinEvaluator.iterate(GenericUDAFMin.java:96)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:183)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:641)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:838)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:735)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:803)
... 15 more
{noformat}
My test is (I'm writing a test .q from HIVE-5998, but the repro does not
involve vectorization):
{noformat}
create table if not exists alltypes_parquet (
cint int,
ctinyint tinyint,
csmallint smallint,
cfloat float,
cdouble double,
cstring1 string) stored as parquet;
insert overwrite table alltypes_parquet
select cint,
ctinyint,
csmallint,
cfloat,
cdouble,
cstring1
from alltypesorc;
explain select * from alltypes_parquet limit 10; select * from
alltypes_parquet limit 10;
explain select ctinyint,
max(cint),
min(csmallint),
count(cstring1),
avg(cfloat),
stddev_pop(cdouble)
from alltypes_parquet
group by ctinyint;
select ctinyint,
max(cint),
min(csmallint),
count(cstring1),
avg(cfloat),
stddev_pop(cdouble)
from alltypes_parquet
group by ctinyint;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6414) ParquetInputFormat provides data values that do not match the object inspectors
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harish Butani updated HIVE-6414:
Fix Version/s: (was: 0.14.0)
0.13.0
ParquetInputFormat provides data values that do not match the object
inspectors
---
Key: HIVE-6414
URL: https://issues.apache.org/jira/browse/HIVE-6414
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Remus Rusanu
Assignee: Justin Coffey
Labels: Parquet
Fix For: 0.13.0
Attachments: HIVE-6414.2.patch, HIVE-6414.3.patch, HIVE-6414.3.patch,
HIVE-6414.3.patch, HIVE-6414.patch
While working on HIVE-5998 I noticed that the ParquetRecordReader returns
IntWritable for all 'int like' types, in disaccord with the row object
inspectors. I though fine, and I worked my way around it. But I see now that
the issue trigger failuers in other places, eg. in aggregates:
{noformat}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
{cint:528534767,ctinyint:31,csmallint:4963,cfloat:31.0,cdouble:4963.0,cstring1:cvLH6Eat2yFsyy7p}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
... 8 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to java.lang.Short
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:808)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:87)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:92)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:524)
... 9 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable
cannot be cast to java.lang.Short
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaShortObjectInspector.get(JavaShortObjectInspector.java:41)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(ObjectInspectorUtils.java:671)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(ObjectInspectorUtils.java:631)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMin$GenericUDAFMinEvaluator.merge(GenericUDAFMin.java:109)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMin$GenericUDAFMinEvaluator.iterate(GenericUDAFMin.java:96)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:183)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:641)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:838)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:735)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:803)
... 15 more
{noformat}
My test is (I'm writing a test .q from HIVE-5998, but the repro does not
involve vectorization):
{noformat}
create table if not exists alltypes_parquet (
cint int,
ctinyint tinyint,
csmallint smallint,
cfloat float,
cdouble double,
cstring1 string) stored as parquet;
insert overwrite table alltypes_parquet
select cint,
ctinyint,
csmallint,
cfloat,
cdouble,
cstring1
from alltypesorc;
explain select * from alltypes_parquet limit 10; select * from
alltypes_parquet limit 10;
explain select ctinyint,
max(cint),
min(csmallint),
count(cstring1),
avg(cfloat),
stddev_pop(cdouble)
from alltypes_parquet
group by ctinyint;
select ctinyint,
max(cint),
min(csmallint),
count(cstring1),
avg(cfloat),
stddev_pop(cdouble)
from alltypes_parquet
group by ctinyint;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
