[jira] [Commented] (HIVE-6981) Remove old website from SVN
[
https://issues.apache.org/jira/browse/HIVE-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985209#comment-13985209
]
Lefty Leverenz commented on HIVE-6981:
--
+1 (sorry for the delay, it was a travel day)
Remove old website from SVN
---
Key: HIVE-6981
URL: https://issues.apache.org/jira/browse/HIVE-6981
Project: Hive
Issue Type: Task
Reporter: Brock Noland
Assignee: Brock Noland
Command to do removal:
{noformat}
svn delete https://svn.apache.org/repos/asf/hive/site/ --message HIVE-6981 -
Remove old website from SVN
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HIVE-6981) Remove old website from SVN
[
https://issues.apache.org/jira/browse/HIVE-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brock Noland resolved HIVE-6981.
Resolution: Fixed
Thank you!!
Remove old website from SVN
---
Key: HIVE-6981
URL: https://issues.apache.org/jira/browse/HIVE-6981
Project: Hive
Issue Type: Task
Reporter: Brock Noland
Assignee: Brock Noland
Command to do removal:
{noformat}
svn delete https://svn.apache.org/repos/asf/hive/site/ --message HIVE-6981 -
Remove old website from SVN
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6989) Error with arithmetic operators with javaXML serialization
[
https://issues.apache.org/jira/browse/HIVE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985247#comment-13985247
]
Hive QA commented on HIVE-6989:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642562/HIVE-6989.1.patch
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 5424 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
org.apache.hadoop.hive.ql.parse.TestParse.testParse_cast1
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input20
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input8
org.apache.hadoop.hive.ql.parse.TestParse.testParse_join2
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample1
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample2
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample3
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample4
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample5
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample6
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample7
org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf4
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/83/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/83/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12642562
Error with arithmetic operators with javaXML serialization
--
Key: HIVE-6989
URL: https://issues.apache.org/jira/browse/HIVE-6989
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-6989.1.patch
A couple of members in GenericUDFBaseNumeric do not have getters/setters,
which prevents them from being serialized as part of the query plan when
using javaXML serialization. As a result, the following query:
{noformat}
select key + key from src limit 5;
{noformat}
fails with the following error:
{noformat}
java.lang.Exception: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:401)
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:425)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:233)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 11 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at
[jira] [Updated] (HIVE-6990) Direct SQL fails when the explicit schema setting is different from the default one
[ https://issues.apache.org/jira/browse/HIVE-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bing Li updated HIVE-6990: -- Status: Patch Available (was: In Progress) Direct SQL fails when the explicit schema setting is different from the default one --- Key: HIVE-6990 URL: https://issues.apache.org/jira/browse/HIVE-6990 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.12.0 Environment: hive + derby Reporter: Bing Li Assignee: Bing Li Fix For: 0.14.0 Attachments: HIVE-6990.1.patch I got the following ERROR in hive.log 2014-04-23 17:30:23,331 ERROR metastore.ObjectStore (ObjectStore.java:handleDirectSqlError(1756)) - Direct SQL failed, falling back to ORM javax.jdo.JDODataStoreException: Error executing SQL query select PARTITIONS.PART_ID from PARTITIONS inner join TBLS on PARTITIONS.TBL_ID = TBLS.TBL_ID inner join DBS on TBLS.DB_ID = DBS.DB_ID inner join PARTITION_KEY_VALS as FILTER0 on FILTER0.PART_ID = PARTITIONS.PART_ID and FILTER0.INTEGER_IDX = 0 where TBLS.TBL_NAME = ? and DBS.NAME = ? and ((FILTER0.PART_KEY_VAL = ?)). at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451) at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:321) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilterInternal(MetaStoreDirectSql.java:181) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilter(MetaStoreDirectSql.java:98) at org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByFilterInternal(ObjectStore.java:1833) at org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByFilter(ObjectStore.java:1806) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:619) at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:124) at com.sun.proxy.$Proxy11.getPartitionsByFilter(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_partitions_by_filter(HiveMetaStore.java:3310) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:619) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:103) at com.sun.proxy.$Proxy12.get_partitions_by_filter(Unknown Source) Reproduce steps: 1. set the following properties in hive-site.xml property namejavax.jdo.mapping.Schema/name valueHIVE/value /property property namejavax.jdo.option.ConnectionUserName/name valueuser1/value /property 2. execute hive queries hive create table mytbl ( key int, value string); hive load data local inpath 'examples/files/kv1.txt' overwrite into table mytbl; hive select * from mytbl; hive create view myview partitioned on (value) as select key, value from mytbl where key=98; hive alter view myview add partition (value='val_98') partition (value='val_xyz'); hive alter view myview drop partition (value='val_xyz'); -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6990) Direct SQL fails when the explicit schema setting is different from the default one
[ https://issues.apache.org/jira/browse/HIVE-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bing Li updated HIVE-6990: -- Attachment: HIVE-6990.1.patch The patch is generated based on trunk Direct SQL fails when the explicit schema setting is different from the default one --- Key: HIVE-6990 URL: https://issues.apache.org/jira/browse/HIVE-6990 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.12.0 Environment: hive + derby Reporter: Bing Li Assignee: Bing Li Fix For: 0.14.0 Attachments: HIVE-6990.1.patch I got the following ERROR in hive.log 2014-04-23 17:30:23,331 ERROR metastore.ObjectStore (ObjectStore.java:handleDirectSqlError(1756)) - Direct SQL failed, falling back to ORM javax.jdo.JDODataStoreException: Error executing SQL query select PARTITIONS.PART_ID from PARTITIONS inner join TBLS on PARTITIONS.TBL_ID = TBLS.TBL_ID inner join DBS on TBLS.DB_ID = DBS.DB_ID inner join PARTITION_KEY_VALS as FILTER0 on FILTER0.PART_ID = PARTITIONS.PART_ID and FILTER0.INTEGER_IDX = 0 where TBLS.TBL_NAME = ? and DBS.NAME = ? and ((FILTER0.PART_KEY_VAL = ?)). at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451) at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:321) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilterInternal(MetaStoreDirectSql.java:181) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilter(MetaStoreDirectSql.java:98) at org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByFilterInternal(ObjectStore.java:1833) at org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByFilter(ObjectStore.java:1806) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:619) at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:124) at com.sun.proxy.$Proxy11.getPartitionsByFilter(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_partitions_by_filter(HiveMetaStore.java:3310) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:619) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:103) at com.sun.proxy.$Proxy12.get_partitions_by_filter(Unknown Source) Reproduce steps: 1. set the following properties in hive-site.xml property namejavax.jdo.mapping.Schema/name valueHIVE/value /property property namejavax.jdo.option.ConnectionUserName/name valueuser1/value /property 2. execute hive queries hive create table mytbl ( key int, value string); hive load data local inpath 'examples/files/kv1.txt' overwrite into table mytbl; hive select * from mytbl; hive create view myview partitioned on (value) as select key, value from mytbl where key=98; hive alter view myview add partition (value='val_98') partition (value='val_xyz'); hive alter view myview drop partition (value='val_xyz'); -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6984) Analyzing partitioned table with NULL values for the partition column failed with NPE
[
https://issues.apache.org/jira/browse/HIVE-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985342#comment-13985342
]
Hive QA commented on HIVE-6984:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642553/HIVE-6984.1.patch
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 5491 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucket_map_join_tez1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucket_map_join_tez2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_count
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_limit_pushdown
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mrr
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dml
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union9
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/84/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/84/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12642553
Analyzing partitioned table with NULL values for the partition column failed
with NPE
-
Key: HIVE-6984
URL: https://issues.apache.org/jira/browse/HIVE-6984
Project: Hive
Issue Type: Bug
Components: Statistics
Affects Versions: 0.14.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Attachments: HIVE-6984.1.patch, HIVE-6984.patch
The following describes how to produce the bug:
{code}
hive desc test2;
name string
age int
hive select * from test2;
666 NULL
555 NULL
tom 15
john NULL
mayr 40
30
NULL
hive create table test3(name string) partitioned by (age int);
hive from test2 insert overwrite table test3 partition(age) select
test2.name, test2.age;
Loading data to table default.test3 partition (age=null)
Loading partition {age=40}
Loading partition {age=__HIVE_DEFAULT_PARTITION__}
Loading partition {age=30}
Loading partition {age=15}
Partition default.test3{age=15} stats: [numFiles=1, numRows=1, totalSize=4,
rawDataSize=3]
Partition default.test3{age=30} stats: [numFiles=1, numRows=1, totalSize=1,
rawDataSize=0]
Partition default.test3{age=40} stats: [numFiles=1, numRows=1, totalSize=5,
rawDataSize=4]
Partition default.test3{age=__HIVE_DEFAULT_PARTITION__} stats: [numFiles=1,
numRows=4, totalSize=46, rawDataSize=42]
hive analyze table test3 partition(age) compute statistics;
...
Task with the most failures(4):
-
Diagnostic Messages for this Task:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row
{name:666,age:null,raw__data__size:19}
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
Re: Review Request 19923: HIVE-6374: Hive job submitted with non-default name node (fs.default.name) doesn't process locations properly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19923/ --- (Updated April 30, 2014, 11:14 a.m.) Review request for hive. Changes --- Updated after sync with latest trunk Fixed bug wrong casing of table/index/db paths Repository: hive-git Description --- Create table/index/database and add partition DDL doesn't work properly if all following conditions are true: -Metastore service is used -fs.default.name is specified and it differs from the default one -Location is not specified or specified as a not fully qualified URI The root cause of this behavior is that Hive client doesn't pass configuration context to the metastore services which tries to resolve the paths. The fix is it too resolve the path in the Hive client if fs.default.name is specified and it differs from the default one (it is must easier then start passing the context, which would be a major change). Diffs (updated) - itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestDDLWithRemoteMetastoreSecondNamenode.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ca51e71 ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 3ea2163 ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java d8ad873 Diff: https://reviews.apache.org/r/19923/diff/ Testing --- Tested in deployed Hadoop cluster. New class with two tests added. Thanks, Benjamin Zhitomirsky
[jira] [Updated] (HIVE-6374) Hive job submitted with non-default name node (fs.default.name) doesn't process locations properly
[ https://issues.apache.org/jira/browse/HIVE-6374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Zhitomirsky updated HIVE-6374: --- Attachment: hive-6374.3.patch Updated after sync with latest trunk Fixed bug with wrong casing of table/index/db paths Please review and vote! (Note: the TestMinimrCliDriver tests fail even without the patch, therefore they are not related to this change) Hive job submitted with non-default name node (fs.default.name) doesn't process locations properly --- Key: HIVE-6374 URL: https://issues.apache.org/jira/browse/HIVE-6374 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.11.0, 0.12.0, 0.13.0 Environment: Any Reporter: Benjamin Zhitomirsky Assignee: Benjamin Zhitomirsky Attachments: Design of the fix HIVE-6374.docx, hive-6374.1.patch, hive-6374.3.patch, hive-6374.patch Original Estimate: 168h Remaining Estimate: 168h Create table/index/database and add partition DDL doesn't work properly if all following conditions are true: - Metastore service is used - fs.default.name is specified and it differs from the default one - Location is not specified or specified as a not fully qualified URI The root cause of this behavior is that Hive client doesn't pass configuration context to the metastore services which tries to resolve the paths. The fix is it too resolve the path in the Hive client if fs.default.name is specified and it differs from the default one (it is must easier then start passing the context, which would be a major change). The CR will submitted shortly after tests are done -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6374) Hive job submitted with non-default name node (fs.default.name) doesn't process locations properly
[ https://issues.apache.org/jira/browse/HIVE-6374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Zhitomirsky updated HIVE-6374: --- Status: In Progress (was: Patch Available) Hive job submitted with non-default name node (fs.default.name) doesn't process locations properly --- Key: HIVE-6374 URL: https://issues.apache.org/jira/browse/HIVE-6374 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.12.0, 0.11.0, 0.13.0 Environment: Any Reporter: Benjamin Zhitomirsky Assignee: Benjamin Zhitomirsky Attachments: Design of the fix HIVE-6374.docx, hive-6374.1.patch, hive-6374.3.patch, hive-6374.patch Original Estimate: 168h Remaining Estimate: 168h Create table/index/database and add partition DDL doesn't work properly if all following conditions are true: - Metastore service is used - fs.default.name is specified and it differs from the default one - Location is not specified or specified as a not fully qualified URI The root cause of this behavior is that Hive client doesn't pass configuration context to the metastore services which tries to resolve the paths. The fix is it too resolve the path in the Hive client if fs.default.name is specified and it differs from the default one (it is must easier then start passing the context, which would be a major change). The CR will submitted shortly after tests are done -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HIVE-6928) Beeline should not chop off describe extended results by default
[ https://issues.apache.org/jira/browse/HIVE-6928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam reassigned HIVE-6928: -- Assignee: Chinna Rao Lalam Beeline should not chop off describe extended results by default -- Key: HIVE-6928 URL: https://issues.apache.org/jira/browse/HIVE-6928 Project: Hive Issue Type: Bug Components: CLI Reporter: Szehon Ho Assignee: Chinna Rao Lalam By default, beeline truncates long results based on the console width like: +-+--+ | col_name | | +-+--+ | pat_id | string | | score | float | | acutes | float | | | | | Detailed Table Information | Table(tableName:refills, dbName:default, owner:hdadmin, createTime:1393882396, lastAccessTime:0, retention:0, sd:Sto | +-+--+ 5 rows selected (0.4 seconds) This can be changed by !outputformat, but the default should behave better to give a better experience to the first-time beeline user. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6928) Beeline should not chop off describe extended results by default
[ https://issues.apache.org/jira/browse/HIVE-6928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-6928: --- Attachment: HIVE-6928.patch Beeline should not chop off describe extended results by default -- Key: HIVE-6928 URL: https://issues.apache.org/jira/browse/HIVE-6928 Project: Hive Issue Type: Bug Components: CLI Reporter: Szehon Ho Assignee: Chinna Rao Lalam Attachments: HIVE-6928.patch By default, beeline truncates long results based on the console width like: +-+--+ | col_name | | +-+--+ | pat_id | string | | score | float | | acutes | float | | | | | Detailed Table Information | Table(tableName:refills, dbName:default, owner:hdadmin, createTime:1393882396, lastAccessTime:0, retention:0, sd:Sto | +-+--+ 5 rows selected (0.4 seconds) This can be changed by !outputformat, but the default should behave better to give a better experience to the first-time beeline user. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6928) Beeline should not chop off describe extended results by default
[ https://issues.apache.org/jira/browse/HIVE-6928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-6928: --- Status: Patch Available (was: Open) In this example the length of the row is more than 1000 characters. These kind of outputs showing in table format wont be look good. When ever the row length is bigger than width present that in vertical format (decide this in run time). Beeline should not chop off describe extended results by default -- Key: HIVE-6928 URL: https://issues.apache.org/jira/browse/HIVE-6928 Project: Hive Issue Type: Bug Components: CLI Reporter: Szehon Ho Assignee: Chinna Rao Lalam Attachments: HIVE-6928.patch By default, beeline truncates long results based on the console width like: +-+--+ | col_name | | +-+--+ | pat_id | string | | score | float | | acutes | float | | | | | Detailed Table Information | Table(tableName:refills, dbName:default, owner:hdadmin, createTime:1393882396, lastAccessTime:0, retention:0, sd:Sto | +-+--+ 5 rows selected (0.4 seconds) This can be changed by !outputformat, but the default should behave better to give a better experience to the first-time beeline user. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6991) History is not able to disable/enable after session is started
Chinna Rao Lalam created HIVE-6991: -- Summary: History is not able to disable/enable after session is started Key: HIVE-6991 URL: https://issues.apache.org/jira/browse/HIVE-6991 Project: Hive Issue Type: Bug Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam By default history is disabled, after session started if enable history through this command set hive.session.history.enabled=true. It is not working. I think it will help to this user query http://mail-archives.apache.org/mod_mbox/hive-user/201404.mbox/%3ccajqy7afapa_pjs6buon0o8zyt2qwfn2wt-mtznwfmurav_8...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6991) History is not able to disable/enable after session is started
[ https://issues.apache.org/jira/browse/HIVE-6991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-6991: --- Attachment: HIVE-6991.patch History is not able to disable/enable after session is started -- Key: HIVE-6991 URL: https://issues.apache.org/jira/browse/HIVE-6991 Project: Hive Issue Type: Bug Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-6991.patch By default history is disabled, after session started if enable history through this command set hive.session.history.enabled=true. It is not working. I think it will help to this user query http://mail-archives.apache.org/mod_mbox/hive-user/201404.mbox/%3ccajqy7afapa_pjs6buon0o8zyt2qwfn2wt-mtznwfmurav_8...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6991) History is not able to disable/enable after session is started
[ https://issues.apache.org/jira/browse/HIVE-6991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-6991: --- Status: Patch Available (was: Open) Added check method before starting the query, to check whether history is enabled or disabled. History is not able to disable/enable after session is started -- Key: HIVE-6991 URL: https://issues.apache.org/jira/browse/HIVE-6991 Project: Hive Issue Type: Bug Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-6991.patch By default history is disabled, after session started if enable history through this command set hive.session.history.enabled=true. It is not working. I think it will help to this user query http://mail-archives.apache.org/mod_mbox/hive-user/201404.mbox/%3ccajqy7afapa_pjs6buon0o8zyt2qwfn2wt-mtznwfmurav_8...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6982) Export all .sh equivalent for windows (.cmd files) in bin, bin/ext
[
https://issues.apache.org/jira/browse/HIVE-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985528#comment-13985528
]
Hive QA commented on HIVE-6982:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642542/HIVE-6982.2.patch
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 5424 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/85/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/85/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12642542
Export all .sh equivalent for windows (.cmd files) in bin, bin/ext
--
Key: HIVE-6982
URL: https://issues.apache.org/jira/browse/HIVE-6982
Project: Hive
Issue Type: Sub-task
Components: Windows
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan
Attachments: HIVE-6982.1.patch, HIVE-6982.2.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-6992) Implement PreparedStatement.getMetaData(), getParmeterMetaData()
Bill Oliver created HIVE-6992: - Summary: Implement PreparedStatement.getMetaData(), getParmeterMetaData() Key: HIVE-6992 URL: https://issues.apache.org/jira/browse/HIVE-6992 Project: Hive Issue Type: New Feature Components: JDBC Reporter: Bill Oliver It would be very helpful to have methods PreparedStatement.getMetaData() and also PreparedStatement.getParameterMetaData() implemented. I especially would like PreparedStatmeent.getMetaData() implemented, as I could prepare a SQL statement, and then get information about the result set, as well as information that the query is valid. I am pretty sure this information is available in some form. When you do an EXPLAIN query, the explain operation shows information about the result set including the column name/aliases and the column types. thanks you -bill -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6993) Update hive for Tez VertexLocationHint and getAVailableResource API changes
Gopal V created HIVE-6993: - Summary: Update hive for Tez VertexLocationHint and getAVailableResource API changes Key: HIVE-6993 URL: https://issues.apache.org/jira/browse/HIVE-6993 Project: Hive Issue Type: Sub-task Affects Versions: 0.13.0 Reporter: Gopal V Assignee: Gopal V Priority: Minor Fix For: 0.14.0 Tez had two breaking changes in API between 0.4.x - 0.5.x. context.getTotalAvailableResource().getMemory(); and context.scheduleVertexTasks(List TaskWithLocationHint); -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6993) Update hive for Tez VertexLocationHint and getAVailableResource API changes
[ https://issues.apache.org/jira/browse/HIVE-6993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-6993: -- Attachment: HIVE-6993.1.patch Update hive for Tez VertexLocationHint and getAVailableResource API changes --- Key: HIVE-6993 URL: https://issues.apache.org/jira/browse/HIVE-6993 Project: Hive Issue Type: Sub-task Affects Versions: 0.13.0 Reporter: Gopal V Assignee: Gopal V Priority: Minor Fix For: 0.14.0 Attachments: HIVE-6993.1.patch Tez had two breaking changes in API between 0.4.x - 0.5.x. context.getTotalAvailableResource().getMemory(); and context.scheduleVertexTasks(List TaskWithLocationHint); -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6031) explain subquery rewrite for where clause predicates
[ https://issues.apache.org/jira/browse/HIVE-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6031: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Harish! explain subquery rewrite for where clause predicates - Key: HIVE-6031 URL: https://issues.apache.org/jira/browse/HIVE-6031 Project: Hive Issue Type: Sub-task Reporter: Harish Butani Assignee: Harish Butani Fix For: 0.14.0 Attachments: HIVE-6031.1.patch, HIVE-6031.2.patch, HIVE-6031.3.patch, HIVE-6031.4.patch, HIVE-6031.5.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6994) parquet-hive createArray strips null elements
Justin Coffey created HIVE-6994: --- Summary: parquet-hive createArray strips null elements Key: HIVE-6994 URL: https://issues.apache.org/jira/browse/HIVE-6994 Project: Hive Issue Type: Bug Affects Versions: 0.13.0, 0.14.0 Reporter: Justin Coffey Assignee: Justin Coffey Fix For: 0.14.0 The createArray method in ParquetHiveSerDe strips null values from resultant ArrayWritables. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6994) parquet-hive createArray strips null elements
[ https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Coffey updated HIVE-6994: Status: Patch Available (was: Open) This patch fixes the issue in ParquetHiveSerDe, but there may be an underlying issue in parquet (this is still under investigation). parquet-hive createArray strips null elements - Key: HIVE-6994 URL: https://issues.apache.org/jira/browse/HIVE-6994 Project: Hive Issue Type: Bug Affects Versions: 0.13.0, 0.14.0 Reporter: Justin Coffey Assignee: Justin Coffey Fix For: 0.14.0 Attachments: HIVE-6994.patch The createArray method in ParquetHiveSerDe strips null values from resultant ArrayWritables. tracked here as well: https://github.com/Parquet/parquet-mr/issues/377 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6994) parquet-hive createArray strips null elements
[ https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Coffey updated HIVE-6994: Attachment: HIVE-6994.patch parquet-hive createArray strips null elements - Key: HIVE-6994 URL: https://issues.apache.org/jira/browse/HIVE-6994 Project: Hive Issue Type: Bug Affects Versions: 0.13.0, 0.14.0 Reporter: Justin Coffey Assignee: Justin Coffey Fix For: 0.14.0 Attachments: HIVE-6994.patch The createArray method in ParquetHiveSerDe strips null values from resultant ArrayWritables. tracked here as well: https://github.com/Parquet/parquet-mr/issues/377 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6994) parquet-hive createArray strips null elements
[ https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Coffey updated HIVE-6994: Description: The createArray method in ParquetHiveSerDe strips null values from resultant ArrayWritables. tracked here as well: https://github.com/Parquet/parquet-mr/issues/377 was:The createArray method in ParquetHiveSerDe strips null values from resultant ArrayWritables. parquet-hive createArray strips null elements - Key: HIVE-6994 URL: https://issues.apache.org/jira/browse/HIVE-6994 Project: Hive Issue Type: Bug Affects Versions: 0.13.0, 0.14.0 Reporter: Justin Coffey Assignee: Justin Coffey Fix For: 0.14.0 The createArray method in ParquetHiveSerDe strips null values from resultant ArrayWritables. tracked here as well: https://github.com/Parquet/parquet-mr/issues/377 -- This message was sent by Atlassian JIRA (v6.2#6252)
Review Request 20899: HIVE-6994 - parquet-hive createArray strips null elements
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20899/ --- Review request for hive. Repository: hive-git Description --- - Fix for bug in createArray() that strips null elements. - In the process refactored serde for simplification purposes. - Refactored tests for better regression testing. Diffs - data/files/parquet_create.txt ccd48ee ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java b689336 ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestParquetSerDe.java be518b9 ql/src/test/org/apache/hadoop/hive/ql/io/parquet/serde/TestParquetHiveSerDe.java PRE-CREATION ql/src/test/queries/clientpositive/parquet_create.q 0b976bd ql/src/test/results/clientpositive/parquet_create.q.out 3220be5 Diff: https://reviews.apache.org/r/20899/diff/ Testing --- Thanks, justin coffey
[jira] [Updated] (HIVE-6920) Parquet Serde Simplification
[ https://issues.apache.org/jira/browse/HIVE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Coffey updated HIVE-6920: Status: Open (was: Patch Available) Please see superceding issue here: #HIVE-6994 Parquet Serde Simplification Key: HIVE-6920 URL: https://issues.apache.org/jira/browse/HIVE-6920 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Justin Coffey Assignee: Justin Coffey Priority: Minor Fix For: 0.14.0 Attachments: HIVE-6920.patch Various fixes and code simplification in the ParquetHiveSerde (with minor optimizations) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6994) parquet-hive createArray strips null elements
[ https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985772#comment-13985772 ] Justin Coffey commented on HIVE-6994: - review board link: https://reviews.apache.org/r/20899/ parquet-hive createArray strips null elements - Key: HIVE-6994 URL: https://issues.apache.org/jira/browse/HIVE-6994 Project: Hive Issue Type: Bug Affects Versions: 0.13.0, 0.14.0 Reporter: Justin Coffey Assignee: Justin Coffey Fix For: 0.14.0 Attachments: HIVE-6994.patch The createArray method in ParquetHiveSerDe strips null values from resultant ArrayWritables. tracked here as well: https://github.com/Parquet/parquet-mr/issues/377 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6920) Parquet Serde Simplification
[ https://issues.apache.org/jira/browse/HIVE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985777#comment-13985777 ] Justin Coffey commented on HIVE-6920: - btw, in the superceding patch, I killed the pom bump to 1.4.1, but I neglected to remove my spurious comments about serde stats :). I'm late out the door right now, but if you have a chance to check the new patch and review board request, I can clean that up before a final commit :D. Parquet Serde Simplification Key: HIVE-6920 URL: https://issues.apache.org/jira/browse/HIVE-6920 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Justin Coffey Assignee: Justin Coffey Priority: Minor Fix For: 0.14.0 Attachments: HIVE-6920.patch Various fixes and code simplification in the ParquetHiveSerde (with minor optimizations) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6984) Analyzing partitioned table with NULL values for the partition column failed with NPE
[
https://issues.apache.org/jira/browse/HIVE-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6984:
--
Attachment: HIVE-6984.2.patch
Patch #2 updated with regenerated test output.
Analyzing partitioned table with NULL values for the partition column failed
with NPE
-
Key: HIVE-6984
URL: https://issues.apache.org/jira/browse/HIVE-6984
Project: Hive
Issue Type: Bug
Components: Statistics
Affects Versions: 0.14.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Attachments: HIVE-6984.1.patch, HIVE-6984.2.patch, HIVE-6984.patch
The following describes how to produce the bug:
{code}
hive desc test2;
name string
age int
hive select * from test2;
666 NULL
555 NULL
tom 15
john NULL
mayr 40
30
NULL
hive create table test3(name string) partitioned by (age int);
hive from test2 insert overwrite table test3 partition(age) select
test2.name, test2.age;
Loading data to table default.test3 partition (age=null)
Loading partition {age=40}
Loading partition {age=__HIVE_DEFAULT_PARTITION__}
Loading partition {age=30}
Loading partition {age=15}
Partition default.test3{age=15} stats: [numFiles=1, numRows=1, totalSize=4,
rawDataSize=3]
Partition default.test3{age=30} stats: [numFiles=1, numRows=1, totalSize=1,
rawDataSize=0]
Partition default.test3{age=40} stats: [numFiles=1, numRows=1, totalSize=5,
rawDataSize=4]
Partition default.test3{age=__HIVE_DEFAULT_PARTITION__} stats: [numFiles=1,
numRows=4, totalSize=46, rawDataSize=42]
hive analyze table test3 partition(age) compute statistics;
...
Task with the most failures(4):
-
Diagnostic Messages for this Task:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row
{name:666,age:null,raw__data__size:19}
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1499)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
{name:666,age:null,raw__data__size:19}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:549)
at org.apache.hado
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
{code}
The following is the stack trace in mapper log:
{code}
2014-04-28 15:39:25,073 FATAL org.apache.hadoop.hive.ql.exec.mr.ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {name:666,age:null,raw__data__size:19}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:549)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1499)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.gatherStats(TableScanOperator.java:149)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:539)
... 9 more
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6982) Export all .sh equivalent for windows (.cmd files) in bin, bin/ext
[ https://issues.apache.org/jira/browse/HIVE-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985857#comment-13985857 ] Hari Sankar Sivarama Subramaniyan commented on HIVE-6982: - The errors are unrelated to the change. Thanks. Hari Export all .sh equivalent for windows (.cmd files) in bin, bin/ext -- Key: HIVE-6982 URL: https://issues.apache.org/jira/browse/HIVE-6982 Project: Hive Issue Type: Sub-task Components: Windows Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Attachments: HIVE-6982.1.patch, HIVE-6982.2.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5092) Fix hiveserver2 mapreduce local job on Windows
[
https://issues.apache.org/jira/browse/HIVE-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hari Sankar Sivarama Subramaniyan updated HIVE-5092:
Status: Open (was: Patch Available)
Fix hiveserver2 mapreduce local job on Windows
--
Key: HIVE-5092
URL: https://issues.apache.org/jira/browse/HIVE-5092
Project: Hive
Issue Type: Bug
Components: HiveServer2, Windows
Reporter: Daniel Dai
Assignee: Hari Sankar Sivarama Subramaniyan
Attachments: HIVE-5092-1.patch, HIVE-5092.2.patch, HIVE-5092.3.patch
Hiveserver2 fail on Mapreduce local job fail. For example:
{code}
select /*+ MAPJOIN(v) */ registration from studenttab10k s join votertab10k v
on (s.name = v.name);
{code}
The root cause is class not found in the local hadoop job
(MapredLocalTask.execute). HADOOP_CLASSPATH does not include $HIVE_HOME/lib.
Set HADOOP_CLASSPATH correctly will fix the issue.
However, there is one complexity in Windows. We start Hiveserver2 using
Windows service console (services.msc), which takes hiveserver2.xml generated
by hive.cmd. There is no way to pass environment variable in hiveserver2.xml
(weird but reality). I attach a patch which pass it through command line
arguments and relay to HADOOP_CLASSPATH in Hive code.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-5092) Fix hiveserver2 mapreduce local job on Windows
[
https://issues.apache.org/jira/browse/HIVE-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985860#comment-13985860
]
Hari Sankar Sivarama Subramaniyan commented on HIVE-5092:
-
[~vaibhavgumashta] Thanks. I am working on another solution which will not
require the introduction of hive.hadoop.classpath parameter. The new patch
requires HIVE-6982 to be committed. I will upload the same once HIVE-6982 is
committed.
Fix hiveserver2 mapreduce local job on Windows
--
Key: HIVE-5092
URL: https://issues.apache.org/jira/browse/HIVE-5092
Project: Hive
Issue Type: Bug
Components: HiveServer2, Windows
Reporter: Daniel Dai
Assignee: Hari Sankar Sivarama Subramaniyan
Attachments: HIVE-5092-1.patch, HIVE-5092.2.patch, HIVE-5092.3.patch
Hiveserver2 fail on Mapreduce local job fail. For example:
{code}
select /*+ MAPJOIN(v) */ registration from studenttab10k s join votertab10k v
on (s.name = v.name);
{code}
The root cause is class not found in the local hadoop job
(MapredLocalTask.execute). HADOOP_CLASSPATH does not include $HIVE_HOME/lib.
Set HADOOP_CLASSPATH correctly will fix the issue.
However, there is one complexity in Windows. We start Hiveserver2 using
Windows service console (services.msc), which takes hiveserver2.xml generated
by hive.cmd. There is no way to pass environment variable in hiveserver2.xml
(weird but reality). I attach a patch which pass it through command line
arguments and relay to HADOOP_CLASSPATH in Hive code.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6966) More fixes for TestCliDriver on Windows
[ https://issues.apache.org/jira/browse/HIVE-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985926#comment-13985926 ] Sushanth Sowmyan commented on HIVE-6966: Looks good to me. +1. More fixes for TestCliDriver on Windows --- Key: HIVE-6966 URL: https://issues.apache.org/jira/browse/HIVE-6966 Project: Hive Issue Type: Bug Components: Tests, Windows Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6966.1.patch This prevents the Test*CliDriver tests from compiling properly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6931) Windows unit test fixes
[ https://issues.apache.org/jira/browse/HIVE-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985936#comment-13985936 ] Sushanth Sowmyan commented on HIVE-6931: Looks good to me. About the only nitpick I have is that it feels wrong to call WindowsPathUtil.getHdfsUriString (after moving the function to WindowsPathUtil) from QTestUtil from a section of code that's not specific to windows. From the actual code, it's perfectly fine, since the function only modifies the uri if it detects it's in windows, but it reads as if we get the windows version of hdfs uri from code. That said, my issue is not really with this change as much as code readability in general, and this is helping remove redundant code and packaging it in the relevant place, it's merely that it reads funnily. Eventually, I would like us to move WindowsPathUtil into PathUtil or somesuch. +1. Windows unit test fixes --- Key: HIVE-6931 URL: https://issues.apache.org/jira/browse/HIVE-6931 Project: Hive Issue Type: Bug Components: Tests, Windows Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6931.1.patch, HIVE-6931.1.patch A few misc fixes for some of the unit tests on Windows. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6931) Windows unit test fixes
[ https://issues.apache.org/jira/browse/HIVE-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985959#comment-13985959 ] Sushanth Sowmyan commented on HIVE-6931: Also, the test failures noted here seem to not be related to the patch. [~jdere], could you please verify/confirm? I can go ahead and commit it after that. Windows unit test fixes --- Key: HIVE-6931 URL: https://issues.apache.org/jira/browse/HIVE-6931 Project: Hive Issue Type: Bug Components: Tests, Windows Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6931.1.patch, HIVE-6931.1.patch A few misc fixes for some of the unit tests on Windows. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6966) More fixes for TestCliDriver on Windows
[ https://issues.apache.org/jira/browse/HIVE-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985960#comment-13985960 ] Sushanth Sowmyan commented on HIVE-6966: The test failures noted here seem to not be related to the patch. [~jdere], could you please verify/confirm? I can go ahead and commit it after that. More fixes for TestCliDriver on Windows --- Key: HIVE-6966 URL: https://issues.apache.org/jira/browse/HIVE-6966 Project: Hive Issue Type: Bug Components: Tests, Windows Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6966.1.patch This prevents the Test*CliDriver tests from compiling properly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6989) Error with arithmetic operators with javaXML serialization
[
https://issues.apache.org/jira/browse/HIVE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-6989:
-
Status: Open (was: Patch Available)
Looks like the TestParse failures need to be addressed by updating the golden
files used in the diff.
Error with arithmetic operators with javaXML serialization
--
Key: HIVE-6989
URL: https://issues.apache.org/jira/browse/HIVE-6989
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-6989.1.patch
A couple of members in GenericUDFBaseNumeric do not have getters/setters,
which prevents them from being serialized as part of the query plan when
using javaXML serialization. As a result, the following query:
{noformat}
select key + key from src limit 5;
{noformat}
fails with the following error:
{noformat}
java.lang.Exception: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:401)
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:425)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:233)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 11 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 16 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 19 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:154)
... 24 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDFBaseNumeric.initialize(GenericUDFBaseNumeric.java:109)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
at
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:127)
at
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorHead.initialize(ExprNodeEvaluatorHead.java:39)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:931)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:957)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
at
[jira] [Commented] (HIVE-5092) Fix hiveserver2 mapreduce local job on Windows
[
https://issues.apache.org/jira/browse/HIVE-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985975#comment-13985975
]
Vaibhav Gumashta commented on HIVE-5092:
[~hsubramaniyan] Thanks for the update. I think that's a more desirable
solution than introducing a new parameter.
Fix hiveserver2 mapreduce local job on Windows
--
Key: HIVE-5092
URL: https://issues.apache.org/jira/browse/HIVE-5092
Project: Hive
Issue Type: Bug
Components: HiveServer2, Windows
Reporter: Daniel Dai
Assignee: Hari Sankar Sivarama Subramaniyan
Attachments: HIVE-5092-1.patch, HIVE-5092.2.patch, HIVE-5092.3.patch
Hiveserver2 fail on Mapreduce local job fail. For example:
{code}
select /*+ MAPJOIN(v) */ registration from studenttab10k s join votertab10k v
on (s.name = v.name);
{code}
The root cause is class not found in the local hadoop job
(MapredLocalTask.execute). HADOOP_CLASSPATH does not include $HIVE_HOME/lib.
Set HADOOP_CLASSPATH correctly will fix the issue.
However, there is one complexity in Windows. We start Hiveserver2 using
Windows service console (services.msc), which takes hiveserver2.xml generated
by hive.cmd. There is no way to pass environment variable in hiveserver2.xml
(weird but reality). I attach a patch which pass it through command line
arguments and relay to HADOOP_CLASSPATH in Hive code.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6983) Remove test.warehouse.scheme parameter
[ https://issues.apache.org/jira/browse/HIVE-6983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6983: --- Status: Open (was: Patch Available) Canceling patch, the failure from TestFolderPermissions is a valid failure - I'll have an update up soon. Remove test.warehouse.scheme parameter -- Key: HIVE-6983 URL: https://issues.apache.org/jira/browse/HIVE-6983 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.14.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-6983.patch There is a parameter test.warehouse.scheme that does not seem to achieve much in the tests that use it, but causes issues in some environments like the windows environment when trying to test for path equality. We also seem to explicitly null it out for hive itests because it has previously been set. We should remove it as something that's not useful anymore. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6989) Error with arithmetic operators with javaXML serialization
[
https://issues.apache.org/jira/browse/HIVE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-6989:
-
Status: Patch Available (was: Open)
Error with arithmetic operators with javaXML serialization
--
Key: HIVE-6989
URL: https://issues.apache.org/jira/browse/HIVE-6989
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-6989.1.patch, HIVE-6989.2.patch
A couple of members in GenericUDFBaseNumeric do not have getters/setters,
which prevents them from being serialized as part of the query plan when
using javaXML serialization. As a result, the following query:
{noformat}
select key + key from src limit 5;
{noformat}
fails with the following error:
{noformat}
java.lang.Exception: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:401)
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:425)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:233)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 11 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 16 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 19 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:154)
... 24 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDFBaseNumeric.initialize(GenericUDFBaseNumeric.java:109)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
at
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:127)
at
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorHead.initialize(ExprNodeEvaluatorHead.java:39)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:931)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:957)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
at
[jira] [Updated] (HIVE-6989) Error with arithmetic operators with javaXML serialization
[
https://issues.apache.org/jira/browse/HIVE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-6989:
-
Attachment: HIVE-6989.2.patch
Fix TestParse failures by updating output xml files.
Error with arithmetic operators with javaXML serialization
--
Key: HIVE-6989
URL: https://issues.apache.org/jira/browse/HIVE-6989
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-6989.1.patch, HIVE-6989.2.patch
A couple of members in GenericUDFBaseNumeric do not have getters/setters,
which prevents them from being serialized as part of the query plan when
using javaXML serialization. As a result, the following query:
{noformat}
select key + key from src limit 5;
{noformat}
fails with the following error:
{noformat}
java.lang.Exception: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:401)
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:425)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:233)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 11 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 16 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 19 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:154)
... 24 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDFBaseNumeric.initialize(GenericUDFBaseNumeric.java:109)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:116)
at
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:127)
at
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorHead.initialize(ExprNodeEvaluatorHead.java:39)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:931)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:957)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
at
[jira] [Commented] (HIVE-6966) More fixes for TestCliDriver on Windows
[ https://issues.apache.org/jira/browse/HIVE-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986037#comment-13986037 ] Jason Dere commented on HIVE-6966: -- These failures do not look related, they appear to be the list of failing tests on hadoop-2 at the time. Most of these should be resolved by now, we can re-run the tests if you want. More fixes for TestCliDriver on Windows --- Key: HIVE-6966 URL: https://issues.apache.org/jira/browse/HIVE-6966 Project: Hive Issue Type: Bug Components: Tests, Windows Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6966.1.patch This prevents the Test*CliDriver tests from compiling properly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6985) sql std auth - privileges grants to public role not being honored
[ https://issues.apache.org/jira/browse/HIVE-6985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6985: Attachment: HIVE-6985.1.patch sql std auth - privileges grants to public role not being honored - Key: HIVE-6985 URL: https://issues.apache.org/jira/browse/HIVE-6985 Project: Hive Issue Type: Bug Components: Authorization, SQLStandardAuthorization Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-6985.1.patch When a privilege is granted to public role, the privilege is supposed to be applicable for all users. However, the privilege check fails for users, even if the have public role in the list of current roles. Note that the issue is only with public role. Grant of privileges of other role are not affected. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6985) sql std auth - privileges grants to public role not being honored
[ https://issues.apache.org/jira/browse/HIVE-6985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6985: Status: Patch Available (was: Open) sql std auth - privileges grants to public role not being honored - Key: HIVE-6985 URL: https://issues.apache.org/jira/browse/HIVE-6985 Project: Hive Issue Type: Bug Components: Authorization, SQLStandardAuthorization Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-6985.1.patch When a privilege is granted to public role, the privilege is supposed to be applicable for all users. However, the privilege check fails for users, even if the have public role in the list of current roles. Note that the issue is only with public role. Grant of privileges of other role are not affected. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6931) Windows unit test fixes
[ https://issues.apache.org/jira/browse/HIVE-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986063#comment-13986063 ] Jason Dere commented on HIVE-6931: -- These do not look related to the patch, looks like the existing set of failing tests on hadoop-2 Windows unit test fixes --- Key: HIVE-6931 URL: https://issues.apache.org/jira/browse/HIVE-6931 Project: Hive Issue Type: Bug Components: Tests, Windows Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6931.1.patch, HIVE-6931.1.patch A few misc fixes for some of the unit tests on Windows. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986093#comment-13986093
]
Hive QA commented on HIVE-6901:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642600/HIVE-6901.8.patch
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 5426 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_serde
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_unset_table_view_property
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/87/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/87/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12642600
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6901.1.patch, HIVE-6901.2.patch, HIVE-6901.3.patch,
HIVE-6901.4.patch, HIVE-6901.5.patch, HIVE-6901.6.patch, HIVE-6901.7.patch,
HIVE-6901.8.patch, HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6990) Direct SQL fails when the explicit schema setting is different from the default one
[
https://issues.apache.org/jira/browse/HIVE-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986094#comment-13986094
]
Hive QA commented on HIVE-6990:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642612/HIVE-6990.1.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/88/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/88/console
Messages:
{noformat}
This message was trimmed, see log for full details
Reverted 'ql/src/test/results/clientpositive/ppd_outer_join3.q.out'
Reverted 'ql/src/test/results/clientpositive/input42.q.out'
Reverted 'ql/src/test/results/clientpositive/having.q.out'
Reverted 'ql/src/test/results/clientpositive/quote1.q.out'
Reverted 'ql/src/test/results/clientpositive/index_auto_unused.q.out'
Reverted 'ql/src/test/results/clientpositive/join_vc.q.out'
Reverted 'ql/src/test/results/clientpositive/vector_non_string_partition.q.out'
Reverted 'ql/src/test/results/clientpositive/auto_join23.q.out'
Reverted 'ql/src/test/results/clientpositive/union15.q.out'
Reverted 'ql/src/test/results/clientpositive/udf5.q.out'
Reverted 'ql/src/test/results/clientpositive/sort_merge_join_desc_4.q.out'
Reverted 'ql/src/test/results/clientpositive/join16.q.out'
Reverted 'ql/src/test/results/clientpositive/nullgroup2.q.out'
Reverted 'ql/src/test/results/clientpositive/vectorized_bucketmapjoin1.q.out'
Reverted 'ql/src/test/results/clientpositive/groupby_grouping_sets5.q.out'
Reverted 'ql/src/test/results/clientpositive/limit_pushdown_negative.q.out'
Reverted 'ql/src/test/results/clientpositive/bucketmapjoin10.q.out'
Reverted 'ql/src/test/results/clientpositive/auto_join32.q.out'
Reverted 'ql/src/test/results/clientpositive/union24.q.out'
Reverted 'ql/src/test/results/clientpositive/index_auto.q.out'
Reverted 'ql/src/test/results/clientpositive/ppd_join.q.out'
Reverted 'ql/src/test/results/clientpositive/join_reorder4.q.out'
Reverted 'ql/src/test/results/clientpositive/metadata_only_queries.q.out'
Reverted 'ql/src/test/results/clientpositive/input2.q.out'
Reverted 'ql/src/test/results/clientpositive/vectorization_div0.q.out'
Reverted 'ql/src/test/results/clientpositive/udtf_parse_url_tuple.q.out'
Reverted 'ql/src/test/results/clientpositive/join11.q.out'
Reverted 'ql/src/test/results/clientpositive/smb_mapjoin_15.q.out'
Reverted 'ql/src/test/results/clientpositive/join_cond_pushdown_3.q.out'
Reverted 'ql/src/test/results/clientpositive/bucketmapjoin1.q.out'
Reverted 'ql/src/test/results/clientpositive/correlationoptimizer14.q.out'
Reverted
'ql/src/test/results/clientpositive/metadata_only_queries_with_filters.q.out'
Reverted 'ql/src/test/results/clientpositive/join20.q.out'
Reverted 'ql/src/test/results/clientpositive/correlationoptimizer9.q.out'
Reverted 'ql/src/test/results/clientpositive/sample10.q.out'
Reverted 'ql/src/test/results/clientpositive/list_bucket_dml_5.q.out'
Reverted 'ql/src/test/results/clientpositive/smb_mapjoin_10.q.out'
Reverted 'ql/src/test/results/clientpositive/groupby_rollup1.q.out'
Reverted 'ql/src/test/results/clientpositive/ppd_gby2.q.out'
Reverted 'ql/src/test/results/clientpositive/list_bucket_dml_13.q.out'
Reverted 'ql/src/test/results/clientpositive/vectorized_context.q.out'
Reverted 'ql/src/test/results/clientpositive/union_view.q.out'
Reverted 'ql/src/test/results/clientpositive/groupby_ppd.q.out'
Reverted 'ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out'
Reverted 'ql/src/test/results/clientpositive/udf_current_database.q.out'
Reverted 'ql/src/test/results/clientpositive/skewjoinopt9.q.out'
Reverted 'ql/src/test/results/clientpositive/input39_hadoop20.q.out'
Reverted 'ql/src/test/results/clientpositive/correlationoptimizer4.q.out'
Reverted 'ql/src/test/results/clientpositive/vectorized_date_funcs.q.out'
Reverted 'ql/src/test/results/clientpositive/list_bucket_query_oneskew_3.q.out'
Reverted 'ql/src/test/results/clientpositive/louter_join_ppr.q.out'
Reverted 'ql/src/test/results/clientpositive/join_reorder.q.out'
Reverted
'ql/src/test/results/clientpositive/list_bucket_query_multiskew_3.q.out'
Reverted 'ql/src/test/results/clientpositive/join_cond_pushdown_unqual3.q.out'
Reverted 'ql/src/test/results/clientpositive/annotate_stats_join.q.out'
Reverted 'ql/src/test/results/clientpositive/auto_join0.q.out'
Reverted 'ql/src/test/results/clientpositive/quotedid_skew.q.out'
Reverted 'ql/src/test/results/clientpositive/subquery_notin.q.out'
Reverted 'ql/src/test/results/clientpositive/subquery_notexists_having.q.out'
Reverted 'ql/src/test/results/clientpositive/skewjoinopt4.q.out'
Reverted 'ql/src/test/results/clientpositive/show_indexes_edge_cases.q.out'
Reverted 'ql/src/test/results/clientpositive/keyword_1.q.out'
Reverted
[jira] [Updated] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6901:
--
Attachment: HIVE-6901.9.patch
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6901.1.patch, HIVE-6901.2.patch, HIVE-6901.3.patch,
HIVE-6901.4.patch, HIVE-6901.5.patch, HIVE-6901.6.patch, HIVE-6901.7.patch,
HIVE-6901.8.patch, HIVE-6901.9.patch, HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6990) Direct SQL fails when the explicit schema setting is different from the default one
[ https://issues.apache.org/jira/browse/HIVE-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986141#comment-13986141 ] Sergey Shelukhin commented on HIVE-6990: Some tests appear to fail. Related? There are tabs in the patch (as well as trailing whitespace in some places). I wonder if names of tables could just be pre-generated into string fields when schema is set, and fields with names like DBS/TBLS/... used. That will make queries easier to read. Direct SQL fails when the explicit schema setting is different from the default one --- Key: HIVE-6990 URL: https://issues.apache.org/jira/browse/HIVE-6990 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.12.0 Environment: hive + derby Reporter: Bing Li Assignee: Bing Li Fix For: 0.14.0 Attachments: HIVE-6990.1.patch I got the following ERROR in hive.log 2014-04-23 17:30:23,331 ERROR metastore.ObjectStore (ObjectStore.java:handleDirectSqlError(1756)) - Direct SQL failed, falling back to ORM javax.jdo.JDODataStoreException: Error executing SQL query select PARTITIONS.PART_ID from PARTITIONS inner join TBLS on PARTITIONS.TBL_ID = TBLS.TBL_ID inner join DBS on TBLS.DB_ID = DBS.DB_ID inner join PARTITION_KEY_VALS as FILTER0 on FILTER0.PART_ID = PARTITIONS.PART_ID and FILTER0.INTEGER_IDX = 0 where TBLS.TBL_NAME = ? and DBS.NAME = ? and ((FILTER0.PART_KEY_VAL = ?)). at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451) at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:321) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilterInternal(MetaStoreDirectSql.java:181) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilter(MetaStoreDirectSql.java:98) at org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByFilterInternal(ObjectStore.java:1833) at org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByFilter(ObjectStore.java:1806) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:619) at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:124) at com.sun.proxy.$Proxy11.getPartitionsByFilter(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_partitions_by_filter(HiveMetaStore.java:3310) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:619) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:103) at com.sun.proxy.$Proxy12.get_partitions_by_filter(Unknown Source) Reproduce steps: 1. set the following properties in hive-site.xml property namejavax.jdo.mapping.Schema/name valueHIVE/value /property property namejavax.jdo.option.ConnectionUserName/name valueuser1/value /property 2. execute hive queries hive create table mytbl ( key int, value string); hive load data local inpath 'examples/files/kv1.txt' overwrite into table mytbl; hive select * from mytbl; hive create view myview partitioned on (value) as select key, value from mytbl where key=98; hive alter view myview add partition (value='val_98') partition (value='val_xyz'); hive alter view myview drop partition (value='val_xyz'); -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6985) sql std auth - privileges grants to public role not being honored
[ https://issues.apache.org/jira/browse/HIVE-6985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6985: Attachment: HIVE-6985.2.patch HIVE-6985.2.patch - includes test sql std auth - privileges grants to public role not being honored - Key: HIVE-6985 URL: https://issues.apache.org/jira/browse/HIVE-6985 Project: Hive Issue Type: Bug Components: Authorization, SQLStandardAuthorization Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-6985.1.patch, HIVE-6985.2.patch When a privilege is granted to public role, the privilege is supposed to be applicable for all users. However, the privilege check fails for users, even if the have public role in the list of current roles. Note that the issue is only with public role. Grant of privileges of other role are not affected. -- This message was sent by Atlassian JIRA (v6.2#6252)
Review Request 20920: HIVE-6985 : sql std auth - privileges grants to public role not being honored
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20920/ --- Review request for hive and Ashutosh Chauhan. Bugs: HIVE-6985 https://issues.apache.org/jira/browse/HIVE-6985 Repository: hive-git Description --- When a privilege is granted to public role, the privilege is supposed to be applicable for all users. However, the privilege check fails for users, even if the have public role in the list of current roles. Note that the issue is only with public role. Grant of privileges of other role are not affected. Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 5fc4af4 metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 3ea87a0 Diff: https://reviews.apache.org/r/20920/diff/ Testing --- New test included. Thanks, Thejas Nair
[jira] [Created] (HIVE-6995) GenericUDFBridge should log exception when it is unable to instantiate UDF object
Jason Dere created HIVE-6995: Summary: GenericUDFBridge should log exception when it is unable to instantiate UDF object Key: HIVE-6995 URL: https://issues.apache.org/jira/browse/HIVE-6995 Project: Hive Issue Type: Bug Components: UDF Reporter: Jason Dere Assignee: Jason Dere If GenericUDFBridge.initialize() is unable to create an instance of the UDF class, it logs a blanket error message of The UDF implementation class udf is not present in the class path. This can make it difficult to determine what the actual error is as there can be other issues besides the class not being in the class path. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 20920: HIVE-6985 : sql std auth - privileges grants to public role not being honored
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20920/ --- (Updated April 30, 2014, 10:26 p.m.) Review request for hive and Ashutosh Chauhan. Changes --- HIVE-6985.3.patch - really includes the test! Bugs: HIVE-6985 https://issues.apache.org/jira/browse/HIVE-6985 Repository: hive-git Description --- When a privilege is granted to public role, the privilege is supposed to be applicable for all users. However, the privilege check fails for users, even if the have public role in the list of current roles. Note that the issue is only with public role. Grant of privileges of other role are not affected. Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 5fc4af4 metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 3ea87a0 ql/src/test/queries/clientpositive/authorization_grant_public_role.q PRE-CREATION ql/src/test/results/clientpositive/authorization_grant_public_role.q.out PRE-CREATION Diff: https://reviews.apache.org/r/20920/diff/ Testing --- New test included. Thanks, Thejas Nair
[jira] [Updated] (HIVE-6985) sql std auth - privileges grants to public role not being honored
[ https://issues.apache.org/jira/browse/HIVE-6985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6985: Attachment: HIVE-6985.3.patch HIVE-6985.3.patch - this one really includes the tests ! sql std auth - privileges grants to public role not being honored - Key: HIVE-6985 URL: https://issues.apache.org/jira/browse/HIVE-6985 Project: Hive Issue Type: Bug Components: Authorization, SQLStandardAuthorization Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-6985.1.patch, HIVE-6985.2.patch, HIVE-6985.3.patch When a privilege is granted to public role, the privilege is supposed to be applicable for all users. However, the privilege check fails for users, even if the have public role in the list of current roles. Note that the issue is only with public role. Grant of privileges of other role are not affected. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6985) sql std auth - privileges grants to public role not being honored
[ https://issues.apache.org/jira/browse/HIVE-6985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6985: Status: Patch Available (was: Open) sql std auth - privileges grants to public role not being honored - Key: HIVE-6985 URL: https://issues.apache.org/jira/browse/HIVE-6985 Project: Hive Issue Type: Bug Components: Authorization, SQLStandardAuthorization Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-6985.1.patch, HIVE-6985.2.patch, HIVE-6985.3.patch When a privilege is granted to public role, the privilege is supposed to be applicable for all users. However, the privilege check fails for users, even if the have public role in the list of current roles. Note that the issue is only with public role. Grant of privileges of other role are not affected. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6985) sql std auth - privileges grants to public role not being honored
[ https://issues.apache.org/jira/browse/HIVE-6985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6985: Status: Open (was: Patch Available) sql std auth - privileges grants to public role not being honored - Key: HIVE-6985 URL: https://issues.apache.org/jira/browse/HIVE-6985 Project: Hive Issue Type: Bug Components: Authorization, SQLStandardAuthorization Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-6985.1.patch, HIVE-6985.2.patch, HIVE-6985.3.patch When a privilege is granted to public role, the privilege is supposed to be applicable for all users. However, the privilege check fails for users, even if the have public role in the list of current roles. Note that the issue is only with public role. Grant of privileges of other role are not affected. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18936: HIVE-6430 MapJoin hash table has large memory overhead
On April 30, 2014, 2:37 a.m., Gopal V wrote:
ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java, line 248
https://reviews.apache.org/r/18936/diff/11/?file=571300#file571300line248
{} block please, I almost missed reading that.
sorry, that is left over from debugging
On April 30, 2014, 2:37 a.m., Gopal V wrote:
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/BytesBytesMultiHashMap.java,
line 620
https://reviews.apache.org/r/18936/diff/11/?file=571303#file571303line620
mark class as final, so that the method inlining kicks in.
hmm... will do, but is it needed for static methods?
On April 30, 2014, 2:37 a.m., Gopal V wrote:
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java,
line 140
https://reviews.apache.org/r/18936/diff/11/?file=571306#file571306line140
+1 - this shows in the older profiles as being very heavy.
?
On April 30, 2014, 2:37 a.m., Gopal V wrote:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HashTableLoader.java, line
119
https://reviews.apache.org/r/18936/diff/11/?file=571311#file571311line119
Is this being checked twice for some reason?
once for each table. I think the key is supposed to be the same now.
- Sergey
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18936/#review41798
---
On April 30, 2014, 2:03 a.m., Sergey Shelukhin wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18936/
---
(Updated April 30, 2014, 2:03 a.m.)
Review request for hive, Gopal V and Gunther Hagleitner.
Repository: hive-git
Description
---
See JIRA
Diffs
-
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 0efe58b
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java 5fe35a5
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
142bfd8
ql/src/java/org/apache/hadoop/hive/ql/Driver.java bf9d4c1
ql/src/java/org/apache/hadoop/hive/ql/debug/Utils.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java
f5d4670
ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java b93ea7a
ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 175d3ab
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/AbstractMapJoinTableContainer.java
8854b19
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/BytesBytesMultiHashMap.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HashMapWrapper.java
9df425b
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java
64f0be2
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinPersistableTableContainer.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinRowContainer.java
008a8db
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainer.java
988959f
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainerSerDe.java
55b7415
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HashTableLoader.java e392592
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapJoinOperator.java
eef7656
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedColumnarSerDe.java
d4be78d
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java
3077d75
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java
f7b499b
ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 157d072
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 118b339
ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestBytesBytesMultiHashMap.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestMapJoinEqualityTableContainer.java
65e3779
ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestMapJoinTableContainer.java
093da55
ql/src/test/queries/clientpositive/mapjoin_decimal.q b65a7be
ql/src/test/queries/clientpositive/mapjoin_mapjoin.q 1eb95f6
ql/src/test/queries/clientpositive/tez_union.q f80d94c
ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out 8350670
ql/src/test/results/clientpositive/tez/mapjoin_decimal.q.out 3c55b5c
ql/src/test/results/clientpositive/tez/mapjoin_mapjoin.q.out 284cc03
serde/src/java/org/apache/hadoop/hive/serde2/ByteStream.java 73d9b29
serde/src/java/org/apache/hadoop/hive/serde2/WriteBuffers.java PRE-CREATION
[jira] [Updated] (HIVE-6995) GenericUDFBridge should log exception when it is unable to instantiate UDF object
[ https://issues.apache.org/jira/browse/HIVE-6995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6995: - Attachment: HIVE-6995.1.patch GenericUDFBridge should log exception when it is unable to instantiate UDF object - Key: HIVE-6995 URL: https://issues.apache.org/jira/browse/HIVE-6995 Project: Hive Issue Type: Bug Components: UDF Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6995.1.patch If GenericUDFBridge.initialize() is unable to create an instance of the UDF class, it logs a blanket error message of The UDF implementation class udf is not present in the class path. This can make it difficult to determine what the actual error is as there can be other issues besides the class not being in the class path. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6995) GenericUDFBridge should log exception when it is unable to instantiate UDF object
[ https://issues.apache.org/jira/browse/HIVE-6995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6995: - Status: Patch Available (was: Open) GenericUDFBridge should log exception when it is unable to instantiate UDF object - Key: HIVE-6995 URL: https://issues.apache.org/jira/browse/HIVE-6995 Project: Hive Issue Type: Bug Components: UDF Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6995.1.patch If GenericUDFBridge.initialize() is unable to create an instance of the UDF class, it logs a blanket error message of The UDF implementation class udf is not present in the class path. This can make it difficult to determine what the actual error is as there can be other issues besides the class not being in the class path. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6932) hive README needs update
[ https://issues.apache.org/jira/browse/HIVE-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6932: Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Patch committed to trunk. Thanks for the reviews! hive README needs update Key: HIVE-6932 URL: https://issues.apache.org/jira/browse/HIVE-6932 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Fix For: 0.14.0 Attachments: HIVE-6932.1.patch, HIVE-6932.2.patch, HIVE-6932.3.patch, HIVE-6932.4.patch, HIVE-6932.5.patch It needs to be updated to include Tez as a runtime. Also, it talks about average latency being in minutes, which is very misleading. NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 20826: HIVE-5823: Support for DECIMAL primitive type in AvroSerDe
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20826/#review41889 --- ql/src/test/queries/clientpositive/avro_decimal.q https://reviews.apache.org/r/20826/#comment75549 Here and below the precison and scale attributes should be JSON integers, not strings, as specified in the spec. - Skye Wanderman-Milne On April 29, 2014, 5:17 p.m., Xuefu Zhang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20826/ --- (Updated April 29, 2014, 5:17 p.m.) Review request for hive. Bugs: HIVE-5823 https://issues.apache.org/jira/browse/HIVE-5823 Repository: hive-git Description --- Support decimal type in Avro storage. The implemenation is based on specifications detailed in AVRO-1402. Diffs - data/files/dec.txt PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java 52a22e5 ql/src/test/queries/clientpositive/avro_decimal.q PRE-CREATION ql/src/test/queries/clientpositive/avro_schema_literal.q d77f310 ql/src/test/results/clientpositive/avro_decimal.q.out PRE-CREATION ql/src/test/results/clientpositive/avro_schema_literal.q.out ca945d5 serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java a28861f serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroGenericRecordWritable.java 8beffd7 serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerDe.java 92799ed serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 9d58d13 serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerializer.java b2c58c7 serde/src/java/org/apache/hadoop/hive/serde2/avro/SchemaToTypeInfo.java 251f04f serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerializer.java b3559ea serde/src/test/org/apache/hadoop/hive/serde2/avro/TestGenericAvroRecordWritable.java a0e5018 serde/src/test/org/apache/hadoop/hive/serde2/avro/Utils.java d5730fa Diff: https://reviews.apache.org/r/20826/diff/ Testing --- Unit tests are added. Test suite passed. Thanks, Xuefu Zhang
[jira] [Created] (HIVE-6996) FS based stats broken with indexed tables
Ashutosh Chauhan created HIVE-6996: -- Summary: FS based stats broken with indexed tables Key: HIVE-6996 URL: https://issues.apache.org/jira/browse/HIVE-6996 Project: Hive Issue Type: Bug Components: Indexing, Statistics Affects Versions: 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v6.2#6252)
Review Request 20912: FS based stats broken with indexed tables
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20912/ --- Review request for hive and Prasanth_J. Bugs: HIVE-6996 https://issues.apache.org/jira/browse/HIVE-6996 Repository: hive-git Description --- FS based stats broken with indexed tables Diffs - ql/src/java/org/apache/hadoop/hive/ql/index/AggregateIndexHandler.java c39e823 ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java 0af8d7d ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 0135a71 ql/src/test/queries/clientpositive/index_auth.q 413b37e ql/src/test/queries/clientpositive/index_auto.q cb8a1d6 ql/src/test/queries/clientpositive/index_auto_empty.q cb32162 ql/src/test/queries/clientpositive/index_auto_file_format.q 790e6c2 ql/src/test/queries/clientpositive/index_auto_mult_tables.q 2bf8481 ql/src/test/queries/clientpositive/index_auto_mult_tables_compact.q 808a04c ql/src/test/queries/clientpositive/index_auto_multiple.q 06e97fa ql/src/test/queries/clientpositive/index_auto_partitioned.q 5013d29 ql/src/test/queries/clientpositive/index_auto_self_join.q 0984a4a ql/src/test/queries/clientpositive/index_auto_unused.q d8f3eda ql/src/test/queries/clientpositive/index_auto_update.q 976de5c ql/src/test/queries/clientpositive/index_bitmap.q f9deb28 ql/src/test/queries/clientpositive/index_bitmap1.q 55633d9 ql/src/test/queries/clientpositive/index_bitmap2.q bd15a21 ql/src/test/queries/clientpositive/index_bitmap_auto_partitioned.q 90d7987 ql/src/test/queries/clientpositive/index_bitmap_compression.q 2f5e5d4 ql/src/test/queries/clientpositive/index_bitmap_rc.q 054df51 ql/src/test/queries/clientpositive/index_compact.q a936f11 ql/src/test/queries/clientpositive/index_compact_1.q 837033b ql/src/test/queries/clientpositive/index_compact_2.q 56119ac ql/src/test/queries/clientpositive/index_compact_3.q f3fcb4a ql/src/test/queries/clientpositive/index_compact_binary_search.q d0d9a32 ql/src/test/queries/clientpositive/index_compression.q 84ed3cc ql/src/test/queries/clientpositive/index_creation.q 062821e ql/src/test/queries/clientpositive/index_serde.q d160a6f ql/src/test/queries/clientpositive/index_stale.q 82e15b9 ql/src/test/queries/clientpositive/index_stale_partitioned.q e7cfeff Diff: https://reviews.apache.org/r/20912/diff/ Testing --- Existing test cases. Thanks, Ashutosh Chauhan
[jira] [Updated] (HIVE-6996) FS based stats broken with indexed tables
[ https://issues.apache.org/jira/browse/HIVE-6996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6996: --- Status: Patch Available (was: Open) FS based stats broken with indexed tables - Key: HIVE-6996 URL: https://issues.apache.org/jira/browse/HIVE-6996 Project: Hive Issue Type: Bug Components: Indexing, Statistics Affects Versions: 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6996.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6996) FS based stats broken with indexed tables
[ https://issues.apache.org/jira/browse/HIVE-6996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6996: --- Attachment: HIVE-6996.patch FS based stats broken with indexed tables - Key: HIVE-6996 URL: https://issues.apache.org/jira/browse/HIVE-6996 Project: Hive Issue Type: Bug Components: Indexing, Statistics Affects Versions: 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6996.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6984) Analyzing partitioned table with NULL values for the partition column failed with NPE
[
https://issues.apache.org/jira/browse/HIVE-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986200#comment-13986200
]
Sergey Shelukhin commented on HIVE-6984:
+1
Analyzing partitioned table with NULL values for the partition column failed
with NPE
-
Key: HIVE-6984
URL: https://issues.apache.org/jira/browse/HIVE-6984
Project: Hive
Issue Type: Bug
Components: Statistics
Affects Versions: 0.14.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Attachments: HIVE-6984.1.patch, HIVE-6984.2.patch, HIVE-6984.patch
The following describes how to produce the bug:
{code}
hive desc test2;
name string
age int
hive select * from test2;
666 NULL
555 NULL
tom 15
john NULL
mayr 40
30
NULL
hive create table test3(name string) partitioned by (age int);
hive from test2 insert overwrite table test3 partition(age) select
test2.name, test2.age;
Loading data to table default.test3 partition (age=null)
Loading partition {age=40}
Loading partition {age=__HIVE_DEFAULT_PARTITION__}
Loading partition {age=30}
Loading partition {age=15}
Partition default.test3{age=15} stats: [numFiles=1, numRows=1, totalSize=4,
rawDataSize=3]
Partition default.test3{age=30} stats: [numFiles=1, numRows=1, totalSize=1,
rawDataSize=0]
Partition default.test3{age=40} stats: [numFiles=1, numRows=1, totalSize=5,
rawDataSize=4]
Partition default.test3{age=__HIVE_DEFAULT_PARTITION__} stats: [numFiles=1,
numRows=4, totalSize=46, rawDataSize=42]
hive analyze table test3 partition(age) compute statistics;
...
Task with the most failures(4):
-
Diagnostic Messages for this Task:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row
{name:666,age:null,raw__data__size:19}
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1499)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
{name:666,age:null,raw__data__size:19}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:549)
at org.apache.hado
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
{code}
The following is the stack trace in mapper log:
{code}
2014-04-28 15:39:25,073 FATAL org.apache.hadoop.hive.ql.exec.mr.ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {name:666,age:null,raw__data__size:19}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:549)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1499)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.gatherStats(TableScanOperator.java:149)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:539)
... 9 more
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6430) MapJoin hash table has large memory overhead
[ https://issues.apache.org/jira/browse/HIVE-6430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-6430: --- Attachment: HIVE-6430.12.patch Fix the tag issue, CR feedback MapJoin hash table has large memory overhead Key: HIVE-6430 URL: https://issues.apache.org/jira/browse/HIVE-6430 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-6430.01.patch, HIVE-6430.02.patch, HIVE-6430.03.patch, HIVE-6430.04.patch, HIVE-6430.05.patch, HIVE-6430.06.patch, HIVE-6430.07.patch, HIVE-6430.08.patch, HIVE-6430.09.patch, HIVE-6430.10.patch, HIVE-6430.11.patch, HIVE-6430.12.patch, HIVE-6430.patch Right now, in some queries, I see that storing e.g. 4 ints (2 for key and 2 for row) can take several hundred bytes, which is ridiculous. I am reducing the size of MJKey and MJRowContainer in other jiras, but in general we don't need to have java hash table there. We can either use primitive-friendly hashtable like the one from HPPC (Apache-licenced), or some variation, to map primitive keys to single row storage structure without an object per row (similar to vectorization). -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18936: HIVE-6430 MapJoin hash table has large memory overhead
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18936/ --- (Updated April 30, 2014, 11:14 p.m.) Review request for hive, Gopal V and Gunther Hagleitner. Repository: hive-git Description --- See JIRA Diffs (updated) - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 0efe58b conf/hive-default.xml.template 6620bd1 hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java 5fe35a5 hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java 142bfd8 ql/src/java/org/apache/hadoop/hive/ql/Driver.java bf9d4c1 ql/src/java/org/apache/hadoop/hive/ql/debug/Utils.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java f5d4670 ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java b93ea7a ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 175d3ab ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/AbstractMapJoinTableContainer.java 8854b19 ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/BytesBytesMultiHashMap.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HashMapWrapper.java 9df425b ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java 64f0be2 ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinPersistableTableContainer.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinRowContainer.java 008a8db ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainer.java 988959f ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainerSerDe.java 55b7415 ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HashTableLoader.java e392592 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapJoinOperator.java eef7656 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedColumnarSerDe.java d4be78d ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java 3077d75 ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java f7b499b ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 157d072 ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 118b339 ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestBytesBytesMultiHashMap.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestMapJoinEqualityTableContainer.java 65e3779 ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestMapJoinTableContainer.java 093da55 ql/src/test/queries/clientpositive/mapjoin_decimal.q b65a7be ql/src/test/queries/clientpositive/mapjoin_mapjoin.q 1eb95f6 ql/src/test/queries/clientpositive/tez_union.q f80d94c ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out 8350670 ql/src/test/results/clientpositive/tez/mapjoin_decimal.q.out 3c55b5c ql/src/test/results/clientpositive/tez/mapjoin_mapjoin.q.out 284cc03 serde/src/java/org/apache/hadoop/hive/serde2/ByteStream.java 73d9b29 serde/src/java/org/apache/hadoop/hive/serde2/WriteBuffers.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 9079b9d serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/OutputByteBuffer.java 1b09d41 serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java 5870884 serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java bab505e serde/src/java/org/apache/hadoop/hive/serde2/dynamic_type/DynamicSerDe.java 6f344bb serde/src/java/org/apache/hadoop/hive/serde2/io/DateWritable.java 1f4ccdd serde/src/java/org/apache/hadoop/hive/serde2/io/HiveDecimalWritable.java a99c7b4 serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java 435d6c6 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazySimpleSerDe.java 82c1263 serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java b188c3f serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryStruct.java caf3517 serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 6c14081 serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/objectinspector/LazyBinaryStructObjectInspector.java e5ea452 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 06d5c5e serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java 868dd4c serde/src/test/org/apache/hadoop/hive/serde2/thrift_test/CreateSequenceFile.java 1fb49e5 Diff: https://reviews.apache.org/r/18936/diff/ Testing --- Thanks, Sergey Shelukhin
[jira] [Commented] (HIVE-6996) FS based stats broken with indexed tables
[ https://issues.apache.org/jira/browse/HIVE-6996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986212#comment-13986212 ] Prasanth J commented on HIVE-6996: -- +1 FS based stats broken with indexed tables - Key: HIVE-6996 URL: https://issues.apache.org/jira/browse/HIVE-6996 Project: Hive Issue Type: Bug Components: Indexing, Statistics Affects Versions: 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6996.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6985) sql std auth - privileges grants to public role not being honored
[ https://issues.apache.org/jira/browse/HIVE-6985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986231#comment-13986231 ] Ashutosh Chauhan commented on HIVE-6985: +1 sql std auth - privileges grants to public role not being honored - Key: HIVE-6985 URL: https://issues.apache.org/jira/browse/HIVE-6985 Project: Hive Issue Type: Bug Components: Authorization, SQLStandardAuthorization Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-6985.1.patch, HIVE-6985.2.patch, HIVE-6985.3.patch When a privilege is granted to public role, the privilege is supposed to be applicable for all users. However, the privilege check fails for users, even if the have public role in the list of current roles. Note that the issue is only with public role. Grant of privileges of other role are not affected. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6765) ASTNodeOrigin unserializable leads to fail when join with view
[ https://issues.apache.org/jira/browse/HIVE-6765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986234#comment-13986234 ] Selina Zhang commented on HIVE-6765: [~adrian-wang] We came across the same problem. Just wonder why this ASTNodeOrigin serialization problem came up suddenly. It is not the first day we join with a view. The root cause maybe something else? Thanks! ASTNodeOrigin unserializable leads to fail when join with view -- Key: HIVE-6765 URL: https://issues.apache.org/jira/browse/HIVE-6765 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Adrian Wang Fix For: 0.13.0 Attachments: HIVE-6765.patch.1 when a view contains a UDF, and the view comes into a JOIN operation, Hive will encounter a bug with stack trace like Caused by: java.lang.InstantiationException: org.apache.hadoop.hive.ql.parse.ASTNodeOrigin at java.lang.Class.newInstance0(Class.java:359) at java.lang.Class.newInstance(Class.java:327) at sun.reflect.GeneratedMethodAccessor84.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6980) Drop table by using direct sql
[ https://issues.apache.org/jira/browse/HIVE-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986240#comment-13986240 ] Sergey Shelukhin commented on HIVE-6980: Hmm... I don't think deleting via cascade will be much faster, DN will probably issue separate queries just like for selects. Batch delete could be done w/join or subquery, that is what I had in mind in original jira Drop table by using direct sql -- Key: HIVE-6980 URL: https://issues.apache.org/jira/browse/HIVE-6980 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.12.0 Reporter: Selina Zhang Assignee: Selina Zhang Dropping table which has lots of partitions is slow. Even after applying the patch of HIVE-6265, the drop table still takes hours (100K+ partitions). The fixes come with two parts: 1. use directSQL to query the partitions protect mode; the current implementation needs to transfer the Partition object to client and check the protect mode for each partition. I'd like to move this part of logic to metastore. The check will be done by direct sql (if direct sql is disabled, execute the same logic in the ObjectStore); 2. use directSQL to drop partitions for table; there maybe two solutions here: 1. add DELETE CASCADE in the schema. In this way we only need to delete entries from partitions table use direct sql. May need to change datanucleus.deletionPolicy = DataNucleus. 2. clean up the dependent tables by issue DELETE statement. This also needs to turn on datanucleus.query.sql.allowAll Both of above solutions should be able to fix the problem. The DELETE CASCADE has to change schemas and prepare upgrade scripts. The second solutions added maintenance cost if new tables added in the future releases. Please advice. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6928) Beeline should not chop off describe extended results by default
[
https://issues.apache.org/jira/browse/HIVE-6928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986266#comment-13986266
]
Hive QA commented on HIVE-6928:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642628/HIVE-6928.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 5426 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/89/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/89/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12642628
Beeline should not chop off describe extended results by default
--
Key: HIVE-6928
URL: https://issues.apache.org/jira/browse/HIVE-6928
Project: Hive
Issue Type: Bug
Components: CLI
Reporter: Szehon Ho
Assignee: Chinna Rao Lalam
Attachments: HIVE-6928.patch
By default, beeline truncates long results based on the console width like:
+-+--+
| col_name |
|
+-+--+
| pat_id | string
|
| score | float
|
| acutes | float
|
| |
|
| Detailed Table Information | Table(tableName:refills, dbName:default,
owner:hdadmin, createTime:1393882396, lastAccessTime:0, retention:0, sd:Sto |
+-+--+
5 rows selected (0.4 seconds)
This can be changed by !outputformat, but the default should behave better to
give a better experience to the first-time beeline user.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6982) Export all .sh equivalent for windows (.cmd files) in bin, bin/ext
[ https://issues.apache.org/jira/browse/HIVE-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6982: --- Resolution: Fixed Fix Version/s: 0.14.0 Release Note: Committed to trunk. Thanks, Hari! Status: Resolved (was: Patch Available) Export all .sh equivalent for windows (.cmd files) in bin, bin/ext -- Key: HIVE-6982 URL: https://issues.apache.org/jira/browse/HIVE-6982 Project: Hive Issue Type: Sub-task Components: Windows Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Fix For: 0.14.0 Attachments: HIVE-6982.1.patch, HIVE-6982.2.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6966) More fixes for TestCliDriver on Windows
[ https://issues.apache.org/jira/browse/HIVE-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6966: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Jason! More fixes for TestCliDriver on Windows --- Key: HIVE-6966 URL: https://issues.apache.org/jira/browse/HIVE-6966 Project: Hive Issue Type: Bug Components: Tests, Windows Reporter: Jason Dere Assignee: Jason Dere Fix For: 0.14.0 Attachments: HIVE-6966.1.patch This prevents the Test*CliDriver tests from compiling properly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6982) Export all .sh equivalent for windows (.cmd files) in bin, bin/ext
[ https://issues.apache.org/jira/browse/HIVE-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6982: --- Release Note: (was: Committed to trunk. Thanks, Hari!) Export all .sh equivalent for windows (.cmd files) in bin, bin/ext -- Key: HIVE-6982 URL: https://issues.apache.org/jira/browse/HIVE-6982 Project: Hive Issue Type: Sub-task Components: Windows Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Fix For: 0.14.0 Attachments: HIVE-6982.1.patch, HIVE-6982.2.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6982) Export all .sh equivalent for windows (.cmd files) in bin, bin/ext
[ https://issues.apache.org/jira/browse/HIVE-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986268#comment-13986268 ] Sushanth Sowmyan commented on HIVE-6982: Committed to trunk. Thanks, Hari! Export all .sh equivalent for windows (.cmd files) in bin, bin/ext -- Key: HIVE-6982 URL: https://issues.apache.org/jira/browse/HIVE-6982 Project: Hive Issue Type: Sub-task Components: Windows Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Fix For: 0.14.0 Attachments: HIVE-6982.1.patch, HIVE-6982.2.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6931) Windows unit test fixes
[ https://issues.apache.org/jira/browse/HIVE-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6931: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Jason! Windows unit test fixes --- Key: HIVE-6931 URL: https://issues.apache.org/jira/browse/HIVE-6931 Project: Hive Issue Type: Bug Components: Tests, Windows Reporter: Jason Dere Assignee: Jason Dere Fix For: 0.14.0 Attachments: HIVE-6931.1.patch, HIVE-6931.1.patch A few misc fixes for some of the unit tests on Windows. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6741) HiveServer2 startup fails in secure (kerberos) mode due to backward incompatible hadoop change
[
https://issues.apache.org/jira/browse/HIVE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986279#comment-13986279
]
Jason Dere commented on HIVE-6741:
--
These test failures do not look related to this patch.
HiveServer2 startup fails in secure (kerberos) mode due to backward
incompatible hadoop change
--
Key: HIVE-6741
URL: https://issues.apache.org/jira/browse/HIVE-6741
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
Priority: Blocker
Fix For: 0.14.0
Attachments: HIVE-6741.1.patch, HIVE-6741.2.patch, HIVE-6741.3.patch
[HADOOP-10211|https://issues.apache.org/jira/browse/HADOOP-10211] made a
backward incompatible change due to which the following hive call returns a
null map
([HiveAuthFactory-old|https://github.com/apache/hive/blob/fc3fdb19668369c56994d11df3207e14f2c5dba8/service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java#L115]):
{code}
MapString, String hadoopSaslProps = ShimLoader.getHadoopThriftAuthBridge().
getHadoopSaslProperties(conf);
SaslQOP hadoopSaslQOP = SaslQOP.fromString(hadoopSaslProps.get(Sasl.QOP));
if(hadoopSaslQOP.ordinal() saslQOP.ordinal()) {
LOG.warn(MessageFormat.format(\hadoop.rpc.protection\ is set to higher
security level +
{0} then {1} which is set to {2}, hadoopSaslQOP.toString(),
ConfVars.HIVE_SERVER2_THRIFT_SASL_QOP.varname, saslQOP.toString()));
}
{code}
Since this code path is only used for logging hadoop sasl qop values in case
hadoop's qop hive's qop, we can do away with this and add a general log
message.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6960) Set Hive pom to use Hadoop-2.4
[ https://issues.apache.org/jira/browse/HIVE-6960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6960: - Attachment: HIVE-6960.1.patch re-upload patch and try tests again Set Hive pom to use Hadoop-2.4 -- Key: HIVE-6960 URL: https://issues.apache.org/jira/browse/HIVE-6960 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.14.0 Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6960.1.patch, HIVE-6960.1.patch A number of the hadoop-2 unit test failures are due to HADOOP-10425, fixed in Hadoop 2.4. Perhaps we should move onto that version. - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3 - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4 - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database - org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6741) HiveServer2 startup fails in secure (kerberos) mode due to backward incompatible hadoop change
[
https://issues.apache.org/jira/browse/HIVE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-6741:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks for fixing Vaibhav.
HiveServer2 startup fails in secure (kerberos) mode due to backward
incompatible hadoop change
--
Key: HIVE-6741
URL: https://issues.apache.org/jira/browse/HIVE-6741
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
Priority: Blocker
Fix For: 0.14.0
Attachments: HIVE-6741.1.patch, HIVE-6741.2.patch, HIVE-6741.3.patch
[HADOOP-10211|https://issues.apache.org/jira/browse/HADOOP-10211] made a
backward incompatible change due to which the following hive call returns a
null map
([HiveAuthFactory-old|https://github.com/apache/hive/blob/fc3fdb19668369c56994d11df3207e14f2c5dba8/service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java#L115]):
{code}
MapString, String hadoopSaslProps = ShimLoader.getHadoopThriftAuthBridge().
getHadoopSaslProperties(conf);
SaslQOP hadoopSaslQOP = SaslQOP.fromString(hadoopSaslProps.get(Sasl.QOP));
if(hadoopSaslQOP.ordinal() saslQOP.ordinal()) {
LOG.warn(MessageFormat.format(\hadoop.rpc.protection\ is set to higher
security level +
{0} then {1} which is set to {2}, hadoopSaslQOP.toString(),
ConfVars.HIVE_SERVER2_THRIFT_SASL_QOP.varname, saslQOP.toString()));
}
{code}
Since this code path is only used for logging hadoop sasl qop values in case
hadoop's qop hive's qop, we can do away with this and add a general log
message.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6741) HiveServer2 startup fails in secure (kerberos) mode due to backward incompatible hadoop change
[
https://issues.apache.org/jira/browse/HIVE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986297#comment-13986297
]
Vaibhav Gumashta commented on HIVE-6741:
Thanks for review [~jdere].
HiveServer2 startup fails in secure (kerberos) mode due to backward
incompatible hadoop change
--
Key: HIVE-6741
URL: https://issues.apache.org/jira/browse/HIVE-6741
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
Priority: Blocker
Fix For: 0.14.0
Attachments: HIVE-6741.1.patch, HIVE-6741.2.patch, HIVE-6741.3.patch
[HADOOP-10211|https://issues.apache.org/jira/browse/HADOOP-10211] made a
backward incompatible change due to which the following hive call returns a
null map
([HiveAuthFactory-old|https://github.com/apache/hive/blob/fc3fdb19668369c56994d11df3207e14f2c5dba8/service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java#L115]):
{code}
MapString, String hadoopSaslProps = ShimLoader.getHadoopThriftAuthBridge().
getHadoopSaslProperties(conf);
SaslQOP hadoopSaslQOP = SaslQOP.fromString(hadoopSaslProps.get(Sasl.QOP));
if(hadoopSaslQOP.ordinal() saslQOP.ordinal()) {
LOG.warn(MessageFormat.format(\hadoop.rpc.protection\ is set to higher
security level +
{0} then {1} which is set to {2}, hadoopSaslQOP.toString(),
ConfVars.HIVE_SERVER2_THRIFT_SASL_QOP.varname, saslQOP.toString()));
}
{code}
Since this code path is only used for logging hadoop sasl qop values in case
hadoop's qop hive's qop, we can do away with this and add a general log
message.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Review Request 18936: HIVE-6430 MapJoin hash table has large memory overhead
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18936/#review41892 --- conf/hive-default.xml.template https://reviews.apache.org/r/18936/#comment75550 dup, will remove - Sergey Shelukhin On April 30, 2014, 11:14 p.m., Sergey Shelukhin wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18936/ --- (Updated April 30, 2014, 11:14 p.m.) Review request for hive, Gopal V and Gunther Hagleitner. Repository: hive-git Description --- See JIRA Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 0efe58b conf/hive-default.xml.template 6620bd1 hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java 5fe35a5 hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java 142bfd8 ql/src/java/org/apache/hadoop/hive/ql/Driver.java bf9d4c1 ql/src/java/org/apache/hadoop/hive/ql/debug/Utils.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java f5d4670 ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java b93ea7a ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 175d3ab ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/AbstractMapJoinTableContainer.java 8854b19 ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/BytesBytesMultiHashMap.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HashMapWrapper.java 9df425b ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java 64f0be2 ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinPersistableTableContainer.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinRowContainer.java 008a8db ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainer.java 988959f ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainerSerDe.java 55b7415 ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HashTableLoader.java e392592 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapJoinOperator.java eef7656 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedColumnarSerDe.java d4be78d ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java 3077d75 ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java f7b499b ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 157d072 ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 118b339 ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestBytesBytesMultiHashMap.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestMapJoinEqualityTableContainer.java 65e3779 ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestMapJoinTableContainer.java 093da55 ql/src/test/queries/clientpositive/mapjoin_decimal.q b65a7be ql/src/test/queries/clientpositive/mapjoin_mapjoin.q 1eb95f6 ql/src/test/queries/clientpositive/tez_union.q f80d94c ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out 8350670 ql/src/test/results/clientpositive/tez/mapjoin_decimal.q.out 3c55b5c ql/src/test/results/clientpositive/tez/mapjoin_mapjoin.q.out 284cc03 serde/src/java/org/apache/hadoop/hive/serde2/ByteStream.java 73d9b29 serde/src/java/org/apache/hadoop/hive/serde2/WriteBuffers.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 9079b9d serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/OutputByteBuffer.java 1b09d41 serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java 5870884 serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java bab505e serde/src/java/org/apache/hadoop/hive/serde2/dynamic_type/DynamicSerDe.java 6f344bb serde/src/java/org/apache/hadoop/hive/serde2/io/DateWritable.java 1f4ccdd serde/src/java/org/apache/hadoop/hive/serde2/io/HiveDecimalWritable.java a99c7b4 serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java 435d6c6 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazySimpleSerDe.java 82c1263 serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java b188c3f serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryStruct.java caf3517 serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 6c14081
[jira] [Updated] (HIVE-6430) MapJoin hash table has large memory overhead
[ https://issues.apache.org/jira/browse/HIVE-6430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-6430: --- Attachment: HIVE-6430.12.patch fix small bug MapJoin hash table has large memory overhead Key: HIVE-6430 URL: https://issues.apache.org/jira/browse/HIVE-6430 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-6430.01.patch, HIVE-6430.02.patch, HIVE-6430.03.patch, HIVE-6430.04.patch, HIVE-6430.05.patch, HIVE-6430.06.patch, HIVE-6430.07.patch, HIVE-6430.08.patch, HIVE-6430.09.patch, HIVE-6430.10.patch, HIVE-6430.11.patch, HIVE-6430.12.patch, HIVE-6430.12.patch, HIVE-6430.patch Right now, in some queries, I see that storing e.g. 4 ints (2 for key and 2 for row) can take several hundred bytes, which is ridiculous. I am reducing the size of MJKey and MJRowContainer in other jiras, but in general we don't need to have java hash table there. We can either use primitive-friendly hashtable like the one from HPPC (Apache-licenced), or some variation, to map primitive keys to single row storage structure without an object per row (similar to vectorization). -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18936: HIVE-6430 MapJoin hash table has large memory overhead
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18936/ --- (Updated May 1, 2014, 2:29 a.m.) Review request for hive, Gopal V and Gunther Hagleitner. Repository: hive-git Description --- See JIRA Diffs (updated) - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 604bea7 conf/hive-default.xml.template 2552560 hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java 5fe35a5 hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java 142bfd8 ql/src/java/org/apache/hadoop/hive/ql/Driver.java bf9d4c1 ql/src/java/org/apache/hadoop/hive/ql/debug/Utils.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java f5d4670 ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java b93ea7a ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 175d3ab ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/AbstractMapJoinTableContainer.java 8854b19 ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/BytesBytesMultiHashMap.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HashMapWrapper.java 9df425b ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java 64f0be2 ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinPersistableTableContainer.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinRowContainer.java 008a8db ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainer.java 988959f ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainerSerDe.java 55b7415 ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HashTableLoader.java e392592 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapJoinOperator.java eef7656 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedColumnarSerDe.java d4be78d ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java 3077d75 ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java f7b499b ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 157d072 ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 118b339 ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestBytesBytesMultiHashMap.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestMapJoinEqualityTableContainer.java 65e3779 ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/TestMapJoinTableContainer.java 093da55 ql/src/test/queries/clientpositive/mapjoin_decimal.q b65a7be ql/src/test/queries/clientpositive/mapjoin_mapjoin.q 1eb95f6 ql/src/test/queries/clientpositive/tez_union.q f80d94c ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out 8350670 ql/src/test/results/clientpositive/tez/mapjoin_decimal.q.out 3c55b5c ql/src/test/results/clientpositive/tez/mapjoin_mapjoin.q.out 284cc03 serde/src/java/org/apache/hadoop/hive/serde2/ByteStream.java 73d9b29 serde/src/java/org/apache/hadoop/hive/serde2/WriteBuffers.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 9079b9d serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/OutputByteBuffer.java 1b09d41 serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java 5870884 serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java bab505e serde/src/java/org/apache/hadoop/hive/serde2/dynamic_type/DynamicSerDe.java 6f344bb serde/src/java/org/apache/hadoop/hive/serde2/io/DateWritable.java 1f4ccdd serde/src/java/org/apache/hadoop/hive/serde2/io/HiveDecimalWritable.java a99c7b4 serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java 435d6c6 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazySimpleSerDe.java 82c1263 serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java b188c3f serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryStruct.java caf3517 serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 6c14081 serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/objectinspector/LazyBinaryStructObjectInspector.java e5ea452 serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 06d5c5e serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java 868dd4c serde/src/test/org/apache/hadoop/hive/serde2/thrift_test/CreateSequenceFile.java 1fb49e5 Diff: https://reviews.apache.org/r/18936/diff/ Testing --- Thanks, Sergey Shelukhin
[jira] [Created] (HIVE-6997) Semantic Exception for string comparison filter
Johndee Burks created HIVE-6997: --- Summary: Semantic Exception for string comparison filter Key: HIVE-6997 URL: https://issues.apache.org/jira/browse/HIVE-6997 Project: Hive Issue Type: New Feature Reporter: Johndee Burks Priority: Minor -- Consider that you have the following dataset: one,1 two,2 three,three -- The DDL is the following: create table j1 (a string, b string) row format delimited fields terminated by ','; -- If you do a filter on column b like this: select * from j1 where b!=1; -- I would expect to see the data from row 2 and 3. However you only see row 2. If you make the following change to the query you get the correct results: select * from j1 where b!='1'; -- Looking at an explain of both queries it appears that in the query without the quotes the 1 is being changed to 1.0 and then being compared to the string three incorrectly typed so the result is filtered out. I think that this should be improved with a Semantic exception when type of the column is string and the value being passed in the filter is not in quotes. == Without Quotes STAGE PLANS: Stage: Stage-1 Map Reduce Alias - Map Operator Tree: j1 TableScan alias: j1 Filter Operator predicate: expr: (b 1.0) type: boolean Select Operator expressions: expr: a type: string expr: b type: string outputColumnNames: _col0, _col1 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat == With Quotes STAGE PLANS: Stage: Stage-1 Map Reduce Alias - Map Operator Tree: j1 TableScan alias: j1 Filter Operator predicate: expr: (b '1') type: boolean Select Operator expressions: expr: a type: string expr: b type: string outputColumnNames: _col0, _col1 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6991) History is not able to disable/enable after session is started
[
https://issues.apache.org/jira/browse/HIVE-6991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986338#comment-13986338
]
Hive QA commented on HIVE-6991:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642636/HIVE-6991.patch
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 5426 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/90/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/90/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12642636
History is not able to disable/enable after session is started
--
Key: HIVE-6991
URL: https://issues.apache.org/jira/browse/HIVE-6991
Project: Hive
Issue Type: Bug
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
Attachments: HIVE-6991.patch
By default history is disabled, after session started if enable history
through this command set hive.session.history.enabled=true. It is not working.
I think it will help to this user query
http://mail-archives.apache.org/mod_mbox/hive-user/201404.mbox/%3ccajqy7afapa_pjs6buon0o8zyt2qwfn2wt-mtznwfmurav_8...@mail.gmail.com%3E
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6994) parquet-hive createArray strips null elements
[
https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986339#comment-13986339
]
Hive QA commented on HIVE-6994:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642684/HIVE-6994.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/91/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/91/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-91/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/Driver.java'
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target
shims/0.20S/target shims/0.23/target shims/aggregator/target
shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target
itests/target itests/hcatalog-unit/target itests/test-serde/target
itests/qtest/target itests/hive-minikdc/target itests/hive-unit/target
itests/custom-serde/target itests/util/target hcatalog/target
hcatalog/core/target hcatalog/streaming/target
hcatalog/server-extensions/target hcatalog/webhcat/svr/target
hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target
hwi/target common/target common/src/gen service/target contrib/target
serde/target beeline/target odbc/target cli/target
ql/dependency-reduced-pom.xml ql/target
+ svn update
Umetastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
Abin/beeline.cmd
Abin/hive.cmd
Abin/derbyserver.cmd
Abin/ext/hiveserver2.cmd
Abin/ext/lineage.cmd
Abin/ext/cli.cmd
Abin/ext/util/execHiveCmd.cmd
Abin/ext/jar.cmd
Abin/ext/hiveserver.cmd
Abin/ext/debug.cmd
Abin/ext/hwi.cmd
Abin/ext/help.cmd
Abin/ext/rcfilecat.cmd
Abin/ext/schemaTool.cmd
Abin/ext/metastore.cmd
Abin/hive-config.cmd
Uql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java
Uql/src/test/org/apache/hadoop/hive/ql/WindowsPathUtil.java
Uql/src/test/queries/clientpositive/scriptfile1_win.q
Uql/src/test/queries/clientpositive/tez_insert_overwrite_local_directory_1.q
Uql/src/test/results/clientpositive/scriptfile1_win.q.out
Uconf/hive-default.xml.template
Uservice/src/java/org/apache/hive/service/auth/HiveAuthFactory.java
Uitests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java
Uant/src/org/apache/hadoop/hive/ant/QTestGenTask.java
Ucommon/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Upom.xml
Fetching external item into 'hcatalog/src/test/e2e/harness'
Updated external to revision 1591545.
Updated to revision 1591545.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12642684
parquet-hive createArray strips null elements
-
Key: HIVE-6994
URL: https://issues.apache.org/jira/browse/HIVE-6994
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0, 0.14.0
Reporter: Justin Coffey
Assignee: Justin Coffey
Fix For: 0.14.0
Attachments: HIVE-6994.patch
The createArray method in ParquetHiveSerDe strips null values from resultant
ArrayWritables.
tracked here as well: https://github.com/Parquet/parquet-mr/issues/377
--
This message was sent by Atlassian JIRA
(v6.2#6252)
