[jira] [Updated] (HIVE-4303) 2 empty java files in hcatalog
[ https://issues.apache.org/jira/browse/HIVE-4303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-4303: Status: Patch Available (was: Open) 2 empty java files in hcatalog -- Key: HIVE-4303 URL: https://issues.apache.org/jira/browse/HIVE-4303 Project: Hive Issue Type: Bug Components: HCatalog Reporter: Owen O'Malley Assignee: Owen O'Malley Two empty java files came in from hcatalog. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4198) Move HCatalog code into Hive
[ https://issues.apache.org/jira/browse/HIVE-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624360#comment-13624360 ] Carl Steinbach commented on HIVE-4198: -- The plan of record is for HCatalog to be one of the artifacts contained in the Hive release. Move HCatalog code into Hive Key: HIVE-4198 URL: https://issues.apache.org/jira/browse/HIVE-4198 Project: Hive Issue Type: Task Components: HCatalog Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.11.0 The HCatalog code needs to be moved into Hive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3840) hive cli null representation in output is inconsistent
[ https://issues.apache.org/jira/browse/HIVE-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-3840: --- Resolution: Fixed Fix Version/s: 0.11.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Thejas! hive cli null representation in output is inconsistent -- Key: HIVE-3840 URL: https://issues.apache.org/jira/browse/HIVE-3840 Project: Hive Issue Type: Bug Reporter: Ashutosh Chauhan Assignee: Thejas M Nair Fix For: 0.11.0 Attachments: HIVE-3840.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3985) Update new UDAFs introduced for Windowing to work with new Decimal Type
[ https://issues.apache.org/jira/browse/HIVE-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-3985: --- Resolution: Fixed Fix Version/s: 0.11.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Brock! Update new UDAFs introduced for Windowing to work with new Decimal Type --- Key: HIVE-3985 URL: https://issues.apache.org/jira/browse/HIVE-3985 Project: Hive Issue Type: Bug Components: PTF-Windowing Affects Versions: 0.11.0 Reporter: Harish Butani Assignee: Brock Noland Fix For: 0.11.0 Attachments: HIVE-3985-0.patch, HIVE-3985-1.patch, HIVE-3985-2.patch, HIVE-3985-3.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4303) 2 empty java files in hcatalog
[ https://issues.apache.org/jira/browse/HIVE-4303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624443#comment-13624443 ] Ashutosh Chauhan commented on HIVE-4303: $ find . -size 0 | grep hcatalog doesn't return anything on latest trunk. 2 empty java files in hcatalog -- Key: HIVE-4303 URL: https://issues.apache.org/jira/browse/HIVE-4303 Project: Hive Issue Type: Bug Components: HCatalog Reporter: Owen O'Malley Assignee: Owen O'Malley Two empty java files came in from hcatalog. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: next hive release
Cool. We have made a good progress since we had this discussion. Windowing and analytical functions are now in trunk. HCatalog merger is done, though we are still working on finer details there. Hiveserver2 is stabilizing. I think its a good time to branch to stabilize for next release. Unless someone thinks otherwise, I will create a 0.11 branch in next few days. Ashutosh On Fri, Mar 22, 2013 at 2:18 PM, Alan Gates ga...@hortonworks.com wrote: On Mar 22, 2013, at 1:17 PM, Carl Steinbach wrote: At one of the previous Hive Contributor meetings we agreed to do time-based releases every four months. I still think this is a worthwhile goal and strongly support releasing 0.11.0 soon. Question for the HCatalog developers: Is it feasible to have HCatalog ready in time for an April release. If not, how should we handle this? I think we can have basic integration done by April 1, so Hive 0.11 will go out with hcat jars that work with it. Alan. Thanks. Carl On Fri, Mar 22, 2013 at 12:41 PM, Brock Noland br...@cloudera.com wrote: Hi, I have just marked https://issues.apache.org/jira/browse/HIVE-2264 as a blocker for the 0.11 release as it's essential for HS2 use. The patch is available, a committer has +1'd, it's just waiting on commit. It'd be great to get https://issues.apache.org/jira/browse/HIVE-4128in as well to support the new decimal type. Brock On Fri, Mar 22, 2013 at 2:12 PM, Amir Sanjar v1san...@us.ibm.com wrote: I agree. Let me know how we can contribute towards accelerating this process. We are anxiously waiting for release of HIVE 0.11.0 Best Regards Amir Sanjar System Management Architect PowerLinux Open Source Hadoop development lead IBM Senior Software Engineer Phone# 512-286-8393 Fax# 512-838-8858 [image: Inactive hide details for Ashutosh Chauhan ---03/22/2013 01:32:40 PM---Hey all, Its not been long since we released 0.10 but a]Ashutosh Chauhan ---03/22/2013 01:32:40 PM---Hey all, Its not been long since we released 0.10 but a lot has gone in since then. From: Ashutosh Chauhan hashut...@apache.org To: dev@hive.apache.org, Date: 03/22/2013 01:32 PM Subject: next hive release -- Hey all, Its not been long since we released 0.10 but a lot has gone in since then. There are quite a few query plan optimizations which has gone in. Also, ptf-windowing work is winding down and should be ready for merge in a week or so. After the merge, I think it will be a good time to create a branch for 0.11 and than start to stabilize it. I propose we cut the branch in first week of April. What do other folks think? Thanks, Ashutosh -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
[jira] [Commented] (HIVE-2597) Repeated key in GROUP BY is erroneously displayed when using DISTINCT
[ https://issues.apache.org/jira/browse/HIVE-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624484#comment-13624484 ] Phabricator commented on HIVE-2597: --- ashutoshc has accepted the revision HIVE-2597 [jira] Repeated key in GROUP BY is erroneously displayed when using DISTINCT. +1 will commit if tests pass. REVISION DETAIL https://reviews.facebook.net/D8967 BRANCH HIVE-2597 ARCANIST PROJECT hive To: JIRA, ashutoshc, navis Cc: njain Repeated key in GROUP BY is erroneously displayed when using DISTINCT - Key: HIVE-2597 URL: https://issues.apache.org/jira/browse/HIVE-2597 Project: Hive Issue Type: Bug Reporter: Alex Rovner Assignee: Navis Attachments: HIVE-2597.D8967.1.patch, HIVE-2597.D8967.2.patch The following query was simplified for illustration purposes. This works correctly: select client_tid, as myvalue1, as myvalue2 from clients cluster by client_tid The intent here is to produce two empty columns in between data. The following query does not work: select distinct client_tid, as myvalue1, as myvalue2 from clients cluster by client_tid FAILED: Error in semantic analysis: Line 1:44 Repeated key in GROUP BY The key is not repeated since the aliases were given. Seems like Hive is ignoring the aliases when the distinct keyword is specified. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3908) create view statement's outputs contains the view and a temporary dir.
[ https://issues.apache.org/jira/browse/HIVE-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624488#comment-13624488 ] Ashutosh Chauhan commented on HIVE-3908: +1 will commit if tests pass. create view statement's outputs contains the view and a temporary dir. -- Key: HIVE-3908 URL: https://issues.apache.org/jira/browse/HIVE-3908 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Namit Jain Assignee: Prasad Mujumdar Attachments: HIVE-3908-1.patch It should only contain the view -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4299) exported metadata by HIVE-3068 cannot be imported because of wrong file name
[
https://issues.apache.org/jira/browse/HIVE-4299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624492#comment-13624492
]
Ashutosh Chauhan commented on HIVE-4299:
[~sho.shimauchi] Patch looks good. Can you also add a .q testcase for this?
exported metadata by HIVE-3068 cannot be imported because of wrong file name
Key: HIVE-4299
URL: https://issues.apache.org/jira/browse/HIVE-4299
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.11.0
Reporter: Sho Shimauchi
Assignee: Sho Shimauchi
Attachments: HIVE-4299.patch
h2. Symptom
When DROP TABLE a table, metadata of the table is generated to be able to
import the dropped table again.
However, the exported metadata name is 'table name.metadata'.
Since ImportSemanticAnalyzer allows only '_metadata' as metadata filename,
user have to rename the metadata file to import the table.
h2. How to reproduce
Set the following setting to hive-site.xml:
{code}
property
namehive.metastore.pre.event.listeners/name
valueorg.apache.hadoop.hive.ql.parse.MetaDataExportListener/value
/property
{code}
Then run the following queries:
{code}
CREATE TABLE test_table (id INT, name STRING);
DROP TABLE test_table;
IMPORT TABLE test_table_imported FROM '/path/to/metadata/file';
FAILED: SemanticException [Error 10027]: Invalid path
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4172) JDBC2 does not support VOID type
[
https://issues.apache.org/jira/browse/HIVE-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624496#comment-13624496
]
Phabricator commented on HIVE-4172:
---
ashutoshc has accepted the revision HIVE-4172 [jira] JDBC2 does not support
VOID type.
+1 will commit if tests pass.
REVISION DETAIL
https://reviews.facebook.net/D9555
BRANCH
HIVE-4172
ARCANIST PROJECT
hive
To: JIRA, ashutoshc, navis
Cc: prasadm
JDBC2 does not support VOID type
Key: HIVE-4172
URL: https://issues.apache.org/jira/browse/HIVE-4172
Project: Hive
Issue Type: Improvement
Components: HiveServer2, JDBC
Reporter: Navis
Assignee: Navis
Priority: Minor
Labels: HiveServer2
Attachments: HIVE-4172.D9555.1.patch, HIVE-4172.D9555.2.patch
In beeline, select key, null from src fails with exception,
{noformat}
org.apache.hive.service.cli.HiveSQLException: Error running query:
java.lang.NullPointerException
at
org.apache.hive.service.cli.operation.SQLOperation.run(SQLOperation.java:112)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:166)
at
org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:148)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:183)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1133)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1118)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:39)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4172) JDBC2 does not support VOID type
[
https://issues.apache.org/jira/browse/HIVE-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624498#comment-13624498
]
Ashutosh Chauhan commented on HIVE-4172:
Sorry..patch has gone stale. Can you rebase it.
JDBC2 does not support VOID type
Key: HIVE-4172
URL: https://issues.apache.org/jira/browse/HIVE-4172
Project: Hive
Issue Type: Improvement
Components: HiveServer2, JDBC
Reporter: Navis
Assignee: Navis
Priority: Minor
Labels: HiveServer2
Attachments: HIVE-4172.D9555.1.patch, HIVE-4172.D9555.2.patch
In beeline, select key, null from src fails with exception,
{noformat}
org.apache.hive.service.cli.HiveSQLException: Error running query:
java.lang.NullPointerException
at
org.apache.hive.service.cli.operation.SQLOperation.run(SQLOperation.java:112)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:166)
at
org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:148)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:183)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1133)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1118)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:39)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4302) Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp
[ https://issues.apache.org/jira/browse/HIVE-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-4302: -- Attachment: HIVE-4302.D10005.1.patch hbutani requested code review of HIVE-4302 [jira] Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp. Reviewers: JIRA create RS RR during genPTFPlan Currently the RowSchema and RowResolver for the ReduceSinkOp just point to the 'input' Op's structures. This causes issues when input Op's structures are changed during Optimization. See Jira 2340 of a problem. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D10005 AFFECTED FILES data/files/flights_tiny.txt data/files/part.rc data/files/part.seq ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/23925/ To: JIRA, hbutani Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp - Key: HIVE-4302 URL: https://issues.apache.org/jira/browse/HIVE-4302 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-4302.D10005.1.patch Currently the RowSchema and RowResolver for the ReduceSinkOp just point to the 'input' Op's structures. This causes issues when input Op's structures are changed during Optimization. See Jira 2340 of a problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4302) Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp
[ https://issues.apache.org/jira/browse/HIVE-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624504#comment-13624504 ] Harish Butani commented on HIVE-4302: - Haven't made the change to ColumnPrunerFactory in the patch as 2340 is not resolved yet. Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp - Key: HIVE-4302 URL: https://issues.apache.org/jira/browse/HIVE-4302 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-4302.D10005.1.patch Currently the RowSchema and RowResolver for the ReduceSinkOp just point to the 'input' Op's structures. This causes issues when input Op's structures are changed during Optimization. See Jira 2340 of a problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4306) PTFDeserializer should reconstruct OIs based on InputOI passed to PTFOperator
Harish Butani created HIVE-4306: --- Summary: PTFDeserializer should reconstruct OIs based on InputOI passed to PTFOperator Key: HIVE-4306 URL: https://issues.apache.org/jira/browse/HIVE-4306 Project: Hive Issue Type: Improvement Components: PTF-Windowing Reporter: Harish Butani Assignee: Prajakta Kalmegh Currently PTFDesc holds onto shape information that is used by the PTFDeserializer to reconstruct OIs during runtime. This could interfere with changes made to OIs during Optimization. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4220) TimestampWritable.toString throws array index exception sometimes
[
https://issues.apache.org/jira/browse/HIVE-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624532#comment-13624532
]
Ashutosh Chauhan commented on HIVE-4220:
Since there is no test-case, I am having hard time envisioning the scenario
where this might occur. Is it that there is a concurrent access of this
non-thread-safe formatter where this will occur? Explaining the scenario where
this bug will showup will be useful.
TimestampWritable.toString throws array index exception sometimes
-
Key: HIVE-4220
URL: https://issues.apache.org/jira/browse/HIVE-4220
Project: Hive
Issue Type: Bug
Reporter: Navis
Assignee: Navis
Attachments: HIVE-4220.D9669.1.patch
{noformat}
org.apache.hive.service.cli.HiveSQLException: java.io.IOException:
java.lang.ArrayIndexOutOfBoundsException: 45
at
org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:215)
at
org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:170)
at
org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:288)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:348)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 45
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:194)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1449)
at
org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:193)
... 11 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 45
at
sun.util.calendar.BaseCalendar.getCalendarDateFromFixedDate(BaseCalendar.java:436)
at
java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2081)
at
java.util.GregorianCalendar.computeFields(GregorianCalendar.java:1996)
at java.util.Calendar.setTimeInMillis(Calendar.java:1110)
at java.util.Calendar.setTime(Calendar.java:1076)
at java.text.SimpleDateFormat.format(SimpleDateFormat.java:875)
at java.text.SimpleDateFormat.format(SimpleDateFormat.java:868)
at java.text.DateFormat.format(DateFormat.java:316)
at
org.apache.hadoop.hive.serde2.io.TimestampWritable.toString(TimestampWritable.java:327)
at
org.apache.hadoop.hive.serde2.lazy.LazyTimestamp.writeUTF8(LazyTimestamp.java:95)
at
org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitiveUTF8(LazyUtils.java:234)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:427)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serializeField(LazySimpleSerDe.java:381)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:365)
at
org.apache.hadoop.hive.ql.exec.ListSinkOperator.processOp(ListSinkOperator.java:96)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:487)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:821)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:487)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:821)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:487)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:474)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:468)
at
org.apache.hadoop.hive.ql.exec.FetchTask.fetchAndPush(FetchTask.java:222)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:188)
... 13 more
{noformat}
data formatter in TimestampWritable is declared static and shared but it's
not thread-safe.
--
This
[jira] [Commented] (HIVE-4222) Timestamp type constants cannot be deserialized in JDK 1.6 or less
[
https://issues.apache.org/jira/browse/HIVE-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624548#comment-13624548
]
Ashutosh Chauhan commented on HIVE-4222:
I use jdk1.6 and have never seen this error. If I am getting this right, this
should occur in any query which accesses timestamp column from table, since we
need to serialize Timestamp objects in plan. I have used such queries on jdk1.6
but never encountered the error you mentioned.
Timestamp type constants cannot be deserialized in JDK 1.6 or less
--
Key: HIVE-4222
URL: https://issues.apache.org/jira/browse/HIVE-4222
Project: Hive
Issue Type: Bug
Components: Types
Reporter: Navis
Assignee: Navis
Attachments: HIVE-4222.D9681.1.patch
For example,
{noformat}
ExprNodeConstantDesc constant = new
ExprNodeConstantDesc(TypeInfoFactory.timestampTypeInfo, new Timestamp(100));
String serialized = Utilities.serializeExpression(constant);
ExprNodeConstantDesc deserilized = (ExprNodeConstantDesc)
Utilities.deserializeExpression(serialized, new Configuration());
{noformat}
logs error message
{noformat}
java.lang.InstantiationException: java.sql.Timestamp
Continuing ...
java.lang.RuntimeException: failed to evaluate: unbound=Class.new();
Continuing ...
{noformat}
and makes NPE in final.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4211) Common column and partition column are defined the same type and union them, it will hints Schema of both sides of union should match.
[ https://issues.apache.org/jira/browse/HIVE-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624553#comment-13624553 ] Ashutosh Chauhan commented on HIVE-4211: I remember this TODO was added by [~namit] while trying to fix something with partition column types handling. He will have more insights on how to handle these kind of scenarios. Common column and partition column are defined the same type and union them, it will hints Schema of both sides of union should match. --- Key: HIVE-4211 URL: https://issues.apache.org/jira/browse/HIVE-4211 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.9.0, 0.11.0 Reporter: Daisy.Yuan Labels: patch Attachments: PartitionColumnTypInfo.patch create table UnionBoolA (id boolean, no boolean) row format delimited fields terminated by ' '; load data local inpath '/opt/files/unionboola.txt' into table UnionBoolA; create table UnionPartionBool (id int) partitioned by (no boolean) row format delimited fields terminated by ' '; load data local inpath '/opt/files/unionpartint.txt' into table UnionPartionBool partition(no=true); unionboola.txt: true true false true true true false true unionpartint.txt: 111 444 1122 44 when I execute select * from( select no from UnionBoolA union all select no from UnionPartionBool) unionResult, it is failed. The exception info is as follows: FAILED: Error in semantic analysis: 1:66 Schema of both sides of union should match: Column no is of type boolean on first table and type string on second table. Error encountered near token 'UnionPartionBool' org.apache.hadoop.hive.ql.parse.SemanticException: 1:66 Schema of both sides of union should match: Column no is of type boolean on first table and type string on second table. Error encountered near token 'UnionPartionBool' at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genUnionPlan(SemanticAnalyzer.java:6295) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6733) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6748) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7556) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:244) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:621) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:525) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1153) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:226) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:630) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:618) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge20S.java:535) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge20S.java:532) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge20S.java:532) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) So I execute explain select no from UnionPartionBool to see the partition column, and find the partition column type is string. And all the partition column type is changed to be TypeInfoFactory.stringTypeInfo. It is in org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genTablePlan(). And it is todo task. Now I modify it to be TypeInfoFactory.getPrimitiveTypeInfo(part_col.getType()).It can fix this bug. And you can see what I modified in the patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see:
[jira] [Commented] (HIVE-1953) Hive should process comments in CliDriver
[ https://issues.apache.org/jira/browse/HIVE-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624557#comment-13624557 ] Ashutosh Chauhan commented on HIVE-1953: [~vikram.dixit] Can you create a phabricator / RB entry for the patch ? Hive should process comments in CliDriver - Key: HIVE-1953 URL: https://issues.apache.org/jira/browse/HIVE-1953 Project: Hive Issue Type: Improvement Reporter: He Yongqiang Assignee: Vikram Dixit K Attachments: HIVE-1953.1.patch, HIVE-1953.2.patch If you put commend before a set command, it will faile. Like this: -- TestSerDe is a user defined serde where the default delimiter is Ctrl-B -- the user is overwriting it with ctrlC set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat; Hive should process the comment in CliDriver, and ignore the comment right away, instead of passing it to the downstream processors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4306) PTFDeserializer should reconstruct OIs based on InputOI passed to PTFOperator
[ https://issues.apache.org/jira/browse/HIVE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-4306: -- Attachment: HIVE-4306.D10017.1.patch hbutani requested code review of HIVE-4306 [jira] PTFDeserializer should reconstruct OIs based on InputOI passed to PTFOperator. Reviewers: JIRA, ashutoshc compute OIs from inputOI during runtime Currently PTFDesc holds onto shape information that is used by the PTFDeserializer to reconstruct OIs during runtime. This could interfere with changes made to OIs during Optimization. TEST PLAN existing tests REVISION DETAIL https://reviews.facebook.net/D10017 AFFECTED FILES data/files/flights_tiny.txt data/files/part.rc data/files/part.seq ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java ql/src/java/org/apache/hadoop/hive/ql/plan/PTFDeserializer.java MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/23937/ To: JIRA, ashutoshc, hbutani PTFDeserializer should reconstruct OIs based on InputOI passed to PTFOperator - Key: HIVE-4306 URL: https://issues.apache.org/jira/browse/HIVE-4306 Project: Hive Issue Type: Improvement Components: PTF-Windowing Reporter: Harish Butani Assignee: Prajakta Kalmegh Attachments: HIVE-4306.D10017.1.patch Currently PTFDesc holds onto shape information that is used by the PTFDeserializer to reconstruct OIs during runtime. This could interfere with changes made to OIs during Optimization. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4302) Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp
[ https://issues.apache.org/jira/browse/HIVE-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624589#comment-13624589 ] Phabricator commented on HIVE-4302: --- hbutani has commented on the revision HIVE-4302 [jira] Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp. Will add the test INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:10543 No this is not about the having clause for windowing. This has to do with the GroupBy RR that can precede window processing. For Having expressions a Aggregation expression is added twice to the RR. This logic is to preserve this. The comment on line 10566 tries to explain what is going on. REVISION DETAIL https://reviews.facebook.net/D10005 BRANCH HIVE-4302 ARCANIST PROJECT hive To: JIRA, ashutoshc, hbutani Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp - Key: HIVE-4302 URL: https://issues.apache.org/jira/browse/HIVE-4302 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-4302.D10005.1.patch Currently the RowSchema and RowResolver for the ReduceSinkOp just point to the 'input' Op's structures. This causes issues when input Op's structures are changed during Optimization. See Jira 2340 of a problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4095) Add exchange partition in Hive
[ https://issues.apache.org/jira/browse/HIVE-4095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dheeraj Kumar Singh updated HIVE-4095: -- Description: (was: It would very useful to support exchange partition in hive, something similar to http://www.orafaq.com/node/2570 in Oracle. ) Add exchange partition in Hive -- Key: HIVE-4095 URL: https://issues.apache.org/jira/browse/HIVE-4095 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Namit Jain Assignee: Dheeraj Kumar Singh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4302) Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp
[ https://issues.apache.org/jira/browse/HIVE-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-4302: -- Attachment: HIVE-4302.D10005.2.patch hbutani updated the revision HIVE-4302 [jira] Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp. - adding test Reviewers: ashutoshc, JIRA REVISION DETAIL https://reviews.facebook.net/D10005 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D10005?vs=31353id=31383#toc AFFECTED FILES data/files/flights_tiny.txt data/files/part.rc data/files/part.seq ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java ql/src/test/queries/clientpositive/windowing_expressions.q ql/src/test/results/clientpositive/windowing_expressions.q.out To: JIRA, ashutoshc, hbutani Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp - Key: HIVE-4302 URL: https://issues.apache.org/jira/browse/HIVE-4302 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-4302.D10005.1.patch, HIVE-4302.D10005.2.patch Currently the RowSchema and RowResolver for the ReduceSinkOp just point to the 'input' Op's structures. This causes issues when input Op's structures are changed during Optimization. See Jira 2340 of a problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-hadoop2 - Build # 143 - Still Failing
Changes for Build #138 [namit] HIVE-4289 HCatalog build fails when behind a firewall (Samuel Yuan via namit) [namit] HIVE-4281 add hive.map.groupby.sorted.testmode (Namit via Gang Tim Liu) [hashutosh] Moving hcatalog site outside of trunk [hashutosh] Moving hcatalog branches outside of trunk [hashutosh] HIVE-4259 : SEL operator created with missing columnExprMap for unions (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-4156 : need to add protobuf classes to hive-exec.jar (Owen Omalley via Ashutosh Chauhan) [hashutosh] HIVE-3464 : Merging join tree may reorder joins which could be invalid (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4138 : ORC's union object inspector returns a type name that isn't parseable by TypeInfoUtils (Owen Omalley via Ashutosh Chauhan) [cws] HIVE-4119. ANALYZE TABLE ... COMPUTE STATISTICS FOR COLUMNS fails with NPE if the table is empty (Shreepadma Venugopalan via cws) [hashutosh] HIVE-4252 : hiveserver2 string representation of complex types are inconsistent with cli (Thejas Nair via Ashutosh Chauhan) [hashutosh] HIVE-4179 : NonBlockingOpDeDup does not merge SEL operators correctly (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-4269 : fix handling of binary type in hiveserver2, jdbc driver (Thejas Nair via Ashutosh Chauhan) [namit] HIVE-4174 Round UDF converts BigInts to double (Chen Chun via namit) [namit] HIVE-4240 optimize hive.enforce.bucketing and hive.enforce sorting insert (Gang Tim Liu via namit) [navis] HIVE-4288 Add IntelliJ project files files to .gitignore (Roshan Naik via Navis) [namit] HIVE-4272 partition wise metadata does not work for text files [hashutosh] HIVE-896 : Add LEAD/LAG/FIRST/LAST analytical windowing functions to Hive. (Harish Butani via Ashutosh Chauhan) [namit] HIVE-4260 union_remove_12, union_remove_13 are failing on hadoop2 (Gunther Hagleitner via namit) [hashutosh] HIVE-3951 : Allow Decimal type columns in Regex Serde (Mark Grover via Ashutosh Chauhan) [namit] HIVE-4270 bug in hive.map.groupby.sorted in the presence of multiple input partitions (Namit via Gang Tim Liu) [hashutosh] HIVE-3850 : hour() function returns 12 hour clock value when using timestamp datatype (Anandha and Franklin via Ashutosh Chauhan) [hashutosh] HIVE-4122 : Queries fail if timestamp data not in expected format (Prasad Mujumdar via Ashutosh Chauhan) [hashutosh] HIVE-4170 : [REGRESSION] FsShell.close closes filesystem, removing temporary directories (Navis via Ashutosh Chauhan) [gates] HIVE-4264 Moved hcatalog trunk code up to hive/trunk/hcatalog [hashutosh] HIVE-4263 : Adjust build.xml package command to move all hcat jars and binaries into build (Alan Gates via Ashutosh Chauhan) [namit] HIVE-4258 Log logical plan tree for debugging (Navis via namit) [navis] HIVE-2264 Hive server is SHUTTING DOWN when invalid queries beeing executed [kevinwilfong] HIVE-4235. CREATE TABLE IF NOT EXISTS uses inefficient way to check if table exists. (Gang Tim Liu via kevinwilfong) [gangtimliu] HIVE-4157: ORC runs out of heap when writing (Kevin Wilfong vi Gang Tim Liu) [gangtimliu] HIVE-4155: Expose ORC's FileDump as a service [gangtimliu] HIVE-4159:RetryingHMSHandler doesn't retry in enough cases (Kevin Wilfong vi Gang Tim Liu) [namit] HIVE-4149 wrong results big outer joins with array of ints (Navis via namit) [namit] HIVE-3958 support partial scan for analyze command - RCFile (Gang Tim Liu via namit) [gates] Removing old branches to limit size of Hive downloads. [gates] Removing tags directory as we no longer need them and they're in the history. [gates] Moving HCatalog into Hive. [gates] Test that perms work for hcatalog [hashutosh] HIVE-4007 : Create abstract classes for serializer and deserializer (Namit Jain via Ashutosh Chauhan) [hashutosh] HIVE-3381 : Result of outer join is not valid (Navis via Ashutosh Chauhan) [hashutosh] HIVE-3980 : Cleanup after 3403 (Namit Jain via Ashutosh Chauhan) [hashutosh] HIVE-4042 : ignore mapjoin hint (Namit Jain via Ashutosh Chauhan) [namit] HIVE-3348 semi-colon in comments in .q file does not work (Nick Collins via namit) [namit] HIVE-4212 sort merge join should work for outer joins for more than 8 inputs (Namit via Gang Tim Liu) [namit] HIVE-4219 explain dependency does not capture the input table (Namit via Gang Tim Liu) [kevinwilfong] HIVE-4092. Store complete names of tables in column access analyzer (Samuel Yuan via kevinwilfong) [namit] HIVE-4208 Clientpositive test parenthesis_star_by is non-deteministic (Mark Grover via namit) [cws] HIVE-4217. Fix show_create_table_*.q test failures (Carl Steinbach via cws) [namit] HIVE-4206 Sort merge join does not work for outer joins for 7 inputs (Namit via Gang Tim Liu) [kevinwilfong] HIVE-4188. TestJdbcDriver2.testDescribeTable failing consistently. (Prasad Mujumdar via kevinwilfong) [hashutosh] HIVE-3820 Consider creating a literal like D or BD for representing Decimal type constants (Gunther Hagleitner
[jira] [Updated] (HIVE-2597) Repeated key in GROUP BY is erroneously displayed when using DISTINCT
[ https://issues.apache.org/jira/browse/HIVE-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2597: --- Status: Open (was: Patch Available) Patch is resulting in tons of test failures. Repeated key in GROUP BY is erroneously displayed when using DISTINCT - Key: HIVE-2597 URL: https://issues.apache.org/jira/browse/HIVE-2597 Project: Hive Issue Type: Bug Reporter: Alex Rovner Assignee: Navis Attachments: HIVE-2597.D8967.1.patch, HIVE-2597.D8967.2.patch The following query was simplified for illustration purposes. This works correctly: select client_tid, as myvalue1, as myvalue2 from clients cluster by client_tid The intent here is to produce two empty columns in between data. The following query does not work: select distinct client_tid, as myvalue1, as myvalue2 from clients cluster by client_tid FAILED: Error in semantic analysis: Line 1:44 Repeated key in GROUP BY The key is not repeated since the aliases were given. Seems like Hive is ignoring the aliases when the distinct keyword is specified. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4302) Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp
[ https://issues.apache.org/jira/browse/HIVE-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624732#comment-13624732 ] Phabricator commented on HIVE-4302: --- ashutoshc has accepted the revision HIVE-4302 [jira] Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp. +1 will commit if tests pass. REVISION DETAIL https://reviews.facebook.net/D10005 BRANCH HIVE-4302 ARCANIST PROJECT hive To: JIRA, ashutoshc, hbutani Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp - Key: HIVE-4302 URL: https://issues.apache.org/jira/browse/HIVE-4302 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-4302.D10005.1.patch, HIVE-4302.D10005.2.patch Currently the RowSchema and RowResolver for the ReduceSinkOp just point to the 'input' Op's structures. This causes issues when input Op's structures are changed during Optimization. See Jira 2340 of a problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
