RE: wrong number of records loaded to a table is returned by Hive

Based on my cursory code inspection, the non-final row count is set when ExecDriver.progress calls ss.getHiveHistory().setCounters(...) inside the while loop, and we need to add the same call after the while loop (after the last updateCounters call at the end) to set the final row count. From:

[jira] Commented: (HIVE-1611) Add alternative search-provider to Hive site

[ https://issues.apache.org/jira/browse/HIVE-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917145#action_12917145 ] Otis Gospodnetic commented on HIVE-1611: +1 for getting the patch in now and adjusti

[jira] Updated: (HIVE-1427) Provide metastore schema migration scripts (0.5 -> 0.6)

[ https://issues.apache.org/jira/browse/HIVE-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1427: - Status: Patch Available (was: Open) > Provide metastore schema migration scripts (0.5 -> 0.6) > -

[jira] Commented: (HIVE-1427) Provide metastore schema migration scripts (0.5 -> 0.6)

[ https://issues.apache.org/jira/browse/HIVE-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917140#action_12917140 ] HBase Review Board commented on HIVE-1427: -- Message from: "Carl Steinbach" --

Review Request: HIVE-1427: Provide metastore schema migration scripts (0.5 -> 0.6)

--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/931/ --- Review request for Hive Developers, namit jain, John Sichi, and Paul Yang. Summar

[jira] Updated: (HIVE-1427) Provide metastore schema migration scripts (0.5 -> 0.6)

[ https://issues.apache.org/jira/browse/HIVE-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1427: - Attachment: HIVE-1427.1.patch.txt HIVE-1427.1.patch.txt: * Upgrade scripts for derby and mysql. *

[jira] Updated: (HIVE-1658) Fix describe [extended] column formatting

[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvel Thirumoolan updated HIVE-1658: --- Attachment: HIVE-1658-PrelimPatch.patch Preliminary patch on the above mentioned appro

Re: mapreduce against hive raw data ?

On Fri, Oct 1, 2010 at 3:08 PM, Jinsong Hu wrote: > Hi, There: >  I wonder if it is possible to run map-reduce again hive's raw data. > hive supports hql, but sometimes, I want to run map-reduce to do more > sophisticated > processing than those simple hql can handle. In that case, I need to run m

[jira] Commented: (HIVE-1611) Add alternative search-provider to Hive site

[ https://issues.apache.org/jira/browse/HIVE-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917021#action_12917021 ] Alex Baranau commented on HIVE-1611: I see. As for reflecting project structure changes

Re: wrong number of records loaded to a table is returned by Hive

One more data point: in Hive History: org.apache.hadoop.hive.ql.exec.FileSinkOperator$TableIdEnum.TABLE_ID_1_ROWCOUNT: 26002996 in JT: org.apache.hadoop.hive.ql.exec.FileSinkOperator$TableIdEnumTABLE_ID_1_ROWCOUNT031,208,09931,208,099 From: gaurav jain To

[jira] Updated: (HIVE-307) "LOAD DATA LOCAL INPATH" fails when the table already contains a file of the same name

[ https://issues.apache.org/jira/browse/HIVE-307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated HIVE-307: --- Status: Patch Available (was: Open) Patch attached. > "LOAD DATA LOCAL INPATH" fails when the table already c

[jira] Commented: (HIVE-1611) Add alternative search-provider to Hive site

[ https://issues.apache.org/jira/browse/HIVE-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917016#action_12917016 ] Edward Capriolo commented on HIVE-1611: --- Now that hive is TLP we likely have to get t

Build failed in Hudson: Hive-trunk-h0.20 #379

See Changes: [heyongqiang] HIVE-1624. Patch to allows scripts in S3 location.(Vaibhav Aggarwal via He Yongqiang) -- [...truncated 14212 lines...] [junit] POSTHOOK: Output: defa...@src

mapreduce against hive raw data ?

Hi, There: I wonder if it is possible to run map-reduce again hive's raw data. hive supports hql, but sometimes, I want to run map-reduce to do more sophisticated processing than those simple hql can handle. In that case, I need to run my own custom map-reduce job against hive's raw data. I wo

Re: wrong number of records loaded to a table is returned by Hive

Hi Ning, I also see the same behavior. Below is some data for your reference. This behavior is observed for large values. I believe HIVE is recording non-final values at the end of insert query: Since hive reads the HIVE History file counters, it may be printing non-final values. Relevant fun

[jira] Commented: (HIVE-1611) Add alternative search-provider to Hive site

[ https://issues.apache.org/jira/browse/HIVE-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917001#action_12917001 ] Ning Zhang commented on HIVE-1611: -- Thanks for the link Alex. I've talked to Ashish and he

[jira] Resolved: (HIVE-1157) UDFs can't be loaded via "add jar" when jar is on HDFS

[ https://issues.apache.org/jira/browse/HIVE-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-1157. -- Resolution: Duplicate > UDFs can't be loaded via "add jar" when jar is on HDFS > ---

[jira] Resolved: (HIVE-1684) intermittent failures in create_escape.q

[ https://issues.apache.org/jira/browse/HIVE-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang resolved HIVE-1684. -- Resolution: Duplicate duplicate of HIVE-1669. > intermittent failures in create_escape.q >

[jira] Commented: (HIVE-1658) Fix describe [extended] column formatting

[ https://issues.apache.org/jira/browse/HIVE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916992#action_12916992 ] Thiruvel Thirumoolan commented on HIVE-1658: Patch under works. Changes: 1. 'd

[jira] Commented: (HIVE-1684) intermittent failures in create_escape.q

[ https://issues.apache.org/jira/browse/HIVE-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916990#action_12916990 ] Ning Zhang commented on HIVE-1684: -- This is the same as HIVE-1669, which was introduced in

[jira] Updated: (HIVE-1376) Simple UDAFs with more than 1 parameter crash on empty row query

[ https://issues.apache.org/jira/browse/HIVE-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-1376: - Attachment: HIVE-1376.2.patch The previous patch failed on several test, particularly count(*) queries. A

[jira] Created: (HIVE-1688) In the MapJoinOperator, the code uses tag as alias, which is not always true

In the MapJoinOperator, the code uses tag as alias, which is not always true Key: HIVE-1688 URL: https://issues.apache.org/jira/browse/HIVE-1688 Project: Hadoop Hive

[jira] Updated: (HIVE-1669) non-deterministic display of storage parameter in test

[ https://issues.apache.org/jira/browse/HIVE-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvel Thirumoolan updated HIVE-1669: --- Attachment: HIVE-1669.patch Just sorting the param keys before displaying them. Test o

[jira] Commented: (HIVE-1686) XMLEncoder failing to serialize classes containing Enums for non-SUN JREs

[ https://issues.apache.org/jira/browse/HIVE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916966#action_12916966 ] Stephen Watt commented on HIVE-1686: This issue will also be fixed in the JRE with the n

[jira] Commented: (HIVE-1686) XMLEncoder failing to serialize classes containing Enums for non-SUN JREs

[ https://issues.apache.org/jira/browse/HIVE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916964#action_12916964 ] Stephen Watt commented on HIVE-1686: To resolve this issue, I have provided a patch that

[jira] Commented: (HIVE-1611) Add alternative search-provider to Hive site

[ https://issues.apache.org/jira/browse/HIVE-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916963#action_12916963 ] Alex Baranau commented on HIVE-1611: Hello, Sorry for misleading you, the patch is appli

[jira] Updated: (HIVE-1686) XMLEncoder failing to serialize classes containing Enums for non-SUN JREs

[ https://issues.apache.org/jira/browse/HIVE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Watt updated HIVE-1686: --- Attachment: HIVE-1686.patch > XMLEncoder failing to serialize classes containing Enums for non-SUN JRE

[jira] Commented: (HIVE-1683) Column aliases cannot be used in a group by clause

[ https://issues.apache.org/jira/browse/HIVE-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916960#action_12916960 ] John Sichi commented on HIVE-1683: -- This is not a bug and should be closed as invalid. Con

[jira] Updated: (HIVE-1673) Create table bug causes the row format property lost when serde is specified.

[ https://issues.apache.org/jira/browse/HIVE-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1673: - Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed Commit

[jira] Created: (HIVE-1687) smb_mapjoin_8.q in TestMinimrCliDriver hangs/fails

smb_mapjoin_8.q in TestMinimrCliDriver hangs/fails -- Key: HIVE-1687 URL: https://issues.apache.org/jira/browse/HIVE-1687 Project: Hadoop Hive Issue Type: Bug Components: Query Proces

[jira] Updated: (HIVE-1157) UDFs can't be loaded via "add jar" when jar is on HDFS

[ https://issues.apache.org/jira/browse/HIVE-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Zeyliger updated HIVE-1157: -- Attachment: HIVE-1157.patch.v6.txt > UDFs can't be loaded via "add jar" when jar is on HDFS > --

[jira] Commented: (HIVE-1157) UDFs can't be loaded via "add jar" when jar is on HDFS

[ https://issues.apache.org/jira/browse/HIVE-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916956#action_12916956 ] Philip Zeyliger commented on HIVE-1157: --- Namit, Thanks for the review. I've fixed th

[jira] Created: (HIVE-1686) XMLEncoder failing to serialize classes containing Enums for non-SUN JREs

XMLEncoder failing to serialize classes containing Enums for non-SUN JREs - Key: HIVE-1686 URL: https://issues.apache.org/jira/browse/HIVE-1686 Project: Hadoop Hive Issu

[jira] Updated: (HIVE-307) "LOAD DATA LOCAL INPATH" fails when the table already contains a file of the same name

[ https://issues.apache.org/jira/browse/HIVE-307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated HIVE-307: --- Attachment: HIVE-307.patch Here is a resubmission of the patch with a CLI-based unit test. Let me know what el

[jira] Commented: (HIVE-1683) Column aliases cannot be used in a group by clause

[ https://issues.apache.org/jira/browse/HIVE-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916953#action_12916953 ] Namit Jain commented on HIVE-1683: -- A workaround is to use the original expression: select

[jira] Commented: (HIVE-1673) Create table bug causes the row format property lost when serde is specified.

[ https://issues.apache.org/jira/browse/HIVE-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916951#action_12916951 ] Namit Jain commented on HIVE-1673: -- scriptfile1.q and smp_mapjoin_8.q in TestMiniMrCliDrive

[jira] Created: (HIVE-1685) scriptfile1.1 in minimr faling intermittently

scriptfile1.1 in minimr faling intermittently - Key: HIVE-1685 URL: https://issues.apache.org/jira/browse/HIVE-1685 Project: Hadoop Hive Issue Type: Bug Components: Query Processor

Build failed in Hudson: Hive-trunk-h0.19 #556

See Changes: [heyongqiang] HIVE-1624. Patch to allows scripts in S3 location.(Vaibhav Aggarwal via He Yongqiang) -- [...truncated 12232 lines...] [junit] POSTHOOK: Output: defa...@src

RE: release 0.6

I am not sure what kind of downtime would it involve for us (facebook). We will have to make a copy of the production metastore, and then perform the changes. If that takes a long time, we will have to come up with some quicker upgrade solutions - We will try to do that today, and get back to yo

[jira] Created: (HIVE-1684) intermittent failures in create_escape.q

intermittent failures in create_escape.q Key: HIVE-1684 URL: https://issues.apache.org/jira/browse/HIVE-1684 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Repo

[jira] Commented: (HIVE-1673) Create table bug causes the row format property lost when serde is specified.

[ https://issues.apache.org/jira/browse/HIVE-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916944#action_12916944 ] Namit Jain commented on HIVE-1673: -- create_escape.q in TestCliDriver is failing intermitten

Build failed in Hudson: Hive-trunk-h0.18 #556

See Changes: [heyongqiang] HIVE-1624. Patch to allows scripts in S3 location.(Vaibhav Aggarwal via He Yongqiang) -- [...truncated 30997 lines...] [junit] POSTHOOK: Output: defa...@src

[jira] Commented: (HIVE-1611) Add alternative search-provider to Hive site

[ https://issues.apache.org/jira/browse/HIVE-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916927#action_12916927 ] Ning Zhang commented on HIVE-1611: -- Hi Alex, some questions: - Hive doesn't have the file

Build failed in Hudson: Hive-trunk-h0.17 #555

See Changes: [heyongqiang] HIVE-1624. Patch to allows scripts in S3 location.(Vaibhav Aggarwal via He Yongqiang) [namit] HIVE-1670 MapJoin throws an error if no column from the mapjoined table is selected (Ning Zhang via namit

[jira] Commented: (HIVE-1611) Add alternative search-provider to Hive site

[ https://issues.apache.org/jira/browse/HIVE-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916880#action_12916880 ] Alex Baranau commented on HIVE-1611: Sorry to ping, but it looks like since the issue is

[jira] Commented: (HIVE-1612) Cannot build hive for hadoop 0.21.0

[ https://issues.apache.org/jira/browse/HIVE-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916869#action_12916869 ] bharath v commented on HIVE-1612: - Any one working on this issue ? > Cannot build hive for

[jira] Created: (HIVE-1683) Column aliases cannot be used in a group by clause

Column aliases cannot be used in a group by clause -- Key: HIVE-1683 URL: https://issues.apache.org/jira/browse/HIVE-1683 Project: Hadoop Hive Issue Type: Bug Components: Metastore

[jira] Commented: (HIVE-1452) Mapside join on non partitioned table with partitioned table causes error

[ https://issues.apache.org/jira/browse/HIVE-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916842#action_12916842 ] Thiruvel Thirumoolan commented on HIVE-1452: > However, I see different results

[jira] Created: (HIVE-1682) Wrong results with MAPJOIN when cols from non-MAPJOINed table are selected

Wrong results with MAPJOIN when cols from non-MAPJOINed table are selected -- Key: HIVE-1682 URL: https://issues.apache.org/jira/browse/HIVE-1682 Project: Hadoop Hive Is

[jira] Resolved: (HIVE-1452) Mapside join on non partitioned table with partitioned table causes error

[ https://issues.apache.org/jira/browse/HIVE-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvel Thirumoolan resolved HIVE-1452. Resolution: Duplicate HIVE-1670 fixes the EOF issue and I dont see the problem with

[jira] Assigned: (HIVE-1678) NPE in MapJoin

[ https://issues.apache.org/jira/browse/HIVE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amareshwari Sriramadasu reassigned HIVE-1678: - Assignee: Amareshwari Sriramadasu > NPE in MapJoin > --- > >

[jira] Updated: (HIVE-1681) ObjectStore.commitTransaction() does not properly handle transactions that have already been rolled back

[ https://issues.apache.org/jira/browse/HIVE-1681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1681: - Attachment: HIVE-1681.1.patch.txt > ObjectStore.commitTransaction() does not properly handle trans

[jira] Created: (HIVE-1681) ObjectStore.commitTransaction() does not properly handle transactions that have already been rolled back

ObjectStore.commitTransaction() does not properly handle transactions that have already been rolled back Key: HIVE-1681 URL: https://issues.apache.org/jira/br

[jira] Created: (HIVE-1680) EXPLAIN for a CTAS query fails if the table already exists in the system

EXPLAIN for a CTAS query fails if the table already exists in the system - Key: HIVE-1680 URL: https://issues.apache.org/jira/browse/HIVE-1680 Project: Hadoop Hive Issu

[jira] Created: (HIVE-1679) MetaStore does not detect and rollback failed transactions

MetaStore does not detect and rollback failed transactions -- Key: HIVE-1679 URL: https://issues.apache.org/jira/browse/HIVE-1679 Project: Hadoop Hive Issue Type: Bug Componen