Query regarding Metastore(Derby) SDS table data

2011-02-03 Thread Chinna
Hi all, While creating hive table using create table command, the code flow will also insert the HDFS location in SDS table present in Derby For Example on executing:- create table sample(rate int) stored as textfile; The SDS table (meta table present in Derby) contains following entry

[jira] Updated: (HIVE-1716) make TestHBaseCliDriver use dynamic ports to avoid conflicts with already-running services

2011-02-03 Thread Namit Jain (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1716: - Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available)

[jira] Created: (HIVE-1950) Block merge for RCFile

2011-02-03 Thread He Yongqiang (JIRA)
Block merge for RCFile -- Key: HIVE-1950 URL: https://issues.apache.org/jira/browse/HIVE-1950 Project: Hive Issue Type: New Feature Reporter: He Yongqiang Assignee: He Yongqiang In our env,

Build failed in Hudson: Hive-trunk-h0.20 #530

2011-02-03 Thread Apache Hudson Server
See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/530/ -- [...truncated 22570 lines...] [junit] POSTHOOK: Output: default@srcbucket [junit] OK [junit] PREHOOK: query: LOAD DATA LOCAL INPATH

Re: Query regarding Metastore(Derby) SDS table data

2011-02-03 Thread Namit Jain
Different partitions can have different paths - A partition's path need not be a sub-direcotry of the table's path. In facebook, we use this regularly, specially for external tables. So, it simplifies things if the full path is stored for the partition in the metastore. Thanks, -namit On

[jira] Updated: (HIVE-1952) fix some outputs and make some tests deterministic

2011-02-03 Thread Namit Jain (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1952: - Attachment: hive.1952.1.patch fix some outputs and make some tests deterministic

[jira] Updated: (HIVE-1952) fix some outputs and make some tests deterministic

2011-02-03 Thread Namit Jain (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1952: - Status: Patch Available (was: Open) fix some outputs and make some tests deterministic

[jira] Created: (HIVE-1952) fix some outputs and make some tests deterministic

2011-02-03 Thread Namit Jain (JIRA)
fix some outputs and make some tests deterministic -- Key: HIVE-1952 URL: https://issues.apache.org/jira/browse/HIVE-1952 Project: Hive Issue Type: Bug Reporter: Namit Jain

[jira] Commented: (HIVE-1952) fix some outputs and make some tests deterministic

2011-02-03 Thread He Yongqiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990256#comment-12990256 ] He Yongqiang commented on HIVE-1952: +1, running tests. fix some outputs and make

[jira] Updated: (HIVE-1939) Fix test failure in TestContribCliDriver/url_hook.q

2011-02-03 Thread John Sichi (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-1939: - Fix Version/s: 0.7.0 Fix test failure in TestContribCliDriver/url_hook.q

[jira] Commented: (HIVE-1939) Fix test failure in TestContribCliDriver/url_hook.q

2011-02-03 Thread John Sichi (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990293#comment-12990293 ] John Sichi commented on HIVE-1939: -- I did some bisection on svn commits and found that the

Hive queries consuming 100% cpu

2011-02-03 Thread Vijay
Hi, The simplest of hive queries seem to be consuming 100% cpu. This is with a small 4-node cluster. The machines are pretty beefy (16 cores per machine, tons of RAM, 16 M+R maximum tasks configured, 1GB RAM for mapred.child.java.opts, etc). A simple query like select count(1) from events where

[jira] Updated: (HIVE-1951) input16_cc.q is failing in testminimrclidriver

2011-02-03 Thread He Yongqiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-1951: --- Attachment: HIVE-1951.1.patch changing the test file for a quick fix. Will open a new jira for the

[jira] Commented: (HIVE-1951) input16_cc.q is failing in testminimrclidriver

2011-02-03 Thread He Yongqiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990307#comment-12990307 ] He Yongqiang commented on HIVE-1951: opened jira

[jira] Commented: (HIVE-1939) Fix test failure in TestContribCliDriver/url_hook.q

2011-02-03 Thread Carl Steinbach (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990309#comment-12990309 ] Carl Steinbach commented on HIVE-1939: -- @John: Yup, you're right. The problem is that

[jira] Updated: (HIVE-1922) semantic analysis error, when using group by and order by together

2011-02-03 Thread Carl Steinbach (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1922: - Priority: Critical (was: Blocker) semantic analysis error, when using group by and order by

[jira] Updated: (HIVE-1951) input16_cc.q is failing in testminimrclidriver

2011-02-03 Thread Namit Jain (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1951: - Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available)

[jira] Resolved: (HIVE-1559) Contrib tests not run as part of 'ant test'

2011-02-03 Thread Carl Steinbach (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-1559. -- Resolution: Invalid I think this was fixed a while ago as part of some other ticket. I see

[jira] Commented: (HIVE-1940) Query Optimization Using Column Metadata and Histograms

2011-02-03 Thread Anja Gruenheid (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990331#comment-12990331 ] Anja Gruenheid commented on HIVE-1940: -- I have set up the last stable version, but as

[jira] Created: (HIVE-1954) Allow CLI to connect to Hive server and execute commands remotely

2011-02-03 Thread Ning Zhang (JIRA)
Allow CLI to connect to Hive server and execute commands remotely - Key: HIVE-1954 URL: https://issues.apache.org/jira/browse/HIVE-1954 Project: Hive Issue Type: New Feature

Build failed in Hudson: Hive-trunk-h0.20 #531

2011-02-03 Thread Apache Hudson Server
See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/531/changes Changes: [namit] HIVE-1716 Make TestHBaseCliDriver use dynamic ports to avoid conflicts with already-running services (John Sichi via namit) -- [...truncated 22563 lines...]

Re: Hive queries consuming 100% cpu

2011-02-03 Thread Viral Bajaria
Hey Vijay, You can go to the mapred ui, normally it runs on port 50030 of the namenode and see how many map jobs got created for your submitted query. You said that the events table has daily partitions but the example query that you have does not prune the partitions by specifying a WHERE

[jira] Commented: (HIVE-1954) Allow CLI to connect to Hive server and execute commands remotely

2011-02-03 Thread Edward Capriolo (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990346#comment-12990346 ] Edward Capriolo commented on HIVE-1954: --- This might be a dupe of

[jira] Created: (HIVE-1955) Support non-constant expressions for array indexes.

2011-02-03 Thread Adam Kramer (JIRA)
Support non-constant expressions for array indexes. --- Key: HIVE-1955 URL: https://issues.apache.org/jira/browse/HIVE-1955 Project: Hive Issue Type: Improvement Reporter: Adam

[jira] Commented: (HIVE-1954) Allow CLI to connect to Hive server and execute commands remotely

2011-02-03 Thread Ning Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990372#comment-12990372 ] Ning Zhang commented on HIVE-1954: -- Cool. I should have searched the JIRA first. Are you

Re: Hive queries consuming 100% cpu

2011-02-03 Thread Vijay
Sorry i should've given more details. The query was limited by a partition range; I just omitted the WHERE clause in the mail. The table is not that big. For each day, there is one gzipped file. The largest file is about 250MB (close to 2GB uncompressed). I did intend to count and that was just

[jira] Updated: (HIVE-1952) fix some outputs and make some tests deterministic

2011-02-03 Thread He Yongqiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-1952: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed! Thanks Namit! fix

[jira] Updated: (HIVE-1950) Block merge for RCFile

2011-02-03 Thread He Yongqiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-1950: --- Attachment: HIVE-1950.1.patch A patch for review. The code now is kind of very clean. Comments

[jira] Commented: (HIVE-1950) Block merge for RCFile

2011-02-03 Thread He Yongqiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990389#comment-12990389 ] He Yongqiang commented on HIVE-1950: review board: https://reviews.apache.org/r/388/

Review Request: HIVE-1950

2011-02-03 Thread Yongqiang He
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/388/ --- Review request for hive. Summary --- early review This addresses bug

[jira] Created: (HIVE-1956) Provide DFS initialization script for Hive

2011-02-03 Thread JIRA
Provide DFS initialization script for Hive --- Key: HIVE-1956 URL: https://issues.apache.org/jira/browse/HIVE-1956 Project: Hive Issue Type: Improvement Components: Configuration, Server

[jira] Updated: (HIVE-1956) Provide DFS initialization script for Hive

2011-02-03 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Mahé updated HIVE-1956: - Attachment: HIVE-1956.patch Provide DFS initialization script for Hive

[jira] Commented: (HIVE-1956) Provide DFS initialization script for Hive

2011-02-03 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990395#comment-12990395 ] Bruno Mahé commented on HIVE-1956: -- Review request: https://reviews.apache.org/r/389/

Review Request: HIVE-1941: support explicit view partitioning

2011-02-03 Thread John Sichi
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/390/ --- Review request for hive. Summary --- review request from JVS This

[jira] Updated: (HIVE-1941) support explicit view partitioning

2011-02-03 Thread John Sichi (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-1941: - Status: Patch Available (was: Open) https://reviews.apache.org/r/390/ support explicit view

[jira] Commented: (HIVE-1940) Query Optimization Using Column Metadata and Histograms

2011-02-03 Thread John Sichi (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990411#comment-12990411 ] John Sichi commented on HIVE-1940: -- If you just svn update to the tip of trunk and

[jira] Updated: (HIVE-1956) Provide DFS initialization script for Hive

2011-02-03 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Mahé updated HIVE-1956: - Status: Patch Available (was: Open) Provide DFS initialization script for Hive

Review Request: HIVE-1694: Accelerate GROUP BY execution using indexes

2011-02-03 Thread John Sichi
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/392/ --- Review request for hive. Summary --- Preliminary review. This addresses

[jira] Updated: (HIVE-1694) Accelerate GROUP BY execution using indexes

2011-02-03 Thread John Sichi (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-1694: - Description: The index building patch (Hive-417) is checked into trunk, this JIRA issue tracks

Build failed in Hudson: Hive-trunk-h0.20 #532

2011-02-03 Thread Apache Hudson Server
See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/532/changes Changes: [namit] HIVE-1951 input16_cc.q is failing in testminimrclidriver (He Yongqiang via namit) -- [...truncated 22598 lines...] [junit] POSTHOOK: Output: default@srcbucket

[jira] Commented: (HIVE-1956) Provide DFS initialization script for Hive

2011-02-03 Thread Namit Jain (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990429#comment-12990429 ] Namit Jain commented on HIVE-1956: -- +1 Provide DFS initialization script for Hive

Build failed in Hudson: Hive-trunk-h0.20 #533

2011-02-03 Thread Apache Hudson Server
See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/533/changes Changes: [heyongqiang] HIVE-1952. fix some outputs and make some tests deterministic (namit via He Yongqiang) -- [...truncated 21915 lines...] [junit] POSTHOOK: Output:

[jira] Updated: (HIVE-1956) Provide DFS initialization script for Hive

2011-02-03 Thread Namit Jain (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1956: - Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available)

[jira] Commented: (HIVE-1694) Accelerate GROUP BY execution using indexes

2011-02-03 Thread Prajakta Kalmegh (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12990466#comment-12990466 ] Prajakta Kalmegh commented on HIVE-1694: Thanks John. We will ensure that

[jira] Updated: (HIVE-1948) Have audit logging in the Metastore

2011-02-03 Thread Devaraj Das (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HIVE-1948: -- Attachment: audit-log.1.patch A slightly updated patch. Have audit logging in the Metastore