Query Optimization in Hive

Hi! I'm a grad student at Georgia Tech and I'm currently working with Hive for a university project. The project is on query optimization techniques and possibilities in Hive. I know that there have been a lot of additions to the ql and metastore components since the latest release and I was

[jira] Resolved: (HIVE-1533) Use ZooKeeper from maven

[ https://issues.apache.org/jira/browse/HIVE-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi resolved HIVE-1533. -- Resolution: Fixed Fixed as part of HIVE-1235. Use ZooKeeper from maven

[jira] Updated: (HIVE-1434) Cassandra Storage Handler

[ https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1434: -- Attachment: hive.diff Started re basing for 7.0. Also working on using class names to

[jira] Commented: (HIVE-1211) Tapping logs from child processes

[ https://issues.apache.org/jira/browse/HIVE-1211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12989351#comment-12989351 ] John Sichi commented on HIVE-1211: -- Will commit if tests pass. Tapping logs from child

Build failed in Hudson: Hive-trunk-h0.20 #524

See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/524/ -- [...truncated 22339 lines...] [junit] Deleted https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/contrib/test/data/warehouse/dest1 [junit] diff -a -I file: -I pfile:

[jira] Commented: (HIVE-329) start and stop hive thrift server in daemon mode

[ https://issues.apache.org/jira/browse/HIVE-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12989358#comment-12989358 ] Carl Steinbach commented on HIVE-329: - Tomcat uses the commons-daemon package for this

[jira] Commented: (HIVE-329) start and stop hive thrift server in daemon mode

[ https://issues.apache.org/jira/browse/HIVE-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12989360#comment-12989360 ] Carl Steinbach commented on HIVE-329: - See also

CFP Fourth IEEE International Scalable Computing Challenge (SCALE 2011) - Deadline 28 Feb 2011

Hi all, Please consider submitting to the: The Fourth IEEE International Scalable Computing Challenge (SCALE 2011), sponsored by the IEEE Computer Society Technical Committee on Scalable Computing (TCSC). Objective and Focus: The objective of the Fourth IEEE International Scalable Computing

Re: Query Optimization in Hive

Absolutely, that is one direction we are looking into, and none of us has actively started on that. Bharath is planning to start on join-reordering https://issues.apache.org/jira/browse/HIVE-1938. Similar to the above jira, can you file one and we can discuss in more detail in the jira ?

[jira] Created: (HIVE-1939) Fix test failure in TestContribCliDriver/url_hook.q

Fix test failure in TestContribCliDriver/url_hook.q --- Key: HIVE-1939 URL: https://issues.apache.org/jira/browse/HIVE-1939 Project: Hive Issue Type: Bug Reporter: Carl Steinbach

[jira] Updated: (HIVE-1938) Cost Based Query optimization for Joins in Hive

[ https://issues.apache.org/jira/browse/HIVE-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1938: - Summary: Cost Based Query optimization for Joins in Hive (was: Cost Based Query optimization in Hive)

[jira] Assigned: (HIVE-1938) Cost Based Query optimization for Joins in Hive

[ https://issues.apache.org/jira/browse/HIVE-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain reassigned HIVE-1938: Assignee: bharath v Cost Based Query optimization for Joins in Hive

[jira] Commented: (HIVE-1938) Cost Based Query optimization for Joins in Hive

[ https://issues.apache.org/jira/browse/HIVE-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12989389#comment-12989389 ] Namit Jain commented on HIVE-1938: -- Currently, Hive does not maintain statistics (distinct

[jira] Updated: (HIVE-1934) alter table rename messes the location

[ https://issues.apache.org/jira/browse/HIVE-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1934: - Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available)

[jira] Commented: (HIVE-1938) Cost Based Query optimization for Joins in Hive

[ https://issues.apache.org/jira/browse/HIVE-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12989404#comment-12989404 ] bharath v commented on HIVE-1938: - Can you please see

[jira] Updated: (HIVE-1896) HBase and Contrib JAR names are missing version numbers

[ https://issues.apache.org/jira/browse/HIVE-1896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-1896: - Status: Open (was: Patch Available) testCliDriver_java_mr_example is failing due to the expanded paths

[jira] Updated: (HIVE-1906) Add Eclipse launch configurations for HiveServer and MetaStoreServer

[ https://issues.apache.org/jira/browse/HIVE-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1906: - Attachment: HIVE-1906.2.patch.txt Second revision: * Fixes eclipse classpath broken by recent

[jira] Created: (HIVE-1940) Query Optimization Using Column Metadata and Histograms

Query Optimization Using Column Metadata and Histograms --- Key: HIVE-1940 URL: https://issues.apache.org/jira/browse/HIVE-1940 Project: Hive Issue Type: New Feature Components:

[jira] Created: (HIVE-1941) support explicit view partitioning

support explicit view partitioning -- Key: HIVE-1941 URL: https://issues.apache.org/jira/browse/HIVE-1941 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions:

[jira] Commented: (HIVE-1938) Cost Based Query optimization for Joins in Hive

[ https://issues.apache.org/jira/browse/HIVE-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12989488#comment-12989488 ] Namit Jain commented on HIVE-1938: -- Yes, but do need column stats for this Cost Based

[jira] Created: (HIVE-1942) change the value of hive.input.format to CombineHiveInputFormat for tests

change the value of hive.input.format to CombineHiveInputFormat for tests - Key: HIVE-1942 URL: https://issues.apache.org/jira/browse/HIVE-1942 Project: Hive Issue

[jira] Updated: (HIVE-1941) support explicit view partitioning

[ https://issues.apache.org/jira/browse/HIVE-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-1941: - Attachment: HIVE-1941.1.patch HIVE-1941.1.patch is preliminary; I haven't run through tests yet.

[jira] Commented: (HIVE-1940) Query Optimization Using Column Metadata and Histograms

[ https://issues.apache.org/jira/browse/HIVE-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12989494#comment-12989494 ] Anja Gruenheid commented on HIVE-1940: -- As first step, I would like to take a closer

Build failed in Hudson: Hive-trunk-h0.20 #525

See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/525/changes Changes: [namit] HIVE-1934 Alter table rename messes the location (Paul Yang via namit) Mmetastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java MCHANGES.txt M

[jira] Created: (HIVE-1943) Metastore operations should do a 'rollback' for HDFS failures

Metastore operations should do a 'rollback' for HDFS failures - Key: HIVE-1943 URL: https://issues.apache.org/jira/browse/HIVE-1943 Project: Hive Issue Type: Bug

[jira] Updated: (HIVE-1943) Metastore operations should do a 'rollback' for HDFS failures

[ https://issues.apache.org/jira/browse/HIVE-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1943: - Component/s: Metastore Metastore operations should do a 'rollback' for HDFS failures

Build failed in Hudson: Hive-trunk-h0.20 #526

See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/526/changes Changes: [jvs] Add reviewboard property. -- [...truncated 22216 lines...] [junit] Deleted

[jira] Updated: (HIVE-1942) change the value of hive.input.format to CombineHiveInputFormat for tests

[ https://issues.apache.org/jira/browse/HIVE-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1942: - Attachment: hive.1942.1.patch change the value of hive.input.format to CombineHiveInputFormat for tests

[jira] Updated: (HIVE-1942) change the value of hive.input.format to CombineHiveInputFormat for tests

[ https://issues.apache.org/jira/browse/HIVE-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1942: - Status: Patch Available (was: Open) change the value of hive.input.format to CombineHiveInputFormat for

[jira] Commented: (HIVE-1942) change the value of hive.input.format to CombineHiveInputFormat for tests

[ https://issues.apache.org/jira/browse/HIVE-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12989533#comment-12989533 ] He Yongqiang commented on HIVE-1942: +1, running tests change the value of

[jira] Updated: (HIVE-1944) dynamic partition insert creating different directories for the same partition during merge

[ https://issues.apache.org/jira/browse/HIVE-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-1944: - Attachment: HIVE-1944.patch this patch makes the move task a dependent of the mr task. dynamic

[jira] Updated: (HIVE-1944) dynamic partition insert creating different directories for the same partition during merge

[ https://issues.apache.org/jira/browse/HIVE-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-1944: - Status: Patch Available (was: Open) dynamic partition insert creating different directories for the