[jira] Updated: (HIVE-1973) Getting error when join on tables where name of table has uppercase letters
[
https://issues.apache.org/jira/browse/HIVE-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinna Rao Lalam updated HIVE-1973:
---
Status: Open (was: Patch Available)
Getting error when join on tables where name of table has uppercase letters
---
Key: HIVE-1973
URL: https://issues.apache.org/jira/browse/HIVE-1973
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.5.0, 0.7.0
Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise
Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
When execute a join query on tables containing Uppercase letters in the table
names hit an exception
Ex:
{noformat}
create table a(b int);
create table tabForJoin(b int,c int);
select * from a join tabForJoin on(a.b=tabForJoin.b);
Got an exception like this
FAILED: Error in semantic analysis: Invalid Table Alias tabForJoin
{noformat}
But if i give without capital letters ,It is working
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1973) Getting error when join on tables where name of table has uppercase letters
[
https://issues.apache.org/jira/browse/HIVE-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinna Rao Lalam updated HIVE-1973:
---
Status: Patch Available (was: Open)
Getting error when join on tables where name of table has uppercase letters
---
Key: HIVE-1973
URL: https://issues.apache.org/jira/browse/HIVE-1973
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.5.0, 0.7.0
Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise
Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
When execute a join query on tables containing Uppercase letters in the table
names hit an exception
Ex:
{noformat}
create table a(b int);
create table tabForJoin(b int,c int);
select * from a join tabForJoin on(a.b=tabForJoin.b);
Got an exception like this
FAILED: Error in semantic analysis: Invalid Table Alias tabForJoin
{noformat}
But if i give without capital letters ,It is working
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1973) Getting error when join on tables where name of table has uppercase letters
[
https://issues.apache.org/jira/browse/HIVE-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinna Rao Lalam updated HIVE-1973:
---
Attachment: HIVE-1973.patch
Getting error when join on tables where name of table has uppercase letters
---
Key: HIVE-1973
URL: https://issues.apache.org/jira/browse/HIVE-1973
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.5.0, 0.7.0
Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise
Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
Attachments: HIVE-1973.patch
When execute a join query on tables containing Uppercase letters in the table
names hit an exception
Ex:
{noformat}
create table a(b int);
create table tabForJoin(b int,c int);
select * from a join tabForJoin on(a.b=tabForJoin.b);
Got an exception like this
FAILED: Error in semantic analysis: Invalid Table Alias tabForJoin
{noformat}
But if i give without capital letters ,It is working
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1973) Getting error when join on tables where name of table has uppercase letters
[
https://issues.apache.org/jira/browse/HIVE-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinna Rao Lalam updated HIVE-1973:
---
Status: Patch Available (was: Open)
Getting error when join on tables where name of table has uppercase letters
---
Key: HIVE-1973
URL: https://issues.apache.org/jira/browse/HIVE-1973
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.5.0, 0.7.0
Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise
Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
Attachments: HIVE-1973.patch
When execute a join query on tables containing Uppercase letters in the table
names hit an exception
Ex:
{noformat}
create table a(b int);
create table tabForJoin(b int,c int);
select * from a join tabForJoin on(a.b=tabForJoin.b);
Got an exception like this
FAILED: Error in semantic analysis: Invalid Table Alias tabForJoin
{noformat}
But if i give without capital letters ,It is working
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1974) In error scenario some opened streams may not closed in ScriptOperator.java, Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-1974: --- Attachment: HIVE-1974.patch In error scenario some opened streams may not closed in ScriptOperator.java, Utilities.java Key: HIVE-1974 URL: https://issues.apache.org/jira/browse/HIVE-1974 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.5.0, 0.7.0 Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5). Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-1974.patch 1)In error scenario StreamProcessor may not be closed in ScriptOperator.java 2)In error scenario XMLEncoder may not be closed in Utilities.java -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1980) Merging using mapreduce rather than map-only job failed in case of dynamic partition inserts
[ https://issues.apache.org/jira/browse/HIVE-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12997962#comment-12997962 ] Namit Jain commented on HIVE-1980: -- Wont it also apply to non-dynamic partitions. Won't it apply to all partitions where merge is being performed ? Merging using mapreduce rather than map-only job failed in case of dynamic partition inserts Key: HIVE-1980 URL: https://issues.apache.org/jira/browse/HIVE-1980 Project: Hive Issue Type: Bug Reporter: Ning Zhang Assignee: Ning Zhang Attachments: HIVE-1980.patch In dynamic partition insert and if merge is set to true and hive.mergejob.maponly=false, the merge MapReduce job will fail. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-967) Implement show create table
[ https://issues.apache.org/jira/browse/HIVE-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Ramadorai updated HIVE-967: --- Attachment: HiveShowCreateTable.jar Jar of Ed's Show Create Table utility Implement show create table - Key: HIVE-967 URL: https://issues.apache.org/jira/browse/HIVE-967 Project: Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Adam Kramer Assignee: Edward Capriolo Attachments: HiveShowCreateTable.jar, show_create.txt SHOW CREATE TABLE would be very useful in cases where you are trying to figure out the partitioning and/or bucketing scheme for a table. Perhaps this could be implemented by having new tables automatically SET PROPERTIES (create_command='raw text of the create statement')? -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1980) Merging using mapreduce rather than map-only job failed in case of dynamic partition inserts
[ https://issues.apache.org/jira/browse/HIVE-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12997988#comment-12997988 ] Namit Jain commented on HIVE-1980: -- Talked offline with Ning +1 Merging using mapreduce rather than map-only job failed in case of dynamic partition inserts Key: HIVE-1980 URL: https://issues.apache.org/jira/browse/HIVE-1980 Project: Hive Issue Type: Bug Reporter: Ning Zhang Assignee: Ning Zhang Attachments: HIVE-1980.patch In dynamic partition insert and if merge is set to true and hive.mergejob.maponly=false, the merge MapReduce job will fail. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1974) In error scenario some opened streams may not closed in ScriptOperator.java, Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12997998#comment-12997998 ] Namit Jain commented on HIVE-1974: -- +1 In error scenario some opened streams may not closed in ScriptOperator.java, Utilities.java Key: HIVE-1974 URL: https://issues.apache.org/jira/browse/HIVE-1974 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.5.0, 0.7.0 Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5). Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-1974.patch 1)In error scenario StreamProcessor may not be closed in ScriptOperator.java 2)In error scenario XMLEncoder may not be closed in Utilities.java -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HIVE-2000) adding comments to Hive Stats JDBC queries
adding comments to Hive Stats JDBC queries --- Key: HIVE-2000 URL: https://issues.apache.org/jira/browse/HIVE-2000 Project: Hive Issue Type: Improvement Reporter: Ning Zhang Assignee: Ning Zhang Priority: Trivial Stats gathering could issue a large number of JDBC queries to stats publisher and aggregator. In a shared DB environment, it's hard to tell which SQL statements are contributed by Hive stats gathering tasks. It would be easier to identify these workload by adding SQL comments that identifies the source of the query. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (HIVE-1982) Group by key shall not duplicate with distinct key
[ https://issues.apache.org/jira/browse/HIVE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain reassigned HIVE-1982: Assignee: Ted Xu Group by key shall not duplicate with distinct key -- Key: HIVE-1982 URL: https://issues.apache.org/jira/browse/HIVE-1982 Project: Hive Issue Type: Bug Affects Versions: 0.6.0, 0.7.0 Reporter: Ted Xu Assignee: Ted Xu Priority: Minor Attachments: HIVE-1982-3.patch, HIVE-1982-v2.patch, HIVE-1982.patch Group by key shall not duplicate with distinct key, or there will be error because RowResolver and ColumnInfo didn't match. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2000) adding comments to Hive Stats JDBC queries
[ https://issues.apache.org/jira/browse/HIVE-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-2000: - Attachment: HIVE-2000.patch adding comments to Hive Stats JDBC queries --- Key: HIVE-2000 URL: https://issues.apache.org/jira/browse/HIVE-2000 Project: Hive Issue Type: Improvement Reporter: Ning Zhang Assignee: Ning Zhang Priority: Trivial Attachments: HIVE-2000.patch Stats gathering could issue a large number of JDBC queries to stats publisher and aggregator. In a shared DB environment, it's hard to tell which SQL statements are contributed by Hive stats gathering tasks. It would be easier to identify these workload by adding SQL comments that identifies the source of the query. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2000) adding comments to Hive Stats JDBC queries
[ https://issues.apache.org/jira/browse/HIVE-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-2000: - Status: Patch Available (was: Open) adding comments to Hive Stats JDBC queries --- Key: HIVE-2000 URL: https://issues.apache.org/jira/browse/HIVE-2000 Project: Hive Issue Type: Improvement Reporter: Ning Zhang Assignee: Ning Zhang Priority: Trivial Attachments: HIVE-2000.patch Stats gathering could issue a large number of JDBC queries to stats publisher and aggregator. In a shared DB environment, it's hard to tell which SQL statements are contributed by Hive stats gathering tasks. It would be easier to identify these workload by adding SQL comments that identifies the source of the query. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Hudson: Hive-trunk-h0.20 #571
See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/571/ -- [...truncated 14118 lines...] [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at junit.framework.TestCase.runTest(TestCase.java:154) [junit] at junit.framework.TestCase.runBare(TestCase.java:127) [junit] at junit.framework.TestResult$1.protect(TestResult.java:106) [junit] at junit.framework.TestResult.runProtected(TestResult.java:124) [junit] at junit.framework.TestResult.run(TestResult.java:109) [junit] at junit.framework.TestCase.run(TestCase.java:118) [junit] at junit.framework.TestSuite.runTest(TestSuite.java:208) [junit] at junit.framework.TestSuite.run(TestSuite.java:203) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785) [junit] Begin query: bad_exec_hooks.q [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I grantTime -I [.][.][.] [0-9]* more -I USING 'java -cp https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/ql/test/logs/clientnegative/bad_exec_hooks.q.out https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ql/src/test/results/clientnegative/bad_exec_hooks.q.out [junit] Done query: bad_exec_hooks.q [junit] Begin query: bad_indextype.q [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I grantTime -I [.][.][.] [0-9]* more -I USING 'java -cp https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/ql/test/logs/clientnegative/bad_indextype.q.out https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ql/src/test/results/clientnegative/bad_indextype.q.out [junit] Done query: bad_indextype.q [junit] Begin query: bad_sample_clause.q [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I grantTime -I [.][.][.] [0-9]* more -I USING 'java -cp https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/ql/test/logs/clientnegative/bad_sample_clause.q.out https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ql/src/test/results/clientnegative/bad_sample_clause.q.out [junit] Done query: bad_sample_clause.q [junit] Begin query: clusterbydistributeby.q [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I grantTime -I [.][.][.] [0-9]* more -I USING 'java -cp https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/ql/test/logs/clientnegative/clusterbydistributeby.q.out https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ql/src/test/results/clientnegative/clusterbydistributeby.q.out [junit] Done query: clusterbydistributeby.q [junit] Begin query: clusterbyorderby.q [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I grantTime -I [.][.][.] [0-9]* more -I USING 'java -cp https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/ql/test/logs/clientnegative/clusterbyorderby.q.out https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ql/src/test/results/clientnegative/clusterbyorderby.q.out [junit] Done query: clusterbyorderby.q [junit] Begin query: clusterbysortby.q [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I transient_lastDdlTime -I
Build failed in Hudson: Hive-0.7.0-h0.20 #14
See https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/14/ -- [...truncated 25728 lines...] [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/build/service/tmp/hive_job_log_hudson_201102221149_1495791935.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select count(1) as cnt from testhivedrivertable [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-02-22_11-49-38_265_1025957573779952629/-mr-1 [junit] Total MapReduce jobs = 1 [junit] Launching Job 1 out of 1 [junit] Number of reduce tasks determined at compile time: 1 [junit] In order to change the average load for a reducer (in bytes): [junit] set hive.exec.reducers.bytes.per.reducer=number [junit] In order to limit the maximum number of reducers: [junit] set hive.exec.reducers.max=number [junit] In order to set a constant number of reducers: [junit] set mapred.reduce.tasks=number [junit] Job running in-process (local Hadoop) [junit] 2011-02-22 11:49:41,277 null map = 100%, reduce = 100% [junit] Ended Job = job_local_0001 [junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-02-22_11-49-38_265_1025957573779952629/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/build/service/tmp/hive_job_log_hudson_201102221149_1639729646.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select * from testhivedrivertable limit 10 [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-02-22_11-49-42_794_6493018026017792984/-mr-1 [junit] POSTHOOK: query: select * from testhivedrivertable limit 10 [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-02-22_11-49-42_794_6493018026017792984/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable
[jira] Created: (HIVE-2001) Add inputs and outputs to authorization ddls.
Add inputs and outputs to authorization ddls. - Key: HIVE-2001 URL: https://issues.apache.org/jira/browse/HIVE-2001 Project: Hive Issue Type: Bug Reporter: He Yongqiang Assignee: He Yongqiang -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-818) Create a Hive CLI that connects to hive ThriftServer
[ https://issues.apache.org/jira/browse/HIVE-818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998108#comment-12998108 ] Ning Zhang commented on HIVE-818: - Yongqiang, have you got any chance to look at the patch? Create a Hive CLI that connects to hive ThriftServer Key: HIVE-818 URL: https://issues.apache.org/jira/browse/HIVE-818 Project: Hive Issue Type: New Feature Components: Clients, Server Infrastructure Reporter: Edward Capriolo Assignee: Ning Zhang Attachments: HIVE-818.2.patch, HIVE-818.3.patch, HIVE-818.4.patch, HIVE-818.patch, Hive-881_2.patch We should have an alternate CLI that works by interacting with the HiveServer, in this way it will be ready when/if we deprecate the current CLI. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-818) Create a Hive CLI that connects to hive ThriftServer
[ https://issues.apache.org/jira/browse/HIVE-818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998109#comment-12998109 ] He Yongqiang commented on HIVE-818: --- sorry, just saw that you already uploaded a new patch. will start running tests after 1517. Create a Hive CLI that connects to hive ThriftServer Key: HIVE-818 URL: https://issues.apache.org/jira/browse/HIVE-818 Project: Hive Issue Type: New Feature Components: Clients, Server Infrastructure Reporter: Edward Capriolo Assignee: Ning Zhang Attachments: HIVE-818.2.patch, HIVE-818.3.patch, HIVE-818.4.patch, HIVE-818.patch, Hive-881_2.patch We should have an alternate CLI that works by interacting with the HiveServer, in this way it will be ready when/if we deprecate the current CLI. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-1918: Add export/import facilities to the hive system
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/339/#review255 --- ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java https://reviews.apache.org/r/339/#comment500 Can we avoid nesting the ternary operator? It makes the code a little confusing. There are several instances of this in the diff, but I've just highlighted the first one. ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java https://reviews.apache.org/r/339/#comment501 If we go with the route of having auto-generated code, then we might want to look into having JDO to handle this for us. Datanucleus/JDO has an option to persist to an XML file that might be applicable for this use case. I would agree that some discretion is required in picking fields to serialize for import/export, but the fear is that adding a field now will require many changes. - Paul On 2011-02-04 17:13:17, Carl Steinbach wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/339/ --- (Updated 2011-02-04 17:13:17) Review request for hive. Summary --- Review for HIVE-1918. This addresses bug HIVE-1918. https://issues.apache.org/jira/browse/HIVE-1918 Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7e5e19f conf/hive-default.xml 46156c0 ql/src/java/org/apache/hadoop/hive/ql/exec/CopyTask.java 30ea670 ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 6fea990 ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java e47992a ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 5f78082 ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java b7c51ae ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java d8442b2 ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 01eef69 ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g c5574b0 ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 15e7a13 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 7655154 ql/src/java/org/apache/hadoop/hive/ql/plan/AddPartitionDesc.java e7be269 ql/src/java/org/apache/hadoop/hive/ql/plan/CopyWork.java 7a62ec7 ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java e484fe2 ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java d5bccae ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveUtils.java PRE-CREATION ql/src/test/queries/clientnegative/exim_00_unsupported_schema.q PRE-CREATION ql/src/test/queries/clientnegative/exim_01_nonpart_over_loaded.q PRE-CREATION ql/src/test/queries/clientnegative/exim_02_all_part_over_overlap.q PRE-CREATION ql/src/test/queries/clientnegative/exim_03_nonpart_noncompat_colschema.q PRE-CREATION ql/src/test/queries/clientnegative/exim_04_nonpart_noncompat_colnumber.q PRE-CREATION ql/src/test/queries/clientnegative/exim_05_nonpart_noncompat_coltype.q PRE-CREATION ql/src/test/queries/clientnegative/exim_06_nonpart_noncompat_storage.q PRE-CREATION ql/src/test/queries/clientnegative/exim_07_nonpart_noncompat_ifof.q PRE-CREATION ql/src/test/queries/clientnegative/exim_08_nonpart_noncompat_serde.q PRE-CREATION ql/src/test/queries/clientnegative/exim_09_nonpart_noncompat_serdeparam.q PRE-CREATION ql/src/test/queries/clientnegative/exim_10_nonpart_noncompat_bucketing.q PRE-CREATION ql/src/test/queries/clientnegative/exim_11_nonpart_noncompat_sorting.q PRE-CREATION ql/src/test/queries/clientnegative/exim_12_nonnative_export.q PRE-CREATION ql/src/test/queries/clientnegative/exim_13_nonnative_import.q PRE-CREATION ql/src/test/queries/clientnegative/exim_14_nonpart_part.q PRE-CREATION ql/src/test/queries/clientnegative/exim_15_part_nonpart.q PRE-CREATION ql/src/test/queries/clientnegative/exim_16_part_noncompat_schema.q PRE-CREATION ql/src/test/queries/clientnegative/exim_17_part_spec_underspec.q PRE-CREATION ql/src/test/queries/clientnegative/exim_18_part_spec_missing.q PRE-CREATION ql/src/test/queries/clientnegative/exim_19_external_over_existing.q PRE-CREATION ql/src/test/queries/clientnegative/exim_20_managed_location_over_existing.q PRE-CREATION ql/src/test/queries/clientnegative/exim_21_part_managed_external.q PRE-CREATION ql/src/test/queries/clientpositive/exim_00_nonpart_empty.q PRE-CREATION
[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system
[ https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998129#comment-12998129 ] Paul Yang commented on HIVE-1918: - Made a couple of comments on reviewboard. Add export/import facilities to the hive system --- Key: HIVE-1918 URL: https://issues.apache.org/jira/browse/HIVE-1918 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Krishna Kumar Assignee: Krishna Kumar Attachments: HIVE-1918.patch.1.txt, HIVE-1918.patch.2.txt, HIVE-1918.patch.3.txt, HIVE-1918.patch.4.txt, HIVE-1918.patch.txt, hive-metastore-er.pdf This is an enhancement request to add export/import features to hive. With this language extension, the user can export the data of the table - which may be located in different hdfs locations in case of a partitioned table - as well as the metadata of the table into a specified output location. This output location can then be moved over to another different hadoop/hive instance and imported there. This should work independent of the source and target metastore dbms used; for instance, between derby and mysql. For partitioned tables, the ability to export/import a subset of the partition must be supported. Howl will add more features on top of this: The ability to create/use the exported data even in the absence of hive, using MR or Pig. Please see http://wiki.apache.org/pig/Howl/HowlImportExport for these details. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2001) Add inputs and outputs to authorization ddls.
[ https://issues.apache.org/jira/browse/HIVE-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-2001: Description: When permissions are changed for a table/partition, the respective object should be present in the read/write entities for hooks to act on. Add inputs and outputs to authorization ddls. - Key: HIVE-2001 URL: https://issues.apache.org/jira/browse/HIVE-2001 Project: Hive Issue Type: Bug Reporter: He Yongqiang Assignee: He Yongqiang Attachments: hive-2001.patch When permissions are changed for a table/partition, the respective object should be present in the read/write entities for hooks to act on. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-2001) Add inputs and outputs to authorization ddls.
[ https://issues.apache.org/jira/browse/HIVE-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998135#comment-12998135 ] Paul Yang commented on HIVE-2001: - +1 will test and commit Add inputs and outputs to authorization ddls. - Key: HIVE-2001 URL: https://issues.apache.org/jira/browse/HIVE-2001 Project: Hive Issue Type: Bug Reporter: He Yongqiang Assignee: He Yongqiang Attachments: hive-2001.patch When permissions are changed for a table/partition, the respective object should be present in the read/write entities for hooks to act on. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2001) Add inputs and outputs to authorization DDL commands
[ https://issues.apache.org/jira/browse/HIVE-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-2001: Component/s: Query Processor Affects Version/s: 0.8.0 Summary: Add inputs and outputs to authorization DDL commands (was: Add inputs and outputs to authorization ddls.) Add inputs and outputs to authorization DDL commands Key: HIVE-2001 URL: https://issues.apache.org/jira/browse/HIVE-2001 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.8.0 Reporter: He Yongqiang Assignee: He Yongqiang Attachments: hive-2001.patch When permissions are changed for a table/partition, the respective object should be present in the read/write entities for hooks to act on. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HIVE-2002) Expand exceptions caught for metastore operations
Expand exceptions caught for metastore operations
-
Key: HIVE-2002
URL: https://issues.apache.org/jira/browse/HIVE-2002
Project: Hive
Issue Type: Improvement
Components: Metastore
Affects Versions: 0.8.0
Reporter: Paul Yang
Priority: Minor
Currently, HiveMetaStore.executeWithRetry() catches two classes of exceptions
and retries the metastore call when such exceptions occur. However, it does not
catch some exceptions that could benefit from a retry:
{code}
Failed with exception javax.jdo.JDOException: Couldnt obtain a new sequence
(unique id) : The MySQL server is running with the --read-only option so it
cannot execute this statement
NestedThrowables:
java.sql.SQLException: The MySQL server is running with the --read-only option
so it cannot execute this statement
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MoveTask
{code}
In this case, the MySQL server could be temporarily in a read-only mode, and a
later DB call may succeed. To handle these situations, this JIRA proposes to
expand the class of exceptions caught for retries.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-2001) Add inputs and outputs to authorization DDL commands
[ https://issues.apache.org/jira/browse/HIVE-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-2001: --- Attachment: HIVE-2001.2.patch Add inputs and outputs to authorization DDL commands Key: HIVE-2001 URL: https://issues.apache.org/jira/browse/HIVE-2001 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.8.0 Reporter: He Yongqiang Assignee: He Yongqiang Attachments: HIVE-2001.2.patch, hive-2001.patch When permissions are changed for a table/partition, the respective object should be present in the read/write entities for hooks to act on. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1644) use filter pushdown for automatically accessing indexes
[ https://issues.apache.org/jira/browse/HIVE-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Russell Melick updated HIVE-1644: - Attachment: HIVE-1644.2.patch Includes modification of MapReduce tasks to use index input format and broken temp file name. use filter pushdown for automatically accessing indexes --- Key: HIVE-1644 URL: https://issues.apache.org/jira/browse/HIVE-1644 Project: Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: Russell Melick Attachments: HIVE-1644.1.patch, HIVE-1644.2.patch HIVE-1226 provides utilities for analyzing filters which have been pushed down to a table scan. The next step is to use these for selecting available indexes and generating access plans for those indexes. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1644) use filter pushdown for automatically accessing indexes
[ https://issues.apache.org/jira/browse/HIVE-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998155#comment-12998155 ] Russell Melick commented on HIVE-1644: -- We also spoke about changing the re-entrant query construction to live within the IndexHandler class. Unfortunately, the Index object can only give us access to the Handler's name as a string, not an instance of it (IndexWhereProcessor.rewriteForIndex). I looked through the codebase some to figure out how classes are loaded from strings, and found several examples of using Class.forName(...). Any suggestions here? use filter pushdown for automatically accessing indexes --- Key: HIVE-1644 URL: https://issues.apache.org/jira/browse/HIVE-1644 Project: Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: Russell Melick Attachments: HIVE-1644.1.patch, HIVE-1644.2.patch HIVE-1226 provides utilities for analyzing filters which have been pushed down to a table scan. The next step is to use these for selecting available indexes and generating access plans for those indexes. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1994) Support new annotation @UDFType(stateful = true)
[ https://issues.apache.org/jira/browse/HIVE-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998171#comment-12998171 ] John Sichi commented on HIVE-1994: -- Note that for CASE expressions, we *always* want short circuiting, otherwise it's impossible to do something like case when x 0 then sqrt(-x) else sqrt(x) end (to avoid trying to take the square root of a negative number). So if we detect a stateful UDF inside of a CASE expression, we'll throw an exception. Support new annotation @UDFType(stateful = true) Key: HIVE-1994 URL: https://issues.apache.org/jira/browse/HIVE-1994 Project: Hive Issue Type: Improvement Components: Query Processor, UDF Reporter: John Sichi Assignee: John Sichi Attachments: HIVE-1994.0.patch Because Hive does not yet support window functions from SQL/OLAP, people have started hacking around it by writing stateful UDF's for things like cumulative sum. An example is row_sequence in contrib. To clearly mark these, I think we should add a new annotation (with separate semantics from the existing deterministic annotation). I'm proposing the name stateful for lack of a better idea, but I'm open to suggestions. The semantics are as follows: * A stateful UDF can only be used in the SELECT list, not in other clauses such as WHERE/ON/ORDER/GROUP * When a stateful UDF is present in a query, there's an implication that its SELECT needs to be treated as similar to TRANSFORM, i.e. when there's DISTRIBUTE/CLUSTER/SORT clause, then run inside the corresponding reducer to make sure that the results are as expected. For the first one, an example of why we need this is AND/OR short-circuiting; we don't want these optimizations to cause the invocation to be skipped in a confusing way, so we should just ban it outright (which is what SQL/OLAP does for window functions). For the second one, I'm not entirely certain about the details since some of it is lost in the mists in Hive prehistory, but at least if we have the annotation, we'll be able to preserve backwards compatibility as we start adding new cost-based optimizations which might otherwise break it. A specific example would be inserting a materialization step (e.g. for global query optimization) in between the DISTRIBUTE/CLUSTER/SORT and the outer SELECT containing the stateful UDF invocation; this could be a problem if the mappers in the second job subdivides the buckets generated by the first job. So we wouldn't do anything immediately, but the presence of the annotation will help us going forward. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1994) Support new annotation @UDFType(stateful = true)
[ https://issues.apache.org/jira/browse/HIVE-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998197#comment-12998197 ] John Sichi commented on HIVE-1994: -- HIVE-1994.1.patch addresses short-circuiting. I'm running it through tests now. Support new annotation @UDFType(stateful = true) Key: HIVE-1994 URL: https://issues.apache.org/jira/browse/HIVE-1994 Project: Hive Issue Type: Improvement Components: Query Processor, UDF Reporter: John Sichi Assignee: John Sichi Attachments: HIVE-1994.0.patch, HIVE-1994.1.patch Because Hive does not yet support window functions from SQL/OLAP, people have started hacking around it by writing stateful UDF's for things like cumulative sum. An example is row_sequence in contrib. To clearly mark these, I think we should add a new annotation (with separate semantics from the existing deterministic annotation). I'm proposing the name stateful for lack of a better idea, but I'm open to suggestions. The semantics are as follows: * A stateful UDF can only be used in the SELECT list, not in other clauses such as WHERE/ON/ORDER/GROUP * When a stateful UDF is present in a query, there's an implication that its SELECT needs to be treated as similar to TRANSFORM, i.e. when there's DISTRIBUTE/CLUSTER/SORT clause, then run inside the corresponding reducer to make sure that the results are as expected. For the first one, an example of why we need this is AND/OR short-circuiting; we don't want these optimizations to cause the invocation to be skipped in a confusing way, so we should just ban it outright (which is what SQL/OLAP does for window functions). For the second one, I'm not entirely certain about the details since some of it is lost in the mists in Hive prehistory, but at least if we have the annotation, we'll be able to preserve backwards compatibility as we start adding new cost-based optimizations which might otherwise break it. A specific example would be inserting a materialization step (e.g. for global query optimization) in between the DISTRIBUTE/CLUSTER/SORT and the outer SELECT containing the stateful UDF invocation; this could be a problem if the mappers in the second job subdivides the buckets generated by the first job. So we wouldn't do anything immediately, but the presence of the annotation will help us going forward. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-818) Create a Hive CLI that connects to hive ThriftServer
[ https://issues.apache.org/jira/browse/HIVE-818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998201#comment-12998201 ] He Yongqiang commented on HIVE-818: --- Ning, can you update a new patch? (Sorry, I just committed 1517, and seems there are some conflicts with this one.) Create a Hive CLI that connects to hive ThriftServer Key: HIVE-818 URL: https://issues.apache.org/jira/browse/HIVE-818 Project: Hive Issue Type: New Feature Components: Clients, Server Infrastructure Reporter: Edward Capriolo Assignee: Ning Zhang Attachments: HIVE-818.2.patch, HIVE-818.3.patch, HIVE-818.4.patch, HIVE-818.patch, Hive-881_2.patch We should have an alternate CLI that works by interacting with the HiveServer, in this way it will be ready when/if we deprecate the current CLI. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1982) Group by key shall not duplicate with distinct key
[ https://issues.apache.org/jira/browse/HIVE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998204#comment-12998204 ] He Yongqiang commented on HIVE-1982: Ted, can you update your patch based on the trunk? (probably just need to regenerate some output files) sorry, after 1517 committed, the patch can not be applied cleanly. Group by key shall not duplicate with distinct key -- Key: HIVE-1982 URL: https://issues.apache.org/jira/browse/HIVE-1982 Project: Hive Issue Type: Bug Affects Versions: 0.6.0, 0.7.0 Reporter: Ted Xu Assignee: Ted Xu Priority: Minor Attachments: HIVE-1982-3.patch, HIVE-1982-v2.patch, HIVE-1982.patch Group by key shall not duplicate with distinct key, or there will be error because RowResolver and ColumnInfo didn't match. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1994) Support new annotation @UDFType(stateful = true)
[ https://issues.apache.org/jira/browse/HIVE-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-1994: - Attachment: HIVE-1994.2.patch Support new annotation @UDFType(stateful = true) Key: HIVE-1994 URL: https://issues.apache.org/jira/browse/HIVE-1994 Project: Hive Issue Type: Improvement Components: Query Processor, UDF Reporter: John Sichi Assignee: John Sichi Attachments: HIVE-1994.0.patch, HIVE-1994.1.patch, HIVE-1994.2.patch Because Hive does not yet support window functions from SQL/OLAP, people have started hacking around it by writing stateful UDF's for things like cumulative sum. An example is row_sequence in contrib. To clearly mark these, I think we should add a new annotation (with separate semantics from the existing deterministic annotation). I'm proposing the name stateful for lack of a better idea, but I'm open to suggestions. The semantics are as follows: * A stateful UDF can only be used in the SELECT list, not in other clauses such as WHERE/ON/ORDER/GROUP * When a stateful UDF is present in a query, there's an implication that its SELECT needs to be treated as similar to TRANSFORM, i.e. when there's DISTRIBUTE/CLUSTER/SORT clause, then run inside the corresponding reducer to make sure that the results are as expected. For the first one, an example of why we need this is AND/OR short-circuiting; we don't want these optimizations to cause the invocation to be skipped in a confusing way, so we should just ban it outright (which is what SQL/OLAP does for window functions). For the second one, I'm not entirely certain about the details since some of it is lost in the mists in Hive prehistory, but at least if we have the annotation, we'll be able to preserve backwards compatibility as we start adding new cost-based optimizations which might otherwise break it. A specific example would be inserting a materialization step (e.g. for global query optimization) in between the DISTRIBUTE/CLUSTER/SORT and the outer SELECT containing the stateful UDF invocation; this could be a problem if the mappers in the second job subdivides the buckets generated by the first job. So we wouldn't do anything immediately, but the presence of the annotation will help us going forward. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1644) use filter pushdown for automatically accessing indexes
[ https://issues.apache.org/jira/browse/HIVE-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998211#comment-12998211 ] John Sichi commented on HIVE-1644: -- For loading index handlers, see HiveUtils.getIndexHandler. For the splitTasks, we'll have to take a closer look. use filter pushdown for automatically accessing indexes --- Key: HIVE-1644 URL: https://issues.apache.org/jira/browse/HIVE-1644 Project: Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: Russell Melick Attachments: HIVE-1644.1.patch, HIVE-1644.2.patch HIVE-1226 provides utilities for analyzing filters which have been pushed down to a table scan. The next step is to use these for selecting available indexes and generating access plans for those indexes. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
