[jira] [Commented] (HIVE-2629) Make a single Hive binary work with both 0.20.x and 0.23.0
[ https://issues.apache.org/jira/browse/HIVE-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13186775#comment-13186775 ] Hudson commented on HIVE-2629: -- Integrated in Hive-trunk-h0.21-dbg #3 (See [https://builds.apache.org/job/Hive-trunk-h0.21-dbg/3/]) HIVE-2629. Make a single Hive binary work with both 0.20.x and 0.23.0. (Thomas Weise via amareshwari) amareshwari : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1230391 Files : * /hive/trunk/build-common.xml * /hive/trunk/build.properties * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SchemaAwareCompressionInputStream.java * /hive/trunk/shims/build.xml * /hive/trunk/shims/ivy.xml * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenIdentifier.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenSelector.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java * /hive/trunk/shims/src/0.23/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java * /hive/trunk/shims/src/0.23/java/org/apache/hadoop/hive/thrift/DelegationTokenIdentifier23.java * /hive/trunk/shims/src/0.23/java/org/apache/hadoop/hive/thrift/DelegationTokenSelector23.java * /hive/trunk/shims/src/common-secure * /hive/trunk/shims/src/common-secure/java * /hive/trunk/shims/src/common-secure/java/org * /hive/trunk/shims/src/common-secure/java/org/apache * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/shims * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenIdentifier.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenSelector.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/client * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/security * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/security/token * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/security/token/delegation * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/ShimLoader.java Make a single Hive binary work with both 0.20.x and 0.23.0 -- Key: HIVE-2629 URL: https://issues.apache.org/jira/browse/HIVE-2629 Project: Hive Issue Type: Bug Components: Shims Reporter: Carl Steinbach Assignee: Thomas Weise Fix For: 0.8.1, 0.9.0 Attachments: HIVE-2629-InputSplitShim.patch, HIVE-2629.3.patch, HIVE-2629.D711.1.patch, HIVE-2629.D711.2.patch, HIVE-2629.D711.3.patch, HIVE-2629.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2589) Newly created partition should inherit properties from table
[ https://issues.apache.org/jira/browse/HIVE-2589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13186779#comment-13186779 ] Hudson commented on HIVE-2589: -- Integrated in Hive-trunk-h0.21-dbg #3 (See [https://builds.apache.org/job/Hive-trunk-h0.21-dbg/3/]) HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1230390 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java * /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q * /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q * /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q * /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out * /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out * /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out Newly created partition should inherit properties from table Key: HIVE-2589 URL: https://issues.apache.org/jira/browse/HIVE-2589 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.9.0 Attachments: hive-2589.patch, hive-2589.patch, hive-2589_1.patch, hive-2589_2.patch This will make all the info contained in table properties available to partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2504) Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory
[ https://issues.apache.org/jira/browse/HIVE-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13186776#comment-13186776 ] Hudson commented on HIVE-2504: -- Integrated in Hive-trunk-h0.21-dbg #3 (See [https://builds.apache.org/job/Hive-trunk-h0.21-dbg/3/]) HIVE-2504 Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory (Chinna Rao Lalam via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1230774 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml.template * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory - Key: HIVE-2504 URL: https://issues.apache.org/jira/browse/HIVE-2504 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Chinna Rao Lalam Attachments: HIVE-2504.patch When the Hive Metastore creates a subdirectory in the Hive warehouse for a new table it does so with the default HDFS permissions. Since the default dfs.umask value is 022, this means that the new subdirectory will not inherit the group write permissions of the hive warehouse directory. We should make the umask used by Warehouse.mkdirs() configurable, and set it to use a default value of 002. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2695) Add PRINTF() Udf
[ https://issues.apache.org/jira/browse/HIVE-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13186778#comment-13186778 ] Hudson commented on HIVE-2695: -- Integrated in Hive-trunk-h0.21-dbg #3 (See [https://builds.apache.org/job/Hive-trunk-h0.21-dbg/3/]) HIVE-2695 Add PRINTF() Udf (Zhenxiao Luo via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1230758 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPrintf.java * /hive/trunk/ql/src/test/queries/clientnegative/udf_printf_wrong1.q * /hive/trunk/ql/src/test/queries/clientnegative/udf_printf_wrong2.q * /hive/trunk/ql/src/test/queries/clientnegative/udf_printf_wrong3.q * /hive/trunk/ql/src/test/queries/clientnegative/udf_printf_wrong4.q * /hive/trunk/ql/src/test/queries/clientpositive/udf_printf.q * /hive/trunk/ql/src/test/results/clientnegative/udf_printf_wrong1.q.out * /hive/trunk/ql/src/test/results/clientnegative/udf_printf_wrong2.q.out * /hive/trunk/ql/src/test/results/clientnegative/udf_printf_wrong3.q.out * /hive/trunk/ql/src/test/results/clientnegative/udf_printf_wrong4.q.out * /hive/trunk/ql/src/test/results/clientpositive/show_functions.q.out * /hive/trunk/ql/src/test/results/clientpositive/udf_printf.q.out Add PRINTF() Udf Key: HIVE-2695 URL: https://issues.apache.org/jira/browse/HIVE-2695 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Attachments: HIVE-2695.D1155.1.patch, HIVE-2695.D1161.1.patch, HIVE-2695.D1173.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2682) Clean-up logs
[ https://issues.apache.org/jira/browse/HIVE-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13186777#comment-13186777 ] Hudson commented on HIVE-2682: -- Integrated in Hive-trunk-h0.21-dbg #3 (See [https://builds.apache.org/job/Hive-trunk-h0.21-dbg/3/]) HIVE-2682: Clean-up logs (Rajat Goel via Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1230379 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/stats/jdbc/JDBCStatsAggregator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/stats/jdbc/JDBCStatsPublisher.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyStruct.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryStruct.java Clean-up logs - Key: HIVE-2682 URL: https://issues.apache.org/jira/browse/HIVE-2682 Project: Hive Issue Type: Wish Components: Logging Affects Versions: 0.8.1, 0.9.0 Reporter: Rajat Goel Assignee: Rajat Goel Priority: Trivial Labels: logging Fix For: 0.9.0 Attachments: HIVE-2682.D1035.1.patch, HIVE-2682.D1035.2.patch, HIVE-2682.D1035.3.patch, hive-2682.patch Original Estimate: 24h Remaining Estimate: 24h Just wanted to cleanup some logs being printed at wrong loglevel - 1. org.apache.hadoop.hive.ql.exec.CommonJoinOperator prints table 0 has 1000 rows for join key [...] as WARNING. Is it really that? 2. org.apache.hadoop.hive.ql.exec.GroupByOperator prints Hash Table completed flushed and Begin Hash Table flush at close: size = 21 as WARNING. It shouldn't be. 3. org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher prints Warning. Invalid statistic. which looks fishy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2673) Eclipse launch configurations fail due to unsatisfied builtins JAR dependency
[ https://issues.apache.org/jira/browse/HIVE-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reassigned HIVE-2673: Assignee: Carl Steinbach (was: John Sichi) Eclipse launch configurations fail due to unsatisfied builtins JAR dependency - Key: HIVE-2673 URL: https://issues.apache.org/jira/browse/HIVE-2673 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.8.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2673) Eclipse launch configurations fail due to unsatisfied builtins JAR dependency
[ https://issues.apache.org/jira/browse/HIVE-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2673: - Status: Patch Available (was: Open) Eclipse launch configurations fail due to unsatisfied builtins JAR dependency - Key: HIVE-2673 URL: https://issues.apache.org/jira/browse/HIVE-2673 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.8.1 Attachments: HIVE-2673.1.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2673) Eclipse launch configurations fail due to unsatisfied builtins JAR dependency
[ https://issues.apache.org/jira/browse/HIVE-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2673: - Attachment: HIVE-2673.1.patch.txt Eclipse launch configurations fail due to unsatisfied builtins JAR dependency - Key: HIVE-2673 URL: https://issues.apache.org/jira/browse/HIVE-2673 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.8.1 Attachments: HIVE-2673.1.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2203) Extend concat_ws() UDF to support arrays of strings
[ https://issues.apache.org/jira/browse/HIVE-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2203: - Status: Open (was: Patch Available) @Zhenxiao: Please attach a copy of the patch (D1137) and give license rights to Apache. Extend concat_ws() UDF to support arrays of strings --- Key: HIVE-2203 URL: https://issues.apache.org/jira/browse/HIVE-2203 Project: Hive Issue Type: Improvement Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Priority: Minor Attachments: HIVE-2203.D1065.1.patch, HIVE-2203.D1071.1.patch, HIVE-2203.D1113.1.patch, HIVE-2203.D1119.1.patch, HIVE-2203.D1137.1.patch concat_ws() should support the following type of input parameters: concat_ws(string separator, arraystring) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2203) Extend concat_ws() UDF to support arrays of strings
[ https://issues.apache.org/jira/browse/HIVE-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13186824#comment-13186824 ] Phabricator commented on HIVE-2203: --- cwsteinbach has accepted the revision HIVE-2203 [jira] Extend concat_ws() UDF to support arrays of strings. +1 REVISION DETAIL https://reviews.facebook.net/D1137 Extend concat_ws() UDF to support arrays of strings --- Key: HIVE-2203 URL: https://issues.apache.org/jira/browse/HIVE-2203 Project: Hive Issue Type: Improvement Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Priority: Minor Attachments: HIVE-2203.D1065.1.patch, HIVE-2203.D1071.1.patch, HIVE-2203.D1113.1.patch, HIVE-2203.D1119.1.patch, HIVE-2203.D1137.1.patch concat_ws() should support the following type of input parameters: concat_ws(string separator, arraystring) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive JOIN fails if SELECT statement contains fields from the first table.
1. I create two Hive table:
Hive CREATE EXTERNAL TABLE student_details (studentid INT,studentname
STRING,age INT,gpa FLOAT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE LOCATION ‘/home/biadmin/hivetbl';
HiveCREATE EXTERNAL TABLE student_score(studentid INT, classid INT,score
FLOAT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE
LOCATION '/home/biadmin/hivetbl';
2. Load data
HIVELOAD DATA LOCAL INPATH '/home/biadmin/hivetbl/student_details.txt'
OVERWRITE INTO TABLE student_details;
HIVELOAD DATA LOCAL INPATH '/home/biadmin/hivetbl/student_score.txt'
OVERWRITE INTO TABLE student_score;
3. Run inner join
Hive SELECT a.studentid,a.studentname,a.age,b.classid,b.score,c.classname
FROM student_details a JOIN student_score b ON (a.studentid = b.studentid);
Result:
There are the following exception:
cannot find field studentname from [0:studentid, 1:classid, 2:score]
[My Question]: studentname is a field of the table student_details (The
first table), why search it in the table student_score(the second table)?
log is like that;
... ...
2012-01-15 23:24:41,727 INFO org.apache.hadoop.mapred.TaskInProgress: Error
from attempt_201201152221_0014_m_00_3: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {studentid:106,classid:null,score:635.0}
at
org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row {studentid:106,classid:null,score:635.0}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:550)
at
org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
... 4 more
Caused by: java.lang.RuntimeException: cannot find field studentname from
[0:studentid, 1:classid, 2:score]
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:345)
at
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldRef(LazySimpleStructObjectInspector.java:168)
at
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:57)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:896)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:922)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:200)
at
org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at
org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:531)
... 5 more
[jira] [Updated] (HIVE-2694) Add FORMAT UDF
[ https://issues.apache.org/jira/browse/HIVE-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2694: - Status: Open (was: Patch Available) Add FORMAT UDF -- Key: HIVE-2694 URL: https://issues.apache.org/jira/browse/HIVE-2694 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Attachments: HIVE-2694.D1149.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2694) Add FORMAT UDF
[ https://issues.apache.org/jira/browse/HIVE-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13186848#comment-13186848 ] Phabricator commented on HIVE-2694: --- cwsteinbach has requested changes to the revision HIVE-2694 [jira] Add FORMAT UDF. INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFormatNumber.java:65 Instead of string comparisons on the type name, please do the following to verify the types: * Verify that the two OIs are PrimitiveObjectInspectors. * Cast both OIs to PrimtiveObjectInspector and then validate the type in a switch statement using PrimitiveObjectInspector.getPrimitiveCategory(). ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFormatNumber.java:106 Let's make the StringBuilder a private instance variable and initialize it once in the constructor, and then clear it every time evaluate gets called: pattern.delete(0, pattern.length()) ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFormatNumber.java:139 I think it should only be necessary to construct a new DecimalFormat object when evaluate is called with a new dValue. Otherwise, it seems like it should be possible to reuse the DecimalFormat object from the previous evaluation. ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFormatNumber.java:114 This doesn't look right to me. It's not possible to construct a single pattern that tells DecimalFormat to use the grouping separator for thousands? REVISION DETAIL https://reviews.facebook.net/D1149 Add FORMAT UDF -- Key: HIVE-2694 URL: https://issues.apache.org/jira/browse/HIVE-2694 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Attachments: HIVE-2694.D1149.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2691) Specify location of log4j configuration files via configuration properties
[
https://issues.apache.org/jira/browse/HIVE-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13186862#comment-13186862
]
Phabricator commented on HIVE-2691:
---
cwsteinbach has requested changes to the revision HIVE-2691 [jira] Specify
location of log4j configuration files via configuration properties.
INLINE COMMENTS
cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java:566 Shouldn't
CliSessionState be initialized after logging is initialized?
cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java:565 Better to just
call this conf instead of log4jconf
common/src/java/org/apache/hadoop/hive/common/LogUtils.java:105 Please move
this piece of code back to ExecDriver. It's out of place here.
common/src/java/org/apache/hadoop/hive/common/LogUtils.java:87 This code does
not belong here.
common/src/java/org/apache/hadoop/hive/common/LogUtils.java:102 What about
HIVE_EXEC_L4J?
common/src/java/org/apache/hadoop/hive/common/LogUtils.java:119 Please change
this method to return void and instead write to the log directly from this
method.
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:902 Remove this
method. Folks can call getPath() on their own.
common/src/java/org/apache/hadoop/hive/common/LogUtils.java:55 initHiveLog4j
can create its own HiveConf object, in which case you don't need to check to
see if the input arg is null. I think this approach is preferable.
REVISION DETAIL
https://reviews.facebook.net/D1203
Specify location of log4j configuration files via configuration properties
--
Key: HIVE-2691
URL: https://issues.apache.org/jira/browse/HIVE-2691
Project: Hive
Issue Type: New Feature
Components: Configuration, Logging
Reporter: Carl Steinbach
Assignee: Zhenxiao Luo
Attachments: HIVE-2691.D1131.1.patch, HIVE-2691.D1203.1.patch,
HIVE-2691.D1203.2.patch, HIVE-2691.D1203.3.patch, HIVE-2691.D1203.4.patch,
HIVE-2691.D1203.5.patch
Oozie needs to be able to override the default location of the log4j
configuration
files from the Hive command line, e.g:
{noformat}
hive -hiveconf hive.log4j.file=/home/carl/hive-log4j.properties -hiveconf
hive.log4j.exec.file=/home/carl/hive-exec-log4j.properties
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2589) Newly created partition should inherit properties from table
[ https://issues.apache.org/jira/browse/HIVE-2589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13186895#comment-13186895 ] Hudson commented on HIVE-2589: -- Integrated in Hive-trunk-h0.21-dbg #4 (See [https://builds.apache.org/job/Hive-trunk-h0.21-dbg/4/]) HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1230390 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java * /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q * /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q * /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q * /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out * /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out * /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out Newly created partition should inherit properties from table Key: HIVE-2589 URL: https://issues.apache.org/jira/browse/HIVE-2589 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.9.0 Attachments: hive-2589.patch, hive-2589.patch, hive-2589_1.patch, hive-2589_2.patch This will make all the info contained in table properties available to partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2682) Clean-up logs
[ https://issues.apache.org/jira/browse/HIVE-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13187020#comment-13187020 ] Hudson commented on HIVE-2682: -- Integrated in Hive-trunk-h0.21-dbg #5 (See [https://builds.apache.org/job/Hive-trunk-h0.21-dbg/5/]) HIVE-2682: Clean-up logs (Rajat Goel via Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1230379 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/stats/jdbc/JDBCStatsAggregator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/stats/jdbc/JDBCStatsPublisher.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyStruct.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryStruct.java Clean-up logs - Key: HIVE-2682 URL: https://issues.apache.org/jira/browse/HIVE-2682 Project: Hive Issue Type: Wish Components: Logging Affects Versions: 0.8.1, 0.9.0 Reporter: Rajat Goel Assignee: Rajat Goel Priority: Trivial Labels: logging Fix For: 0.9.0 Attachments: HIVE-2682.D1035.1.patch, HIVE-2682.D1035.2.patch, HIVE-2682.D1035.3.patch, hive-2682.patch Original Estimate: 24h Remaining Estimate: 24h Just wanted to cleanup some logs being printed at wrong loglevel - 1. org.apache.hadoop.hive.ql.exec.CommonJoinOperator prints table 0 has 1000 rows for join key [...] as WARNING. Is it really that? 2. org.apache.hadoop.hive.ql.exec.GroupByOperator prints Hash Table completed flushed and Begin Hash Table flush at close: size = 21 as WARNING. It shouldn't be. 3. org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher prints Warning. Invalid statistic. which looks fishy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2705) SemanticAnalyzer twice swallows an exception it shouldn't
[ https://issues.apache.org/jira/browse/HIVE-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13187021#comment-13187021 ] Hudson commented on HIVE-2705: -- Integrated in Hive-trunk-h0.21-dbg #5 (See [https://builds.apache.org/job/Hive-trunk-h0.21-dbg/5/]) HIVE-2705: SemanticAnalyzer twice swallows an exception it shouldn't (jghoman via hashutosh) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1230091 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java SemanticAnalyzer twice swallows an exception it shouldn't - Key: HIVE-2705 URL: https://issues.apache.org/jira/browse/HIVE-2705 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.1 Reporter: Jakob Homan Assignee: Jakob Homan Fix For: 0.9.0 Attachments: HIVE-2705.patch Twice SemanticAnalyzer catches an exception and drops it, just passing on the original message's in a new SemanticException. This means that those that see the message in the output cannot tell what generated the original exception. These original exceptions should be wrapped, as they are in other parts of the code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2589) Newly created partition should inherit properties from table
[ https://issues.apache.org/jira/browse/HIVE-2589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13187022#comment-13187022 ] Hudson commented on HIVE-2589: -- Integrated in Hive-trunk-h0.21-dbg #5 (See [https://builds.apache.org/job/Hive-trunk-h0.21-dbg/5/]) HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1230390 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java * /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q * /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q * /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q * /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out * /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out * /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out Newly created partition should inherit properties from table Key: HIVE-2589 URL: https://issues.apache.org/jira/browse/HIVE-2589 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.9.0 Attachments: hive-2589.patch, hive-2589.patch, hive-2589_1.patch, hive-2589_2.patch This will make all the info contained in table properties available to partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
large sql file creating large num of columns
Hi All, I have a SQL file of size 30mb which is a single create table statement with about 800,000 columns, hence the size. I am trying to execute it using hive -f file. Initially, hive ran the command with 256mb heap size and gave me an OOM error. I increased the heap size using export HADOOP_HEAPSIZE to 1 gb and eventually 2gb which made the OOM error go away. However, the hive command ran for 5 hours without actually creating the table. The JVM was running. However, 1. running a strace on the process showed that it was stuck on a futex call. 2. I am using mysql for metastore and there were no rows added to either TBLS or COLUMNS table. Question. 1. can hive do this create table of 800k columns from a sql file of 30mb? 2. if theoretically possible, what could be happening that's taking it over 5 hours and still not succeeding? Running it with debug, it spews the following, ---snip stored as textfile location 'myfile' 12/01/16 11:28:54 INFO parse.ParseDriver: Parse Completed 12/01/16 11:28:54 INFO parse.SemanticAnalyzer: Starting Semantic Analysis 12/01/16 11:28:54 INFO parse.SemanticAnalyzer: Creating table my_table position=22 and it's stuck there at SemanticAnalyzer... thanks for any insight. sincerely, ameet
Re: large sql file creating large num of columns
thanks, Running it with debug, it spews the following, stored as textfile location 'myfile' 12/01/16 11:28:54 INFO parse.ParseDriver: Parse Completed 12/01/16 11:28:54 INFO parse.SemanticAnalyzer: Starting Semantic Analysis 12/01/16 11:28:54 INFO parse.SemanticAnalyzer: Creating table my_table position=22 and it's waiting there at SemanticAnalyzer... Sincerely, Ameet From: Bejoy Ks bejoy...@yahoo.com To: u...@hive.apache.org u...@hive.apache.org; Edward Capriolo edlinuxg...@gmail.com; ameet chaubal ameetchau...@yahoo.com Sent: Monday, January 16, 2012 10:24 AM Subject: Re: large sql file creating large num of columns Hey Ameet Please find some pointers inline. All that hive is supposed to do is to load the definition into mysql, right? [Bejoy] Yes you are right Are you suggesting that it's reading the datafile in HDFS? [Bejoy] AFAIK it won't do that at the time of table creation. Just meta data entries happen at this stage. That should not be happening since the external table does not need the data to be present, right? [Bejoy] Again your understanding is right. I can't give you much of a hint on why the query takes 5 hrs cos I've never tried out such large number of columns. What you can do at this point is enable DEBUG logging in hive and try to get more some stack trace and see whether it is some issue with parsing the DDL or while making data base calls for storing metadata. Regards Bejoy.K.S From: ameet chaubal ameetchau...@yahoo.com To: Edward Capriolo edlinuxg...@gmail.com; u...@hive.apache.org u...@hive.apache.org Sent: Monday, January 16, 2012 8:44 PM Subject: Re: large sql file creating large num of columns thanks, this is an external table; so at the DDL stage, there is no data loading that is happening. All that hive is supposed to do is to load the definition into mysql, right? Are you suggesting that it's reading the datafile in HDFS? That should not be happening since the external table does not need the data to be present, right? Sincerely, Ameet From: Edward Capriolo edlinuxg...@gmail.com To: u...@hive.apache.org; ameet chaubal ameetchau...@yahoo.com Sent: Monday, January 16, 2012 10:06 AM Subject: Re: large sql file creating large num of columns I highly doubt this will work. I think that many things in hadoop and hive will try to buffer an entire row so even if you make it past the metastore I do not think it will be of any use. On Mon, Jan 16, 2012 at 9:42 AM, ameet chaubal ameetchau...@yahoo.com wrote: Hi All, I have a SQL file of size 30mb which is a single create table statement with about 800,000 columns, hence the size. I am trying to execute it using hive -f file. Initially, hive ran the command with 256mb heap size and gave me an OOM error. I increased the heap size using export HADOOP_HEAPSIZE to 1 gb and eventually 2gb which made the OOM error go away. However, the hive command ran for 5 hours without actually creating the table. The JVM was running. However, 1. running a strace on the process showed that it was stuck on a futex call. 2. I am using mysql for metastore and there were no rows added to either TBLS or COLUMNS table. Question. 1. can hive do this create table of 800k columns from a sql file of 30mb? 2. if theoretically possible, what could be happening that's taking it over 5 hours and still not succeeding? any insight is much appreciated. Sincerely, Ameet
Re: Hive JOIN fails if SELECT statement contains fields from the first table.
Hi Bing,
Something seems wrong about your create table statements.
You are using LOAD DATA LOCAL INPATH to load data into Hive tables. This
makes me think that the files /home/biadmin/hivetbl/student_details.txt and
/home/biadmin/hivetbl/student_score.txt are on the local drive.
In such a case you want to copy them onto HDFS for use by Hive tables and if
so, doing LOAD DATA LOCAL INPATH is the right thing to do.
However, when you create a table and specify the location
...STORED AS TEXTFILE LOCATION ‘/home/biadmin/hivetbl'...
This location refers to the HDFS location. If you don't specify this, Hive will
choose this by for you by default. However, if you do specify it, it's your
responsibility to ensure that this location doesn't conflict with another Hive
table.
Long story short, a leaf level directory in HDFS stores the partitions/buckets
and contents of 1 table. In your create table statement, you were pointing both
Hive tables to the same HDFS location /home/biadmin/hivetbl.
If you could make your two tables, point to different HDFS locations like,
'/usr/hive/warehouse/student_score' and '/usr/hive/warehouse/student_details'
and then do your load data inpath statements, that should fix the problem.
Good luck!
Mark
Mark Grover, Business Intelligence Analyst
OANDA Corporation
www: oanda.com www: fxtrade.com
e: mgro...@oanda.com
Best Trading Platform - World Finance's Forex Awards 2009.
The One to Watch - Treasury Today's Adam Smith Awards 2009.
- Original Message -
From: Bing Li smallpu...@gmail.com
To: dev@hive.apache.org, u...@hive.apache.org
Sent: Monday, January 16, 2012 5:06:00 AM
Subject: Hive JOIN fails if SELECT statement contains fields from the first
table.
1. I create two Hive table:
Hive CREATE EXTERNAL TABLE student_details (studentid INT,studentname
STRING,age INT,gpa FLOAT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED
AS TEXTFILE LOCATION ‘/home/biadmin/hivetbl';
HiveCREATE EXTERNAL TABLE student_score(studentid INT, classid INT,score
FLOAT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE
LOCATION '/home/biadmin/hivetbl';
2. Load data
HIVELOAD DATA LOCAL INPATH '/home/biadmin/hivetbl/student_details.txt'
OVERWRITE INTO TABLE student_details;
HIVELOAD DATA LOCAL INPATH '/home/biadmin/hivetbl/student_score.txt' OVERWRITE
INTO TABLE student_score;
3. Run inner join
Hive SELECT a.studentid,a.studentname,a.age,b.classid,b.score,c.classname FROM
student_details a JOIN student_score b ON (a.studentid = b.studentid);
Result:
There are the following exception:
cannot find field studentname from [0:studentid, 1:classid, 2:score]
[My Question]: studentname is a field of the table student_details (The first
table), why search it in the table student_score(the second table)?
log is like that;
... ...
2012-01-15 23:24:41,727 INFO org.apache.hadoop.mapred.TaskInProgress: Error
from attempt_201201152221_0014_m_00_3: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {studentid:106,classid:null,score:635.0}
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row {studentid:106,classid:null,score:635.0}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:550)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
... 4 more
Caused by: java.lang.RuntimeException: cannot find field studentname from
[0:studentid, 1:classid, 2:score]
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:345)
at
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldRef(LazySimpleStructObjectInspector.java:168)
at
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:57)
at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:896)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:922)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:200)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:531)
... 5 more
[jira] [Updated] (HIVE-2279) Implement sort(array) UDF
[ https://issues.apache.org/jira/browse/HIVE-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo updated HIVE-2279: --- Attachment: HIVE-2279.D1143.1.patch Implement sort(array) UDF - Key: HIVE-2279 URL: https://issues.apache.org/jira/browse/HIVE-2279 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Attachments: HIVE-2279.D1059.1.patch, HIVE-2279.D1101.1.patch, HIVE-2279.D1107.1.patch, HIVE-2279.D1125.1.patch, HIVE-2279.D1143.1.patch, HIVE-2279.D1143.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2279) Implement sort(array) UDF
[ https://issues.apache.org/jira/browse/HIVE-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo updated HIVE-2279: --- Status: Patch Available (was: Open) Implement sort(array) UDF - Key: HIVE-2279 URL: https://issues.apache.org/jira/browse/HIVE-2279 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Attachments: HIVE-2279.D1059.1.patch, HIVE-2279.D1101.1.patch, HIVE-2279.D1107.1.patch, HIVE-2279.D1125.1.patch, HIVE-2279.D1143.1.patch, HIVE-2279.D1143.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2203) Extend concat_ws() UDF to support arrays of strings
[ https://issues.apache.org/jira/browse/HIVE-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo updated HIVE-2203: --- Status: Patch Available (was: Open) Extend concat_ws() UDF to support arrays of strings --- Key: HIVE-2203 URL: https://issues.apache.org/jira/browse/HIVE-2203 Project: Hive Issue Type: Improvement Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Priority: Minor Attachments: HIVE-2203.D1065.1.patch, HIVE-2203.D1071.1.patch, HIVE-2203.D1113.1.patch, HIVE-2203.D1119.1.patch, HIVE-2203.D1137.1.patch, HIVE-2203.D1137.1.patch concat_ws() should support the following type of input parameters: concat_ws(string separator, arraystring) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2203) Extend concat_ws() UDF to support arrays of strings
[ https://issues.apache.org/jira/browse/HIVE-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo updated HIVE-2203: --- Attachment: HIVE-2203.D1137.1.patch Extend concat_ws() UDF to support arrays of strings --- Key: HIVE-2203 URL: https://issues.apache.org/jira/browse/HIVE-2203 Project: Hive Issue Type: Improvement Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Priority: Minor Attachments: HIVE-2203.D1065.1.patch, HIVE-2203.D1071.1.patch, HIVE-2203.D1113.1.patch, HIVE-2203.D1119.1.patch, HIVE-2203.D1137.1.patch, HIVE-2203.D1137.1.patch concat_ws() should support the following type of input parameters: concat_ws(string separator, arraystring) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2719) Fix test failures caused by HIVE-2589
[ https://issues.apache.org/jira/browse/HIVE-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2719: - Summary: Fix test failures caused by HIVE-2589 (was: Fix test failures caused by HIVE-2589 or HIVE-2682) Looks like the test failures are tied to HIVE-2589: https://builds.apache.org/job/Hive-trunk-h0.21-dbg/4/ @Ashutosh: Do you think you can have a fix for this ready in the next couple of days? If not, I think we should consider temporarily backing this patch out. Fix test failures caused by HIVE-2589 - Key: HIVE-2719 URL: https://issues.apache.org/jira/browse/HIVE-2719 Project: Hive Issue Type: Task Reporter: Carl Steinbach Assignee: Ashutosh Chauhan Priority: Blocker 149 new test failures appeared on Hudson after HIVE-2589 and HIVE-2682 were committed: https://builds.apache.org/job/Hive-trunk-h0.21/1195/ @Ashutosh: Can you please take a look? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2719) Fix test failures caused by HIVE-2589
[ https://issues.apache.org/jira/browse/HIVE-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13187082#comment-13187082 ] Carl Steinbach commented on HIVE-2719: -- I was also able to reproduce the test failures on our internal Jenkins server, so this is not something unique to Apache Hudson. Fix test failures caused by HIVE-2589 - Key: HIVE-2719 URL: https://issues.apache.org/jira/browse/HIVE-2719 Project: Hive Issue Type: Task Reporter: Carl Steinbach Assignee: Ashutosh Chauhan Priority: Blocker 149 new test failures appeared on Hudson after HIVE-2589 and HIVE-2682 were committed: https://builds.apache.org/job/Hive-trunk-h0.21/1195/ @Ashutosh: Can you please take a look? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-0.8.1-SNAPSHOT-h0.21 - Build # 163 - Failure
Changes for Build #163 1 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:9440) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785) The Apache Jenkins build system has built Hive-0.8.1-SNAPSHOT-h0.21 (build #163) Status: Failure Check console output at https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/163/ to view the results.
[jira] [Updated] (HIVE-2691) Specify location of log4j configuration files via configuration properties
[
https://issues.apache.org/jira/browse/HIVE-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2691:
--
Attachment: HIVE-2691.D1203.6.patch
zhenxiao updated the revision HIVE-2691 [jira] Specify location of log4j
configuration files via configuration properties.
Reviewers: JIRA, cwsteinbach
HIVE-2691: Carl's comments addressed
initHiveLog4j() and initHiveExecLog4j() constucts their own
HiveConf
Log4j init callers' code remains unchanged
REVISION DETAIL
https://reviews.facebook.net/D1203
AFFECTED FILES
common/src/java/org/apache/hadoop/hive/common/LogUtils.java
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java
common/src/test/org/apache/hadoop/hive/conf/TestHiveLogging.java
common/src/test/resources/hive-exec-log4j-test.properties
common/src/test/resources/hive-log4j-test.properties
conf/hive-default.xml.template
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java
service/src/java/org/apache/hadoop/hive/service/HiveServer.java
Specify location of log4j configuration files via configuration properties
--
Key: HIVE-2691
URL: https://issues.apache.org/jira/browse/HIVE-2691
Project: Hive
Issue Type: New Feature
Components: Configuration, Logging
Reporter: Carl Steinbach
Assignee: Zhenxiao Luo
Attachments: HIVE-2691.D1131.1.patch, HIVE-2691.D1203.1.patch,
HIVE-2691.D1203.2.patch, HIVE-2691.D1203.3.patch, HIVE-2691.D1203.4.patch,
HIVE-2691.D1203.5.patch, HIVE-2691.D1203.6.patch
Oozie needs to be able to override the default location of the log4j
configuration
files from the Hive command line, e.g:
{noformat}
hive -hiveconf hive.log4j.file=/home/carl/hive-log4j.properties -hiveconf
hive.log4j.exec.file=/home/carl/hive-exec-log4j.properties
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2656) Add a command line flag to specify a jar whence to load UDFs
[ https://issues.apache.org/jira/browse/HIVE-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13187253#comment-13187253 ] Phabricator commented on HIVE-2656: --- jsichi has commented on the revision HIVE-2656 [jira] Add a command line flag to specify a jar whence to load UDFs. Got a boatload of errors when running through latest with ant test...haven't looked into them yet. REVISION DETAIL https://reviews.facebook.net/D1023 Add a command line flag to specify a jar whence to load UDFs Key: HIVE-2656 URL: https://issues.apache.org/jira/browse/HIVE-2656 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang Attachments: HIVE-2656.D1023.1.patch, HIVE-2656.D1023.2.patch To extend the functionality of Hive, it is useful to be able to easily deploy packages of UDFs and register the functions therein. Right now it is a bit cumbersome to do ADD JAR and CREATE TEMPORARY FUNCTION. This can be greatly simplified by adding a simple command line flag that will leverage the code in HIVE-2523 to load all the functions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2720) Merge MetaStoreListener and HiveMetaHook interfaces
Merge MetaStoreListener and HiveMetaHook interfaces --- Key: HIVE-2720 URL: https://issues.apache.org/jira/browse/HIVE-2720 Project: Hive Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar MetaStoreListener and HiveMetaHook both serve as a notification mechanism for metastore-related events. The former is used by hcat and the latter is by the hbase-storage handler, and invoked by the client. I propose to merge these interfaces, and extend the MetaStoreListener, to add most of the on- and pre- methods at the Thrift interface. This way, extending metastore will be easier, and validation, storage-driver notification, and enforcement can be delegated to individual listeners. Besides, more functionality can be plugged-in by Hcat at this level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2694) Add FORMAT UDF
[ https://issues.apache.org/jira/browse/HIVE-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2694: -- Attachment: HIVE-2694.D1149.2.patch zhenxiao updated the revision HIVE-2694 [jira] Add FORMAT UDF. Reviewers: JIRA, cwsteinbach HIVE-2694: Carl's comments addressed. New DecimalFormat constructed only if evaluated with a new dValue REVISION DETAIL https://reviews.facebook.net/D1149 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFormatNumber.java ql/src/test/queries/clientnegative/udf_format_number_wrong1.q ql/src/test/queries/clientnegative/udf_format_number_wrong2.q ql/src/test/queries/clientnegative/udf_format_number_wrong3.q ql/src/test/queries/clientnegative/udf_format_number_wrong4.q ql/src/test/queries/clientnegative/udf_format_number_wrong5.q ql/src/test/queries/clientnegative/udf_format_number_wrong6.q ql/src/test/queries/clientnegative/udf_format_number_wrong7.q ql/src/test/queries/clientnegative/udf_format_number_wrong8.q ql/src/test/queries/clientpositive/udf_format_number.q ql/src/test/results/clientnegative/udf_format_number_wrong1.q.out ql/src/test/results/clientnegative/udf_format_number_wrong2.q.out ql/src/test/results/clientnegative/udf_format_number_wrong3.q.out ql/src/test/results/clientnegative/udf_format_number_wrong4.q.out ql/src/test/results/clientnegative/udf_format_number_wrong5.q.out ql/src/test/results/clientnegative/udf_format_number_wrong6.q.out ql/src/test/results/clientnegative/udf_format_number_wrong7.q.out ql/src/test/results/clientnegative/udf_format_number_wrong8.q.out ql/src/test/results/clientpositive/show_functions.q.out ql/src/test/results/clientpositive/udf_format_number.q.out Add FORMAT UDF -- Key: HIVE-2694 URL: https://issues.apache.org/jira/browse/HIVE-2694 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Attachments: HIVE-2694.D1149.1.patch, HIVE-2694.D1149.2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2694) Add FORMAT UDF
[ https://issues.apache.org/jira/browse/HIVE-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo updated HIVE-2694: --- Status: Patch Available (was: Open) Add FORMAT UDF -- Key: HIVE-2694 URL: https://issues.apache.org/jira/browse/HIVE-2694 Project: Hive Issue Type: New Feature Components: UDF Reporter: Carl Steinbach Assignee: Zhenxiao Luo Attachments: HIVE-2694.D1149.1.patch, HIVE-2694.D1149.2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2720) Merge MetaStoreListener and HiveMetaHook interfaces
[ https://issues.apache.org/jira/browse/HIVE-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13187362#comment-13187362 ] Enis Soztutar commented on HIVE-2720: - MetaStoreListener.onXXX() methods take an Event object to wrap the function call context, however HiveMetaHook functions does not. We can do either way, but the first one will introduce a lot of classes (50+) in case we do add corresponding methods. Since we want both preXXX() and onXXX() methods, we can do smt like: preXXX() takes normal function args + context (containing back pointer to handler) onXXX() takes normal function args + returned object + context (containing back pointer to handler + method status) Anyone care to comment ? Merge MetaStoreListener and HiveMetaHook interfaces --- Key: HIVE-2720 URL: https://issues.apache.org/jira/browse/HIVE-2720 Project: Hive Issue Type: Sub-task Components: JDBC, Metastore, ODBC, Security Reporter: Enis Soztutar Assignee: Enis Soztutar MetaStoreListener and HiveMetaHook both serve as a notification mechanism for metastore-related events. The former is used by hcat and the latter is by the hbase-storage handler, and invoked by the client. I propose to merge these interfaces, and extend the MetaStoreListener, to add most of the on- and pre- methods at the Thrift interface. This way, extending metastore will be easier, and validation, storage-driver notification, and enforcement can be delegated to individual listeners. Besides, more functionality can be plugged-in by Hcat at this level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2721) ability to select a view qualified by the database / schema name
ability to select a view qualified by the database / schema name Key: HIVE-2721 URL: https://issues.apache.org/jira/browse/HIVE-2721 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.8.0, 0.7.1, 0.7.0 Reporter: Robert Morton Priority: Blocker HIVE-1517 added support for selecting tables from different databases (aka schemas) by qualifying the tables with the database name. The feature work did not however extend this support to views. Note that this point came up in the earlier JIRA, but was not addressed. See the following two comments: https://issues.apache.org/jira/browse/HIVE-1517?focusedCommentId=12996641page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12996641 https://issues.apache.org/jira/browse/HIVE-1517?focusedCommentId=12996679page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12996679 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2722) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better Key: HIVE-2722 URL: https://issues.apache.org/jira/browse/HIVE-2722 Project: Hive Issue Type: Improvement Components: UDF Affects Versions: 0.8.0 Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux java version 1.6.0_25 hadoop-0.20.2-cdh3u0 hive-0.7.0-cdh3u0 Reporter: caofangkun Priority: Minor Fix For: 0.9.0 select instr(中文字符测试-第一行,-) from testTable limit 1; result:19 (one Chinese Character was considered as 3 Unicode bits) select substr(中文字符测试-第一行,1,2) from testTable limit 1; result: 中文 (one Chinese Character was considered as 1 Unicode Unit ) instr should considered one chinese character as one Unicode Unit too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2722) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
[ https://issues.apache.org/jira/browse/HIVE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caofangkun updated HIVE-2722: - Attachment: HIVE-2722.patch use CharBuffer GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better Key: HIVE-2722 URL: https://issues.apache.org/jira/browse/HIVE-2722 Project: Hive Issue Type: Improvement Components: UDF Affects Versions: 0.8.0 Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux java version 1.6.0_25 hadoop-0.20.2-cdh3u0 hive-0.7.0-cdh3u0 Reporter: caofangkun Priority: Minor Labels: udf Fix For: 0.9.0 Attachments: HIVE-2722.patch Original Estimate: 1h Remaining Estimate: 1h select instr(中文字符测试-第一行,-) from testTable limit 1; result:19 (one Chinese Character was considered as 3 Unicode bits) select substr(中文字符测试-第一行,1,2) from testTable limit 1; result: 中文 (one Chinese Character was considered as 1 Unicode Unit ) instr should considered one chinese character as one Unicode Unit too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2722) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
[ https://issues.apache.org/jira/browse/HIVE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caofangkun updated HIVE-2722: - Attachment: udf_instr_1.q GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better Key: HIVE-2722 URL: https://issues.apache.org/jira/browse/HIVE-2722 Project: Hive Issue Type: Improvement Components: UDF Affects Versions: 0.8.0 Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux java version 1.6.0_25 hadoop-0.20.2-cdh3u0 hive-0.7.0-cdh3u0 Reporter: caofangkun Priority: Minor Labels: udf Fix For: 0.9.0 Attachments: HIVE-2722.patch, udf_instr_1.q Original Estimate: 1h Remaining Estimate: 1h select instr(中文字符测试-第一行,-) from testTable limit 1; result:19 (one Chinese Character was considered as 3 Unicode bits) select substr(中文字符测试-第一行,1,2) from testTable limit 1; result: 中文 (one Chinese Character was considered as 1 Unicode Unit ) instr should considered one chinese character as one Unicode Unit too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2722) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
[ https://issues.apache.org/jira/browse/HIVE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caofangkun updated HIVE-2722: - Release Note: change ByteBuffer to CharBuffer in GenericUDFUtil.findText Hadoop Flags: Reviewed Status: Patch Available (was: Open) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better Key: HIVE-2722 URL: https://issues.apache.org/jira/browse/HIVE-2722 Project: Hive Issue Type: Improvement Components: UDF Affects Versions: 0.8.0 Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux java version 1.6.0_25 hadoop-0.20.2-cdh3u0 hive-0.7.0-cdh3u0 Reporter: caofangkun Priority: Minor Labels: udf Fix For: 0.9.0 Attachments: HIVE-2722.patch, udf_instr_1.q Original Estimate: 1h Remaining Estimate: 1h select instr(中文字符测试-第一行,-) from testTable limit 1; result:19 (one Chinese Character was considered as 3 Unicode bits) select substr(中文字符测试-第一行,1,2) from testTable limit 1; result: 中文 (one Chinese Character was considered as 1 Unicode Unit ) instr should considered one chinese character as one Unicode Unit too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2722) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
[ https://issues.apache.org/jira/browse/HIVE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caofangkun updated HIVE-2722: - Resolution: Fixed Status: Resolved (was: Patch Available) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better Key: HIVE-2722 URL: https://issues.apache.org/jira/browse/HIVE-2722 Project: Hive Issue Type: Improvement Components: UDF Affects Versions: 0.8.0 Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux java version 1.6.0_25 hadoop-0.20.2-cdh3u0 hive-0.7.0-cdh3u0 Reporter: caofangkun Priority: Minor Labels: udf Fix For: 0.9.0 Attachments: HIVE-2722.patch, udf_instr_1.q Original Estimate: 1h Remaining Estimate: 1h select instr(中文字符测试-第一行,-) from testTable limit 1; result:19 (one Chinese Character was considered as 3 Unicode bits) select substr(中文字符测试-第一行,1,2) from testTable limit 1; result: 中文 (one Chinese Character was considered as 1 Unicode Unit ) instr should considered one chinese character as one Unicode Unit too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2722) GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
[
https://issues.apache.org/jira/browse/HIVE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13187398#comment-13187398
]
caofangkun commented on HIVE-2722:
--
hive -e SELECT instr('ab中国c字d', '国') from dual limit 1
Result : 4
hive -e select instr('ab中国c字d', '字') from dual limit 1
Result: 6
GenericUDFUtils.findText use CharBuffer other than ByteBuffer will be better
Key: HIVE-2722
URL: https://issues.apache.org/jira/browse/HIVE-2722
Project: Hive
Issue Type: Improvement
Components: UDF
Affects Versions: 0.8.0
Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP
Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux
java version 1.6.0_25
hadoop-0.20.2-cdh3u0
hive-0.7.0-cdh3u0
Reporter: caofangkun
Priority: Minor
Labels: udf
Fix For: 0.9.0
Attachments: HIVE-2722.patch, udf_instr_1.q
Original Estimate: 1h
Remaining Estimate: 1h
select instr(中文字符测试-第一行,-) from testTable limit 1;
result:19 (one Chinese Character was considered as 3 Unicode bits)
select substr(中文字符测试-第一行,1,2) from testTable limit 1;
result: 中文 (one Chinese Character was considered as 1 Unicode Unit )
instr should considered one chinese character as one Unicode Unit too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2723) should throw Ambiguous column reference key Exception in particular join condition
should throw Ambiguous column reference key Exception in particular join condition -- Key: HIVE-2723 URL: https://issues.apache.org/jira/browse/HIVE-2723 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.8.0 Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux java version 1.6.0_25 hadoop-0.20.2-cdh3u0 hive-0.7.0-cdh3u0 Reporter: caofangkun Priority: Minor Fix For: 0.9.0 This Bug can be Repeated as following : create table test(key string, value string); create table test1(key string, value string); 1: Correct! select t.key from (select a.key, b.key from (select * from src ) a right outer join (select * from src1) b on (a.key = b.key)) t; FAILED: Error in semantic analysis: Ambiguous column reference key 2: Uncorrect!! Should throw Exception as above too! select t.key --Is this a.key or b.key ? It's ambiduous! from (select a.*, b.* from (select * from src ) a right outer join (select * from src1) b on (a.value = b.value)) t; Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks not specified. Defaulting to jobconf value of: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=number In order to limit the maximum number of reducers: set hive.exec.reducers.max=number In order to set a constant number of reducers: set mapred.reduce.tasks=number Starting Job = job_201201170959_0004, Tracking URL = http://zongren-VirtualBox:50030/jobdetails.jsp?jobid=job_201201170959_0004 Kill Command = /home/zongren/workspace/hadoop-adh/bin/hadoop job -Dmapred.job.tracker=zongren-VirtualBox:9001 -kill job_201201170959_0004 Hadoop job information for Stage-1: number of mappers: 2; number of reducers: 1 2012-01-17 11:02:47,507 Stage-1 map = 0%, reduce = 0% 2012-01-17 11:02:55,002 Stage-1 map = 100%, reduce = 0% 2012-01-17 11:03:04,240 Stage-1 map = 100%, reduce = 33% 2012-01-17 11:03:05,258 Stage-1 map = 100%, reduce = 100% Ended Job = job_201201170959_0004 MapReduce Jobs Launched: Job 0: Map: 2 Reduce: 1 HDFS Read: 669 HDFS Write: 216 SUCESS Total MapReduce CPU Time Spent: 0 msec OK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2723) should throw Ambiguous column reference key Exception in particular join condition
[ https://issues.apache.org/jira/browse/HIVE-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caofangkun updated HIVE-2723: - Description: This Bug can be Repeated as following : create table test(key string, value string); create table test1(key string, value string); 1: Correct! select t.key from (select a.key, b.key from (select * from src ) a right outer join (select * from src1) b on (a.key = b.key)) t; FAILED: Error in semantic analysis: Ambiguous column reference key 2: Uncorrect!! Should throw Exception as above too! select t.key --Is this a.key or b.key ? It's ambiduous! from (select a.\*, b.\* from (select * from src ) a right outer join (select * from src1) b on (a.value = b.value)) t; Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks not specified. Defaulting to jobconf value of: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=number In order to limit the maximum number of reducers: set hive.exec.reducers.max=number In order to set a constant number of reducers: set mapred.reduce.tasks=number Starting Job = job_201201170959_0004, Tracking URL = http://zongren-VirtualBox:50030/jobdetails.jsp?jobid=job_201201170959_0004 Kill Command = /home/zongren/workspace/hadoop-adh/bin/hadoop job -Dmapred.job.tracker=zongren-VirtualBox:9001 -kill job_201201170959_0004 Hadoop job information for Stage-1: number of mappers: 2; number of reducers: 1 2012-01-17 11:02:47,507 Stage-1 map = 0%, reduce = 0% 2012-01-17 11:02:55,002 Stage-1 map = 100%, reduce = 0% 2012-01-17 11:03:04,240 Stage-1 map = 100%, reduce = 33% 2012-01-17 11:03:05,258 Stage-1 map = 100%, reduce = 100% Ended Job = job_201201170959_0004 MapReduce Jobs Launched: Job 0: Map: 2 Reduce: 1 HDFS Read: 669 HDFS Write: 216 SUCESS Total MapReduce CPU Time Spent: 0 msec OK was: This Bug can be Repeated as following : create table test(key string, value string); create table test1(key string, value string); 1: Correct! select t.key from (select a.key, b.key from (select * from src ) a right outer join (select * from src1) b on (a.key = b.key)) t; FAILED: Error in semantic analysis: Ambiguous column reference key 2: Uncorrect!! Should throw Exception as above too! select t.key --Is this a.key or b.key ? It's ambiduous! from (select a.*, b.* from (select * from src ) a right outer join (select * from src1) b on (a.value = b.value)) t; Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks not specified. Defaulting to jobconf value of: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=number In order to limit the maximum number of reducers: set hive.exec.reducers.max=number In order to set a constant number of reducers: set mapred.reduce.tasks=number Starting Job = job_201201170959_0004, Tracking URL = http://zongren-VirtualBox:50030/jobdetails.jsp?jobid=job_201201170959_0004 Kill Command = /home/zongren/workspace/hadoop-adh/bin/hadoop job -Dmapred.job.tracker=zongren-VirtualBox:9001 -kill job_201201170959_0004 Hadoop job information for Stage-1: number of mappers: 2; number of reducers: 1 2012-01-17 11:02:47,507 Stage-1 map = 0%, reduce = 0% 2012-01-17 11:02:55,002 Stage-1 map = 100%, reduce = 0% 2012-01-17 11:03:04,240 Stage-1 map = 100%, reduce = 33% 2012-01-17 11:03:05,258 Stage-1 map = 100%, reduce = 100% Ended Job = job_201201170959_0004 MapReduce Jobs Launched: Job 0: Map: 2 Reduce: 1 HDFS Read: 669 HDFS Write: 216 SUCESS Total MapReduce CPU Time Spent: 0 msec OK should throw Ambiguous column reference key Exception in particular join condition -- Key: HIVE-2723 URL: https://issues.apache.org/jira/browse/HIVE-2723 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.8.0 Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux java version 1.6.0_25 hadoop-0.20.2-cdh3u0 hive-0.7.0-cdh3u0 Reporter: caofangkun Priority: Minor Labels: exception-handling, query, queryparser Fix For: 0.9.0 This Bug can be Repeated as following : create table test(key string, value string); create table test1(key string, value string); 1: Correct! select t.key from (select a.key, b.key from (select * from src ) a right outer join (select * from src1) b on (a.key = b.key)) t; FAILED: Error in semantic analysis: Ambiguous column reference key 2: Uncorrect!! Should throw Exception as above too! select t.key --Is this a.key or b.key ? It's ambiduous! from (select a.\*, b.\* from (select * from src ) a right outer join (select * from src1) b on (a.value =
[jira] [Commented] (HIVE-2656) Add a command line flag to specify a jar whence to load UDFs
[ https://issues.apache.org/jira/browse/HIVE-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13187507#comment-13187507 ] Phabricator commented on HIVE-2656: --- jonchang has commented on the revision HIVE-2656 [jira] Add a command line flag to specify a jar whence to load UDFs. Can you send me a link to the failures? REVISION DETAIL https://reviews.facebook.net/D1023 Add a command line flag to specify a jar whence to load UDFs Key: HIVE-2656 URL: https://issues.apache.org/jira/browse/HIVE-2656 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang Attachments: HIVE-2656.D1023.1.patch, HIVE-2656.D1023.2.patch To extend the functionality of Hive, it is useful to be able to easily deploy packages of UDFs and register the functions therein. Right now it is a bit cumbersome to do ADD JAR and CREATE TEMPORARY FUNCTION. This can be greatly simplified by adding a simple command line flag that will leverage the code in HIVE-2523 to load all the functions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
