[jira] [Created] (HIVE-2288) Adding the oracle nvl function to the UDF
Adding the oracle nvl function to the UDF - Key: HIVE-2288 URL: https://issues.apache.org/jira/browse/HIVE-2288 Project: Hive Issue Type: New Feature Components: UDF Reporter: Guy Doulberg Priority: Minor It would be nice if we could use the nvl function, described at oracle: http://www.techonthenet.com/oracle/functions/nvl.php -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-649) [UDF] now() for getting current time
[ https://issues.apache.org/jira/browse/HIVE-649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066604#comment-13066604 ] Guy Doulberg commented on HIVE-649: --- Hey, I can think of a problem with the current suggestion. Consider this scenario: A job is running on two days (start at 23:59 and ends at 00:03 ), in that case we can't know what will be the value of the now. will it be the start time or the end time. Can we set the value of now before the job is submitted? and in the UDF use that value? [UDF] now() for getting current time Key: HIVE-649 URL: https://issues.apache.org/jira/browse/HIVE-649 Project: Hive Issue Type: New Feature Components: UDF Reporter: Min Zhou Attachments: HIVE-649.patch http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html#function_now -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-trunk-h0.21 #831
See https://builds.apache.org/job/Hive-trunk-h0.21/831/ -- [...truncated 33256 lines...] [artifact:deploy] Deploying to https://repository.apache.org/content/repositories/snapshots [artifact:deploy] [INFO] Retrieving previous build number from apache.snapshots.https [artifact:deploy] Uploading: org/apache/hive/hive-hbase-handler/0.8.0-SNAPSHOT/hive-hbase-handler-0.8.0-20110717.192803-36.jar to repository apache.snapshots.https at https://repository.apache.org/content/repositories/snapshots [artifact:deploy] Transferring 49K from apache.snapshots.https [artifact:deploy] Uploaded 49K [artifact:deploy] [INFO] Uploading project information for hive-hbase-handler 0.8.0-20110717.192803-36 [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot org.apache.hive:hive-hbase-handler:0.8.0-SNAPSHOT' [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact org.apache.hive:hive-hbase-handler' ivy-init-dirs: ivy-download: [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar [get] To: /x1/jenkins/jenkins-slave/workspace/Hive-trunk-h0.21/hive/build/ivy/lib/ivy-2.1.0.jar [get] Not modified - so not downloaded ivy-probe-antlib: ivy-init-antlib: ivy-init: ivy-resolve-maven-ant-tasks: [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-trunk-h0.21/hive/ivy/ivysettings.xml ivy-retrieve-maven-ant-tasks: [ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead [ivy:cachepath] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-trunk-h0.21/hive/ivy/ivysettings.xml mvn-taskdef: maven-publish-artifact: [artifact:install-provider] Installing provider: org.apache.maven.wagon:wagon-http:jar:1.0-beta-2:runtime [artifact:deploy] Deploying to https://repository.apache.org/content/repositories/snapshots [artifact:deploy] [INFO] Retrieving previous build number from apache.snapshots.https [artifact:deploy] Uploading: org/apache/hive/hive-hwi/0.8.0-SNAPSHOT/hive-hwi-0.8.0-20110717.192804-36.jar to repository apache.snapshots.https at https://repository.apache.org/content/repositories/snapshots [artifact:deploy] Transferring 23K from apache.snapshots.https [artifact:deploy] Uploaded 23K [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot org.apache.hive:hive-hwi:0.8.0-SNAPSHOT' [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact org.apache.hive:hive-hwi' [artifact:deploy] [INFO] Uploading project information for hive-hwi 0.8.0-20110717.192804-36 ivy-init-dirs: ivy-download: [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar [get] To: /x1/jenkins/jenkins-slave/workspace/Hive-trunk-h0.21/hive/build/ivy/lib/ivy-2.1.0.jar [get] Not modified - so not downloaded ivy-probe-antlib: ivy-init-antlib: ivy-init: ivy-resolve-maven-ant-tasks: [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-trunk-h0.21/hive/ivy/ivysettings.xml ivy-retrieve-maven-ant-tasks: [ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead [ivy:cachepath] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-trunk-h0.21/hive/ivy/ivysettings.xml mvn-taskdef: maven-publish-artifact: [artifact:install-provider] Installing provider: org.apache.maven.wagon:wagon-http:jar:1.0-beta-2:runtime [artifact:deploy] Deploying to https://repository.apache.org/content/repositories/snapshots [artifact:deploy] [INFO] Retrieving previous build number from apache.snapshots.https [artifact:deploy] Uploading: org/apache/hive/hive-jdbc/0.8.0-SNAPSHOT/hive-jdbc-0.8.0-20110717.192806-36.jar to repository apache.snapshots.https at https://repository.apache.org/content/repositories/snapshots [artifact:deploy] Transferring 56K from apache.snapshots.https [artifact:deploy] Uploaded 56K [artifact:deploy] [INFO] Uploading project information for hive-jdbc 0.8.0-20110717.192806-36 [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot org.apache.hive:hive-jdbc:0.8.0-SNAPSHOT' [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact org.apache.hive:hive-jdbc' ivy-init-dirs: ivy-download: [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar [get] To:
[jira] [Updated] (HIVE-2247) ALTER TABLE RENAME PARTITION
[ https://issues.apache.org/jira/browse/HIVE-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiyan Wang updated HIVE-2247: -- Attachment: HIVE-2247.4.patch.txt Refactor the code, rename the old partition data directory to new partition data directory. ALTER TABLE RENAME PARTITION Key: HIVE-2247 URL: https://issues.apache.org/jira/browse/HIVE-2247 Project: Hive Issue Type: New Feature Reporter: Siying Dong Assignee: Weiyan Wang Attachments: HIVE-2247.3.patch.txt, HIVE-2247.4.patch.txt We need a ALTER TABLE TABLE RENAME PARTITIONfunction that is similar t ALTER TABLE RENAME. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2247) ALTER TABLE RENAME PARTITION
[ https://issues.apache.org/jira/browse/HIVE-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066756#comment-13066756 ] jirapos...@reviews.apache.org commented on HIVE-2247: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1105/ --- (Updated 2011-07-18 00:10:53.957462) Review request for Siying Dong. Changes --- Change thrift interface to void rename_partition(1:string dbname, 2:string name, 3:liststring part_vals, 4:liststring newpart_vals) Rename the data directory of old partition to that of new partition and change location/name parameter of partition metadata use Alter_partition to update the partition metadata Summary --- Implement ALTER TABLE PARTITION RENAME function to rename a partition. Add HiveQL syntax ALTER TABLE bar PARTITION (k1='v1', k2='v2') RENAME TO PARTITION (k1='v3', k2='v4'); This is my first Hive diff, I just learn everything from existing codebase and may not have a good understanding on it. Feel free to inform me if I make something wrong. Thanks This addresses bug HIVE-2247. https://issues.apache.org/jira/browse/HIVE-2247 Diffs (updated) - trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1145366 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1145366 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1145366 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 1145366 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1145366 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1145366 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1145366 trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1145366 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1145366 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1145366 trunk/ql/src/test/queries/clientnegative/alter_rename_partition_failure.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/alter_rename_partition_failure2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/alter_rename_partition_failure3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/alter_rename_partition.q PRE-CREATION trunk/ql/src/test/results/clientnegative/alter_rename_partition_failure.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/alter_rename_partition_failure2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/alter_rename_partition_failure3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/alter_rename_partition.q.out PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/RenamePartitionDesc.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableDesc.java 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1145366 trunk/metastore/if/hive_metastore.thrift 1145366 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1145366 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1145366 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1145366 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1145366 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1145366 Diff: https://reviews.apache.org/r/1105/diff Testing --- Add a partition A in the table Rename partition A to partition B Show the partitions in the table, it returns partition B. SELECT the data from partition A, it returns no results SELECT the data from partition B, it returns the data originally stored in partition A Thanks, Weiyan ALTER TABLE RENAME PARTITION Key: HIVE-2247 URL: https://issues.apache.org/jira/browse/HIVE-2247 Project: Hive Issue Type: New Feature Reporter: Siying Dong Assignee: Weiyan Wang Attachments: HIVE-2247.3.patch.txt, HIVE-2247.4.patch.txt We need a ALTER TABLE TABLE RENAME PARTITIONfunction that
[jira] [Commented] (HIVE-2247) ALTER TABLE RENAME PARTITION
[ https://issues.apache.org/jira/browse/HIVE-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066757#comment-13066757 ] jirapos...@reviews.apache.org commented on HIVE-2247: - bq. On 2011-07-13 23:58:21, Siying Dong wrote: bq. trunk/metastore/if/hive_metastore.thrift, line 272 bq. https://reviews.apache.org/r/1105/diff/1/?file=25058#file25058line272 bq. bq. Do we really need a new metastore function? Can we just use alter_partition()? alter_partition() just updates the partition metadata and neither check whether the new partition exists nor do physical rename of data directory. It's basically used by alter_table when there is no data in the old partition. bq. On 2011-07-13 23:58:21, Siying Dong wrote: bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java, line 1704 bq. https://reviews.apache.org/r/1105/diff/1/?file=25075#file25075line1704 bq. bq. why we still continue when tab is null? Fixed, throw an exception when tab is null. bq. On 2011-07-13 23:58:21, Siying Dong wrote: bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java, line 39 bq. https://reviews.apache.org/r/1105/diff/1/?file=25080#file25080line39 bq. bq. Privilege.ALTER_METADATA doesn't sound right to me here. Can you use Privilede.DROP and Priviledge.CREATE? Change it to ALTERTABLE_RENAMEPART(ALTERTABLE_RENAMEPART, new Privilege[]{Privilege.DROP}, new Privilege[]{Privilege.CREATE}) Not sure if it sounds right to you. bq. On 2011-07-13 23:58:21, Siying Dong wrote: bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/RenamePartitionDesc.java, line 1 bq. https://reviews.apache.org/r/1105/diff/1/?file=25081#file25081line1 bq. bq. Looks like you copied from the file AddPartitionDesc, but do you need all those class members for renaming partition? Only keep necessary functions - Weiyan --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1105/#review1052 --- On 2011-07-18 00:10:53, Weiyan Wang wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/1105/ bq. --- bq. bq. (Updated 2011-07-18 00:10:53) bq. bq. bq. Review request for Siying Dong. bq. bq. bq. Summary bq. --- bq. bq. Implement ALTER TABLE PARTITION RENAME function to rename a partition. bq. Add HiveQL syntax ALTER TABLE bar PARTITION (k1='v1', k2='v2') RENAME TO PARTITION (k1='v3', k2='v4'); bq. This is my first Hive diff, I just learn everything from existing codebase and may not have a good understanding on it. bq. Feel free to inform me if I make something wrong. Thanks bq. bq. bq. This addresses bug HIVE-2247. bq. https://issues.apache.org/jira/browse/HIVE-2247 bq. bq. bq. Diffs bq. - bq. bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1145366 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1145366 bq.trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1145366 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 1145366 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1145366 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1145366 bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1145366 bq.trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1145366 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1145366 bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1145366 bq. trunk/ql/src/test/queries/clientnegative/alter_rename_partition_failure.q PRE-CREATION bq. trunk/ql/src/test/queries/clientnegative/alter_rename_partition_failure2.q PRE-CREATION bq. trunk/ql/src/test/queries/clientnegative/alter_rename_partition_failure3.q PRE-CREATION bq.trunk/ql/src/test/queries/clientpositive/alter_rename_partition.q PRE-CREATION bq. trunk/ql/src/test/results/clientnegative/alter_rename_partition_failure.q.out PRE-CREATION bq. trunk/ql/src/test/results/clientnegative/alter_rename_partition_failure2.q.out PRE-CREATION bq. trunk/ql/src/test/results/clientnegative/alter_rename_partition_failure3.q.out PRE-CREATION bq.trunk/ql/src/test/results/clientpositive/alter_rename_partition.q.out PRE-CREATION bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/RenamePartitionDesc.java PRE-CREATION bq.
number of maptasks of hive
Does hive split input table file according the ways hadoop provides which a file is splitted by block size? Then the number of map tasks is decided by the split size. But I have a table file which size is 196MB,my hdfs block size is 64MB,but I only see 3 map tasks in my web interface.What is the reason?