[jira] [Created] (HIVE-2288) Adding the oracle nvl function to the UDF
Adding the oracle nvl function to the UDF - Key: HIVE-2288 URL: https://issues.apache.org/jira/browse/HIVE-2288 Project: Hive Issue Type: New Feature Components: UDF Reporter: Guy Doulberg Priority: Minor It would be nice if we could use the nvl function, described at oracle: http://www.techonthenet.com/oracle/functions/nvl.php -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-649) [UDF] now() for getting current time
[ https://issues.apache.org/jira/browse/HIVE-649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066604#comment-13066604 ] Guy Doulberg commented on HIVE-649: --- Hey, I can think of a problem with the current suggestion. Consider this scenario: A job is running on two days (start at 23:59 and ends at 00:03 ), in that case we can't know what will be the value of the now. will it be the start time or the end time. Can we set the value of now before the job is submitted? and in the UDF use that value? [UDF] now() for getting current time Key: HIVE-649 URL: https://issues.apache.org/jira/browse/HIVE-649 Project: Hive Issue Type: New Feature Components: UDF Reporter: Min Zhou Attachments: HIVE-649.patch http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html#function_now -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-trunk-h0.21 #831
See https://builds.apache.org/job/Hive-trunk-h0.21/831/ -- [...truncated 33256 lines...] [artifact:deploy] Deploying to https://repository.apache.org/content/repositories/snapshots [artifact:deploy] [INFO] Retrieving previous build number from apache.snapshots.https [artifact:deploy] Uploading: org/apache/hive/hive-hbase-handler/0.8.0-SNAPSHOT/hive-hbase-handler-0.8.0-20110717.192803-36.jar to repository apache.snapshots.https at https://repository.apache.org/content/repositories/snapshots [artifact:deploy] Transferring 49K from apache.snapshots.https [artifact:deploy] Uploaded 49K [artifact:deploy] [INFO] Uploading project information for hive-hbase-handler 0.8.0-20110717.192803-36 [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot org.apache.hive:hive-hbase-handler:0.8.0-SNAPSHOT' [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact org.apache.hive:hive-hbase-handler' ivy-init-dirs: ivy-download: [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar [get] To: /x1/jenkins/jenkins-slave/workspace/Hive-trunk-h0.21/hive/build/ivy/lib/ivy-2.1.0.jar [get] Not modified - so not downloaded ivy-probe-antlib: ivy-init-antlib: ivy-init: ivy-resolve-maven-ant-tasks: [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-trunk-h0.21/hive/ivy/ivysettings.xml ivy-retrieve-maven-ant-tasks: [ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead [ivy:cachepath] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-trunk-h0.21/hive/ivy/ivysettings.xml mvn-taskdef: maven-publish-artifact: [artifact:install-provider] Installing provider: org.apache.maven.wagon:wagon-http:jar:1.0-beta-2:runtime [artifact:deploy] Deploying to https://repository.apache.org/content/repositories/snapshots [artifact:deploy] [INFO] Retrieving previous build number from apache.snapshots.https [artifact:deploy] Uploading: org/apache/hive/hive-hwi/0.8.0-SNAPSHOT/hive-hwi-0.8.0-20110717.192804-36.jar to repository apache.snapshots.https at https://repository.apache.org/content/repositories/snapshots [artifact:deploy] Transferring 23K from apache.snapshots.https [artifact:deploy] Uploaded 23K [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot org.apache.hive:hive-hwi:0.8.0-SNAPSHOT' [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact org.apache.hive:hive-hwi' [artifact:deploy] [INFO] Uploading project information for hive-hwi 0.8.0-20110717.192804-36 ivy-init-dirs: ivy-download: [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar [get] To: /x1/jenkins/jenkins-slave/workspace/Hive-trunk-h0.21/hive/build/ivy/lib/ivy-2.1.0.jar [get] Not modified - so not downloaded ivy-probe-antlib: ivy-init-antlib: ivy-init: ivy-resolve-maven-ant-tasks: [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-trunk-h0.21/hive/ivy/ivysettings.xml ivy-retrieve-maven-ant-tasks: [ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead [ivy:cachepath] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-trunk-h0.21/hive/ivy/ivysettings.xml mvn-taskdef: maven-publish-artifact: [artifact:install-provider] Installing provider: org.apache.maven.wagon:wagon-http:jar:1.0-beta-2:runtime [artifact:deploy] Deploying to https://repository.apache.org/content/repositories/snapshots [artifact:deploy] [INFO] Retrieving previous build number from apache.snapshots.https [artifact:deploy] Uploading: org/apache/hive/hive-jdbc/0.8.0-SNAPSHOT/hive-jdbc-0.8.0-20110717.192806-36.jar to repository apache.snapshots.https at https://repository.apache.org/content/repositories/snapshots [artifact:deploy] Transferring 56K from apache.snapshots.https [artifact:deploy] Uploaded 56K [artifact:deploy] [INFO] Uploading project information for hive-jdbc 0.8.0-20110717.192806-36 [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot org.apache.hive:hive-jdbc:0.8.0-SNAPSHOT' [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact org.apache.hive:hive-jdbc' ivy-init-dirs: ivy-download: [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar [get] To:
[jira] [Updated] (HIVE-2247) ALTER TABLE RENAME PARTITION
[ https://issues.apache.org/jira/browse/HIVE-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiyan Wang updated HIVE-2247: -- Attachment: HIVE-2247.4.patch.txt Refactor the code, rename the old partition data directory to new partition data directory. ALTER TABLE RENAME PARTITION Key: HIVE-2247 URL: https://issues.apache.org/jira/browse/HIVE-2247 Project: Hive Issue Type: New Feature Reporter: Siying Dong Assignee: Weiyan Wang Attachments: HIVE-2247.3.patch.txt, HIVE-2247.4.patch.txt We need a ALTER TABLE TABLE RENAME PARTITIONfunction that is similar t ALTER TABLE RENAME. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2247) ALTER TABLE RENAME PARTITION
[ https://issues.apache.org/jira/browse/HIVE-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066756#comment-13066756 ] jirapos...@reviews.apache.org commented on HIVE-2247: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1105/ --- (Updated 2011-07-18 00:10:53.957462) Review request for Siying Dong. Changes --- Change thrift interface to void rename_partition(1:string dbname, 2:string name, 3:liststring part_vals, 4:liststring newpart_vals) Rename the data directory of old partition to that of new partition and change location/name parameter of partition metadata use Alter_partition to update the partition metadata Summary --- Implement ALTER TABLE PARTITION RENAME function to rename a partition. Add HiveQL syntax ALTER TABLE bar PARTITION (k1='v1', k2='v2') RENAME TO PARTITION (k1='v3', k2='v4'); This is my first Hive diff, I just learn everything from existing codebase and may not have a good understanding on it. Feel free to inform me if I make something wrong. Thanks This addresses bug HIVE-2247. https://issues.apache.org/jira/browse/HIVE-2247 Diffs (updated) - trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1145366 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1145366 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1145366 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 1145366 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1145366 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1145366 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1145366 trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1145366 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1145366 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1145366 trunk/ql/src/test/queries/clientnegative/alter_rename_partition_failure.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/alter_rename_partition_failure2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/alter_rename_partition_failure3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/alter_rename_partition.q PRE-CREATION trunk/ql/src/test/results/clientnegative/alter_rename_partition_failure.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/alter_rename_partition_failure2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/alter_rename_partition_failure3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/alter_rename_partition.q.out PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/RenamePartitionDesc.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableDesc.java 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1145366 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1145366 trunk/metastore/if/hive_metastore.thrift 1145366 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1145366 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1145366 trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1145366 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1145366 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1145366 Diff: https://reviews.apache.org/r/1105/diff Testing --- Add a partition A in the table Rename partition A to partition B Show the partitions in the table, it returns partition B. SELECT the data from partition A, it returns no results SELECT the data from partition B, it returns the data originally stored in partition A Thanks, Weiyan ALTER TABLE RENAME PARTITION Key: HIVE-2247 URL: https://issues.apache.org/jira/browse/HIVE-2247 Project: Hive Issue Type: New Feature Reporter: Siying Dong Assignee: Weiyan Wang Attachments: HIVE-2247.3.patch.txt, HIVE-2247.4.patch.txt We need a ALTER TABLE TABLE RENAME PARTITIONfunction that
[jira] [Commented] (HIVE-2247) ALTER TABLE RENAME PARTITION
[
https://issues.apache.org/jira/browse/HIVE-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066757#comment-13066757
]
jirapos...@reviews.apache.org commented on HIVE-2247:
-
bq. On 2011-07-13 23:58:21, Siying Dong wrote:
bq. trunk/metastore/if/hive_metastore.thrift, line 272
bq. https://reviews.apache.org/r/1105/diff/1/?file=25058#file25058line272
bq.
bq. Do we really need a new metastore function? Can we just use
alter_partition()?
alter_partition() just updates the partition metadata and neither check whether
the new partition exists nor do physical rename of data directory. It's
basically used by alter_table when there is no data in the old partition.
bq. On 2011-07-13 23:58:21, Siying Dong wrote:
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java,
line 1704
bq. https://reviews.apache.org/r/1105/diff/1/?file=25075#file25075line1704
bq.
bq. why we still continue when tab is null?
Fixed, throw an exception when tab is null.
bq. On 2011-07-13 23:58:21, Siying Dong wrote:
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java,
line 39
bq. https://reviews.apache.org/r/1105/diff/1/?file=25080#file25080line39
bq.
bq. Privilege.ALTER_METADATA doesn't sound right to me here. Can you use
Privilede.DROP and Priviledge.CREATE?
Change it to ALTERTABLE_RENAMEPART(ALTERTABLE_RENAMEPART, new
Privilege[]{Privilege.DROP}, new Privilege[]{Privilege.CREATE})
Not sure if it sounds right to you.
bq. On 2011-07-13 23:58:21, Siying Dong wrote:
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/RenamePartitionDesc.java, line
1
bq. https://reviews.apache.org/r/1105/diff/1/?file=25081#file25081line1
bq.
bq. Looks like you copied from the file AddPartitionDesc, but do you
need all those class members for renaming partition?
Only keep necessary functions
- Weiyan
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1105/#review1052
---
On 2011-07-18 00:10:53, Weiyan Wang wrote:
bq.
bq. ---
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1105/
bq. ---
bq.
bq. (Updated 2011-07-18 00:10:53)
bq.
bq.
bq. Review request for Siying Dong.
bq.
bq.
bq. Summary
bq. ---
bq.
bq. Implement ALTER TABLE PARTITION RENAME function to rename a partition.
bq. Add HiveQL syntax ALTER TABLE bar PARTITION (k1='v1', k2='v2') RENAME TO
PARTITION (k1='v3', k2='v4');
bq. This is my first Hive diff, I just learn everything from existing codebase
and may not have a good understanding on it.
bq. Feel free to inform me if I make something wrong. Thanks
bq.
bq.
bq. This addresses bug HIVE-2247.
bq. https://issues.apache.org/jira/browse/HIVE-2247
bq.
bq.
bq. Diffs
bq. -
bq.
bq.
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
1145366
bq.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
1145366
bq.trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java
1145366
bq.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
1145366
bq.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
1145366
bq.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
1145366
bq.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
1145366
bq.trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1145366
bq.
trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote
1145366
bq.
trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
1145366
bq.
trunk/ql/src/test/queries/clientnegative/alter_rename_partition_failure.q
PRE-CREATION
bq.
trunk/ql/src/test/queries/clientnegative/alter_rename_partition_failure2.q
PRE-CREATION
bq.
trunk/ql/src/test/queries/clientnegative/alter_rename_partition_failure3.q
PRE-CREATION
bq.trunk/ql/src/test/queries/clientpositive/alter_rename_partition.q
PRE-CREATION
bq.
trunk/ql/src/test/results/clientnegative/alter_rename_partition_failure.q.out
PRE-CREATION
bq.
trunk/ql/src/test/results/clientnegative/alter_rename_partition_failure2.q.out
PRE-CREATION
bq.
trunk/ql/src/test/results/clientnegative/alter_rename_partition_failure3.q.out
PRE-CREATION
bq.trunk/ql/src/test/results/clientpositive/alter_rename_partition.q.out
PRE-CREATION
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/RenamePartitionDesc.java
PRE-CREATION
bq.
number of maptasks of hive
Does hive split input table file according the ways hadoop provides which a file is splitted by block size? Then the number of map tasks is decided by the split size. But I have a table file which size is 196MB,my hdfs block size is 64MB,but I only see 3 map tasks in my web interface.What is the reason?
